Wing Lian 0dac2ddeac Llama4 linearized (#2502) ...

* llama4 support for linearized experts

* clean up fsdp2 sharding to prevent hang

* add yaml config

* cleanup example [skip ci]

2025-04-07 20:47:00 -04:00

..

cerebras

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

cloud

native support for modal cloud from CLI (#2237)

2025-01-30 11:34:02 -05:00

code-llama

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

cohere

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

colab-notebooks

fix build w pyproject to respect insalled torch version (#2168)

2024-12-10 16:25:25 -05:00

dbrx

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

deepseek-v2

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

falcon

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

gemma

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

gemma2

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

gemma3

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

gptj

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

jamba

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

jeopardy-bot

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

llama-2

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

llama-3

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

llama-3-vision

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

llama-4

Llama4 linearized (#2502)

2025-04-07 20:47:00 -04:00

llava

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

mamba

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

mistral

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

mpt-7b

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

openllama-3b

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

phi

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

pixtral

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

pythia

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

pythia-12b

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

qwen

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

qwen2

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

qwen2-vl

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

redpajama

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

replit-3b

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

stablelm-2

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

starcoder2

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

tiny-llama

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

xgen-7b

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00

yi-34B-chat

simplify the example configs to be more minimal and less daunting (#2486) [skip ci]

2025-04-04 13:47:26 -04:00