-
60a11a6410
Use Latest Cut Cross Entropy (#2392)
xzuyn
2025-03-10 05:26:40 -04:00
-
46a045e528
chore(doc): add faq when having no default chat_template (#2398)
NanoCode012
2025-03-10 16:25:50 +07:00
-
3b477e08a0
feat(doc): add more info on RewardModel datasets (#2391)
NanoCode012
2025-03-10 16:25:31 +07:00
-
754817c8c6
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-07 13:59:52 +00:00
-
48846db8e3
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-07 13:59:21 +00:00
-
16dc6ee68d
refactor: trl grpo configs to have descriptions (#2386)
NanoCode012
2025-03-07 20:58:53 +07:00
-
fa7c79b3b9
remove lion-pytorch as it's already handled upstream (#2389)
Wing Lian
2025-03-07 08:58:15 -05:00
-
d8eb8703f3
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-06 16:50:20 +00:00
-
b6083ae7b1
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-06 16:49:40 +00:00
-
ae66374156
Optimizer refactor and add Muon support (#2367)
Wing Lian
2025-03-06 11:49:19 -05:00
-
5e21b1a9da
various fixes 20250305 (#2384)
Wing Lian
2025-03-06 11:48:44 -05:00
-
76bb09784d
fix import
optimizers-refactor
Wing Lian
2025-03-05 14:05:27 -05:00
-
92f0ed2e37
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-05 16:16:51 +00:00
-
575e5f28ec
Update Tokenizer Overrides Handling in models.py (#1549)
mhenrichsen
2025-03-05 17:15:12 +01:00
-
0542c7dd56
add muon optimizer
Wing Lian
2025-02-27 23:15:41 -05:00
-
-
-
3ee96165d3
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-05 15:27:11 +00:00
-
0134093acc
Add REX LR Scheduler (#2380)
xzuyn
2025-03-05 10:26:11 -05:00
-
e6453035a7
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-05 15:03:12 +00:00
-
d4de93a7bb
feat(grpo): add reward_weights config and refactor (#2365)
NanoCode012
2025-03-05 22:02:08 +07:00
-
c8191394e9
fix(doc): add missing low_cpu_mem_usage config to docs (#2369) [skip ci]
NanoCode012
2025-03-05 22:01:44 +07:00
-
f18231c653
chore(doc): add clarification about mpi4py error on single gpu deepspeed (#2383) [skip ci]
NanoCode012
2025-03-05 22:01:28 +07:00
-
9ed4f6b3aa
feat(doc): document drop_system_message and clarify limitation (#2381) [skip ci]
NanoCode012
2025-03-05 22:01:16 +07:00
-
05dddfc41d
feat(doc): add docker images explanation (#2379) [skip ci]
NanoCode012
2025-03-05 22:01:00 +07:00
-
8e30917440
chore(docs): remove phorm (#2378) [skip ci]
NanoCode012
2025-03-05 22:00:50 +07:00
-
d883b11b6f
fix(doc): add installation for cce to docs (#2375) [skip ci]
NanoCode012
2025-03-05 22:00:39 +07:00
-
9270933a28
Built site for gh-pages
Quarto GHA Workflow Runner
2025-03-05 13:59:40 +00:00
-
f4910dd2ea
train.py refactor (#2371)
Dan Saunders
2025-03-05 08:58:33 -05:00
-
5c0510a876
review comments
train-refactor
Dan Saunders
2025-03-03 18:44:16 +00:00
-
e1bc18763a
combine like functions
Dan Saunders
2025-02-28 17:47:39 +00:00
-
10d18e6c97
fix(test): replace jackfram llama with smollm
fix/replace_jackllama
NanoCode012
2025-02-28 16:40:49 +07:00
-
-
-
ee489d16bf
wip
seq-parallel-ring
Wing Lian
2025-02-27 11:42:46 -05:00
-
ed5178cd3d
update
Dan Saunders
2025-02-26 21:03:44 +00:00
-
a3224c7c3c
updates
Dan Saunders
2025-02-26 20:31:54 +00:00
-
c4104fc10c
refactor train.py
Dan Saunders
2025-02-26 19:37:42 +00:00
-
-
-
68e97d032a
chunk to prevent overflows in kernel
topk-logprobs-triton
Wing Lian
2025-02-26 04:44:24 -05:00
-
23f029a89c
lint and additional train metric checks for kd
Wing Lian
2025-02-26 03:19:42 -05:00
-
afbb44f08b
more optims
Wing Lian
2025-02-26 01:49:47 -05:00
-
d753ead033
optimize and include bench util
Wing Lian
2025-02-26 01:17:50 -05:00
-
f68aedd1f8
Update __init__.py
tp_support
Sung Ching Liu
2025-02-21 11:13:15 -05:00
-
3dd5c6f8ec
nit
Sunny Liu
2025-02-21 10:59:03 -05:00
-
4caa59a087
auto detect tp_size
Sunny Liu
2025-02-20 11:36:55 -05:00
-
984be14147
add tp_size in config doc
Sunny Liu
2025-02-20 00:01:59 -05:00
-
64adbf1a15
tp plan not needed
Sunny Liu
2025-02-19 19:18:36 -05:00
-
438b623031
prepare accelerate envs for tp
Sunny Liu
2025-02-18 23:34:05 -05:00
-
a74efcecbe
skip move to device
Sunny Liu
2025-02-18 17:13:52 -05:00
-
d663652216
del device_map for tp
Sunny Liu
2025-02-18 17:08:32 -05:00
-
dbd43aa18f
set tp_plan
Sunny Liu
2025-02-18 17:04:13 -05:00
-
dbdf97e828
enabe tp thru tp_size
Sunny Liu
2025-02-18 16:20:24 -05:00
-
-
-
c011405117
fix gradients
Wing Lian
2025-02-25 23:34:27 -05:00
-
f144319697
distributed fix
telemetry
Dan Saunders
2025-02-26 02:55:44 +00:00
-
07bb41812b
fix issue with tests in ci
Dan Saunders
2025-02-24 21:30:34 +00:00
-
cae8c7636b
fixes
Dan Saunders
2025-02-24 20:30:16 +00:00
-
09611cea10
remove duplicate info
Dan Saunders
2025-02-24 20:02:16 +00:00
-
61266ab843
adding runtime metrics / system info additional accelerator support, etc.
Dan Saunders
2025-02-24 19:37:11 +00:00
-
aea0e760e4
adding runtime metrics / system info additional accelerator support, etc.
Dan Saunders
2025-02-24 19:36:31 +00:00
-
5afad670da
improved redaction, send system info during model config load telemetry, etc.
Dan Saunders
2025-02-24 15:39:02 +00:00
-
49ac79ed1e
doc update
Dan Saunders
2025-02-24 01:49:31 +00:00
-
c9af72cd7a
fix
Dan Saunders
2025-02-24 01:31:35 +00:00
-
d3d63c1432
adding back in base_model redaction w/ whitelist
Dan Saunders
2025-02-24 01:16:03 +00:00
-
675b65d711
sleep on all ranks in distributed setting
Dan Saunders
2025-02-24 00:53:58 +00:00
-
b23187daea
simplifying path redaction
Dan Saunders
2025-02-24 00:06:08 +00:00
-
e373a6b8d0
small update / fix
Dan Saunders
2025-02-21 20:35:09 +00:00
-
fd5d5aecdc
tests for runtime metrics telemetry and assoc. callback
Dan Saunders
2025-02-21 20:31:07 +00:00
-
3760175440
adding runtime metrics (cpu + gpu memory, steps/s, etc.)
Dan Saunders
2025-02-21 19:01:35 +00:00
-
7927abff90
updated sanitization logic, tests
Dan Saunders
2025-02-24 20:05:55 +00:00
-
ec36839316
update error file path sanitization function; adding more error tracking
Dan Saunders
2025-02-21 13:57:08 +00:00
-
3076b8df00
progress on telemetry: config load, process, model load, train start / end, error tracking
Dan Saunders
2025-02-19 22:05:12 +00:00
-
c50610375f
updates
Dan Saunders
2025-02-19 13:55:04 +00:00
-
07ffd47f2b
updates
Dan Saunders
2025-02-19 13:54:51 +00:00
-
76d951afd2
adding todo
Dan Saunders
2025-02-17 22:25:54 +00:00
-
5220e8ccf4
initial telemetry manager impl
Dan Saunders
2025-02-17 18:31:42 +00:00
-
-
-
1a3f9bcbbe
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-25 18:51:14 +00:00
-
75cbd15301
Fix(doc): address missing doc changes (#2362)
v0.7.1
NanoCode012
2025-02-26 01:50:02 +07:00
-
a2e52a29e9
chore: lint
Wing Lian
2025-02-25 07:29:46 -05:00
-
efdea0e9ff
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-25 09:11:03 +00:00
-
2efe1b4c09
Feat(doc): Reorganize documentation, fix broken syntax, update notes (#2348)
NanoCode012
2025-02-25 16:09:37 +07:00
-
e82268e580
use triton for kd-loss in trainer
Wing Lian
2025-02-24 22:58:35 -05:00
-
75e1480c10
chunking not necessary
Wing Lian
2025-02-24 22:56:15 -05:00
-
45e1548d59
fix the kernels
Wing Lian
2025-02-24 22:38:55 -05:00
-
165088e7c1
triton kernel for top-k logprob kd
Wing Lian
2025-02-24 22:13:26 -05:00
-
4c1ebd7402
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-24 20:04:15 +00:00
-
1110a37e21
feat: add deepseek_v3 sample packing (#2230)
NanoCode012
2025-02-25 03:03:15 +07:00
-
7f31599da4
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-24 17:41:49 +00:00
-
9850f42204
bump liger to 0.5.3 (#2353)
Wing Lian
2025-02-24 12:40:54 -05:00
-
f897224306
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-24 16:13:58 +00:00
-
00fc8109e4
Correctly reference mount paths (#2347)
Matt Baker
2025-02-24 08:12:57 -08:00
-
-
-
e618244a70
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-23 17:32:31 +00:00
-
2d5826f544
Relicense the logprob KD loss functions as Apache 2.0 (#2358)
Wing Lian
2025-02-23 12:31:35 -05:00
-
d88e071120
sequence parallelism
Wing Lian
2025-02-23 12:19:34 -05:00
-
-
-
17fb239c63
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-22 03:07:26 +00:00
-
a4170030ab
don't install extraneous old version of pydantic in ci and make sre to run multigpu ci (#2355)
Wing Lian
2025-02-21 22:06:29 -05:00
-
328bb0466b
Merge branch 'main' into flx_attn_support
flx_attn_support
Sung Ching Liu
2025-02-21 11:27:25 -05:00
-
-
-
-
e792b54bab
remove unnecessary components
Sunny Liu
2025-02-21 11:23:21 -05:00
-
bf842730a5
fix(doc): add missing auto_find_batch_size (#2339) [skip ci]
NanoCode012
2025-02-21 11:56:38 +07:00
-
248a0b9f6f
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-21 03:57:38 +00:00
-
3d80a2df6d
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-21 03:57:03 +00:00
-
1db6ad60a7
support for passing init_lora_weights to lora_config (#2352)
Wing Lian
2025-02-20 22:56:34 -05:00
-
29b366b2e1
Bumping 0.15.1 TRL version for GRPO+PEFT fix (#2344)
salman
2025-02-21 03:56:04 +00:00
-
a9ebff087c
remove ref_model when peft model is passed into grpo trainer
grpo-ref-model-cleanup
Wing Lian
2025-02-20 21:53:20 -05:00
-
-
-
10518d9ec0
Built site for gh-pages
Quarto GHA Workflow Runner
2025-02-21 02:13:04 +00:00