feat: Add GDPO Support (#3353)

* gdpo support - test left * lint * fixxes for vllm serv * test advantages * docss * lint * lint = * gdpo simple + lint * lint nit * example * lint * trl 0.27.0 * blocklist * test assert rmv * add validation check for GDPO + sum_then_normalize --------- Co-authored-by: Wing Lian <wing@axolotl.ai>
2026-01-22 03:52:45 +05:30
parent 8623dd8a72
commit d0d26d5064
11 changed files with 742 additions and 6 deletions
--- a/requirements.txt
+++ b/requirements.txt
@@ -17,7 +17,7 @@ transformers==4.57.6
 accelerate==1.12.0
 datasets==4.5.0
 deepspeed>=0.18.3
-trl==0.25.1
+trl==0.27.0
 hf_xet==1.2.0
 kernels==0.11.5
 trackio>=0.13.0