4 Commits

Author	SHA1	Message	Date
Wing Lian	fc6188cd76	fix merge conflict of duplicate max_steps in config for relora (#2116)	2024-12-03 07:42:41 -05:00
NanoCode012	822c904092	fix(vlm): handle legacy conversation data format and check image in data (#2018) [skip ci] ... * fix: handle legacy conversation data format and check image in data * feat: add test for llama vision * feat: add max_steps to test * fix: incorrect indent and return preprocess * feat: use smaller model and dataset * chore: add extra config for sharegpt dataset	2024-12-03 00:01:31 -05:00
Wing Lian	ce5bcff750	various tests fixes for flakey tests (#2110) ... * add mhenrichsen/alpaca_2k_test with revision dataset download fixture for flaky tests * log slowest tests * pin pynvml==11.5.3 * fix load local hub path * optimize for speed w smaller models and val_set_size * replace pynvml * make the resume from checkpoint e2e faster * make tests smaller	2024-12-02 17:28:58 -05:00
Wing Lian	8c2e05ade3	relora: magnitude pruning of the optimizer (#1245) ... * magnitude pruning of the optimizer * add alpaca chat template and fix relora patch * fix handling of lora adapter for relora * fix merge and save call * fixes for 8-bit lora merge * save intermediate checkpoint adapters * auto merge * fix eval check * handle relora annealing * fix anneal step logic * chore: lint * misx fix * fix types * Update tests/e2e/test_relora_llama.py * check for safetensors saved from relora	2024-02-06 00:35:30 -05:00