sanity check ranges in freeze.py (#1686)

* sanity check ranges in freeze.py

this will catch problems earlier and more clearly.

in my case, it appears that deepspeed zero3 sets layer tensor shapes
to [0], which doesn't play well with automatically inferred ranges.
through a bit of luck, inverting ranges still appears to work correctly.

* simplify chained comparison
This commit is contained in:
Josh Bleecher Snyder
2024-07-05 06:24:07 -07:00
committed by GitHub
parent c69b7eb2b5
commit b3f680d305

View File

@@ -120,6 +120,9 @@ def _merge_ranges(
processed_ranges = [
(start, end if end is not None else layer_size) for start, end in given_ranges
]
for start, end in processed_ranges:
if start < 0 or end > layer_size > 0 or start >= end:
raise ValueError(f"invalid unfreeze range: start={start}, end={end}")
# No need to merge if there's only one or no ranges
if len(processed_ranges) <= 1: