sanity check ranges in freeze.py (#1686)
* sanity check ranges in freeze.py this will catch problems earlier and more clearly. in my case, it appears that deepspeed zero3 sets layer tensor shapes to [0], which doesn't play well with automatically inferred ranges. through a bit of luck, inverting ranges still appears to work correctly. * simplify chained comparison
This commit is contained in:
committed by
GitHub
parent
c69b7eb2b5
commit
b3f680d305
@@ -120,6 +120,9 @@ def _merge_ranges(
|
||||
processed_ranges = [
|
||||
(start, end if end is not None else layer_size) for start, end in given_ranges
|
||||
]
|
||||
for start, end in processed_ranges:
|
||||
if start < 0 or end > layer_size > 0 or start >= end:
|
||||
raise ValueError(f"invalid unfreeze range: start={start}, end={end}")
|
||||
|
||||
# No need to merge if there's only one or no ranges
|
||||
if len(processed_ranges) <= 1:
|
||||
|
||||
Reference in New Issue
Block a user