Skip to content

feat(training): support WSD and other transformers LR schedulers - #76

Open
Neonkraft wants to merge 1 commit into
mainfrom
feat/lr-schedulers
Open

Neonkraft wants to merge 1 commit into
mainfrom
feat/lr-schedulers

Conversation

@Neonkraft

Copy link
Copy Markdown
Collaborator

Summary

Adds the warmup-stable-decay (WSD) LR schedule and the other transformers schedulers that need kwargs.

TRL (1.7.0 and 1.17.1) passes lr_scheduler_type and lr_scheduler_kwargs to transformers.get_scheduler, so the schedules already existed upstream. The block was LRSchedulerKwargs: it had only min_lr_rate, so OmegaConf rejected all other keys.

  • LRSchedulerKwargs now has the kwargs of all supported schedulers. Only set fields are forwarded.
  • WSD num_decay_steps uses the warmup_steps rule: (0, 1) is a ratio of the training steps, >= 1 is a step count. A ratio is converted to steps before training and is not allowed with num_train_epochs.
  • The new src/post_training/utils/lr_scheduler.py validates the scheduler config in the submission phase. It rejects kwargs that the scheduler does not accept, values out of range, a missing min_lr / min_lr_rate, constant with a warmup, and a WSD warmup + decay that is longer than the run.
  • reduce_lr_on_plateau and greedy are not supported, because they need an evaluation loop.
  • The dataclass default lr_scheduler_type is now linear. The new validation rejects cosine_with_min_lr without min_lr or min_lr_rate, and LRSchedulerKwargs sets neither by default. With linear, a config without scheduler fields stays valid. All shipped configs set the scheduler explicitly, so they do not change.
  • The guardrails summary, the README, and configs/trl/{sft,dpo}.yaml show the new options.
lr_scheduler_type: "warmup_stable_decay"
lr_scheduler_kwargs:
  num_decay_steps: 0.2   # decay over the last 20% of steps
  decay_type: "1-sqrt"

Type of change

  • Bug fix
  • New feature
  • Refactor
  • Performance
  • Documentation
  • Maintenance

Validation

tests/test_lr_scheduler.py (36 tests) builds each scheduler through SFTConfig and Trainer.create_scheduler, checks the WSD LR curve and the ratio-to-steps conversion, and checks each rejection. The tests pass with TRL 1.7.0 and TRL 1.17.1. The pre-commit hooks pass. A real training run was not done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant