Learning Rate Calculator
Choosing How Fast a Model Learns to Slow Down
A learning rate that stays fixed for the entire training run almost never works well — too high and the model overshoots as it approaches a good solution, too low and training crawls. Learning rate schedules solve this by shrinking the rate over time following one of a handful of well-established curves. This calculator computes the rate at a given point in training for four of the most widely used schedules.
The Formulas
Exponential Decay: lr = initial_lr × e^(−decay_rate × epoch)
Time-Based Decay: lr = initial_lr / (1 + decay_rate × epoch)
Cosine Annealing: lr = min_lr + 0.5 × (initial_lr − min_lr) × (1 + cos(π × epoch / total_epochs))
How the Four Schedules Compare
| Schedule | Parameters used | Resulting rate at epoch 25 |
|---|---|---|
| Step decay | decay_rate = 0.9, decay_steps = 10 | 0.0810 |
| Exponential decay | decay_rate = 0.05 | 0.0287 |
| Time-based decay | decay_rate = 0.01 | 0.0800 |
| Cosine annealing | min_lr = 0, total_epochs = 100 | 0.0854 |
Step decay drops in sudden jumps at fixed intervals; exponential and time-based decay shrink continuously but at different rates; cosine annealing follows a smooth curve that decelerates near the start and end of training and falls fastest through the middle.
Where Each Schedule Is Used
- Step decay — common in classic computer vision training recipes, where the rate is cut by a fixed factor every N epochs.
- Exponential and time-based decay — smooth alternatives that avoid the sudden jumps of step decay, useful when a gentler taper is preferred.
- Cosine annealing — widely used in modern deep learning (often with warm restarts), since the smooth curve tends to produce better final convergence than abrupt drops.
How to Use This Calculator
- Select the schedule type: Step Decay, Exponential Decay, Time-Based Decay, or Cosine Annealing.
- Enter the Initial Learning Rate.
- Enter the schedule-specific inputs (Decay Rate, Decay Steps, Current Epoch, Minimum Learning Rate, or Total Epochs, depending on the mode selected).
- Select Calculate to see the learning rate at that point in training.
Related Calculations
Once a rate is set, see its effect on a single update with the Gradient Descent Step Calculator, or plan the full run with the Epoch Time Calculator.