Chapter 21 — Gradient Residuals, Stability, Curvature, and Update Efficiency
Register the optimizer
tracker.set_optimizer(optimizer)
Use the optimizer whose step actually runs. Registration enables the supported closure-free training-dynamics family, including available per-group motion shares and residual readings. It does not make every curvature measurement available.
Optional curvature callbacks
Some readings also need a scalar-loss callback and a fixed evaluation batch:
tracker = IrisTracker(
output_dir="/path/to/run",
spec_dict=my_spec,
probes_path="my_model.probes.json",
loss_fn=lambda model, batch: model(
batch[0], attention_mask=batch[1], labels=batch[0]
).loss,
eval_batch_fn=lambda: (eval_ids, eval_mask),
)
tracker.set_optimizer(optimizer)
This callback is an example for a compatible language-model forward; adapt
it to your model's real scalar objective. Supply callbacks before the first
extraction. set_loss_fn and set_eval_batch_fn are alternatives when called
before that boundary. The output-change efficiency ratio needs an evaluation
batch; not every diagnostic needs both callbacks.
Both callbacks must be side-effect-free. Do not consume the training data iterator, advance a scheduler or counter, or update application state inside them. Measurement can invoke a callback more often than the trainer does. Parameter-tensor hooks can also observe a measurement backward; review custom hooks when integrating.
Read the channel's qualification
Closure-gated landscape curvature differs from a closure-free path reading. The latter can be a lower-bound reading or an estimate, depending on the coverage stated by the run. Preserve the label rather than promoting an estimate to a bound or treating a restricted trajectory as the entire loss surface.
Stability-margin and CFL-style comparisons are directional observables. Under adaptive optimizers, raw landscape curvature is not the fully preconditioned optimizer geometry. A reading near a displayed boundary is not a calibrated guarantee of safety, convergence, or failure.
Residuals, motion, and efficiency
Per-layer/head/expert shares localize the recorded quantity; they do not assign causal credit for learning. Distinguish parameter motion, gradient size, representation motion, and output-change efficiency. A large value in one channel does not establish a corresponding change in another.
Read absent or uncalibrated states explicitly. Memory restrictions, missing callbacks, unsupported topology, or unavailable captures can limit the measurement. Never replace a missing residual or curvature value with zero.
