Chapter 19 — MoE Expert and Router Diagnostics
Experts and routing are different readings
On a supported mixture-of-experts capture, inspect both which experts receive traffic and what the recorded representation or motion readings show for those experts. Routing weight, effective dimensionality, and parameter motion answer different questions.
The expert indices are the model's own identities. A shared expert or dense path is labeled separately when the architecture has one. Do not infer that two nodes with a similar domain mix are the same expert, or that an expert belongs exclusively to a domain because that domain currently dominates its traffic.
Read shares with their denominator
A routing or weight-motion share is relative to the quantity and coverage named by the figure. It is not a loss contribution or a causal importance score. Read whether the figure covers all experts, a displayed subset, a particular domain, or a particular measured interval.
Missing and restricted captures
Per-expert views need supported expert/router captures. Some model wrappers need their integration to expose those captures; other architectures have no routed experts at all. The Bench distinguishes unavailable data from architectural non-applicability. Resource limits can also restrict which experts or frames a view covers.
Use the captions and availability notes before comparing experts across time or runs. A route not measured, a measured zero, and a link omitted from a drawing are different states. The Flow Map makes those distinctions particularly important.
Useful comparisons
Place changes in routing beside trainer-reported outcomes, domain readings, and run interventions. A stable accuracy curve does not imply that routing or internal structure is unchanged. Conversely, reorganization alone does not show that generalization improved or that a particular circuit formed.
