Chapter 17 — Domain Training and Affinity
Domains are labels you supply
Domains and subdomains organize probes by the questions you want to ask. They do not establish that the model has learned the same categories. Keep domain definitions stable when comparing runs and check coverage when one category contains fewer or less varied probes.
Separate the outcome sources
Trainer-reported train and holdout metrics come from your code. Probe
metrics come from IRIS™'s fixed inspection inputs. Answer-span probe
accuracy differs from whole-text token accuracy, and neither is
automatically an independently sampled held-out estimate. Name the source
when sharing a plot.
Training affinity
Positive measured gradient affinity means sampled domain gradients point in similar directions; negative affinity means opposition under that measurement. It does not establish that tasks always transfer or conflict, or that changing their sampling weights will improve training.
Read uncertainty intervals, sample coverage, and missing-data statements. An absent measurement is not zero affinity. Do not generalize a relationship beyond the probes, groups, frames, and measurement conditions it covers.
Representation comparisons
Domain spectra, overlap, composition, and subspace views describe different aspects of how labeled probes are represented. A domain map can use a chosen reference or a declared pooled reference; the reference is part of the measurement, not a neutral origin that every comparison shares.
Single-domain runs cannot support a between-domain comparison. Category views also need categories distinct from domains to add a separate reading. Read the stated applicability rather than treating a pooled fallback as a second independent result.
Form a testable question
Use an observed separation, overlap, or opposition to select examples and frames for closer review. Record any proposed interpretation as a hypothesis and test it against held-out behavior or a controlled experiment. The chart alone does not identify a semantic mechanism or a causal circuit.
Domain maps — Figures 4.18–4.20
These views compare domain organization using a polar map, a domain similarity matrix, and distance from the all-probe average by layer. Read the displayed frame, grouping, reference, and selected layers together. A change of reference or frame changes the comparison; a domain label is supplied by the probe curriculum, not an inferred semantic identity.
