Chapter 4 — Safe Interpretation and the First 15 Minutes
Reading IRIS™ Charts Safely
Read every chart with four questions:
- What does this chart measure? Start with the measurement, not the interpretation.
- Where is the strongest visible pattern? Look for concentration, movement, contrast, or sharp change.
- What investigation could this support? Turn the measurement into a research question.
- What should we not conclude? Avoid jumping from measurement to proof.
Example: IRIS measured negative affinity between arithmetic and grammar in Layer 4 Head 2. This may support inspection of domain interaction in that head. It does not prove that either domain is harmful or that training should be changed.
What to Inspect First
When opening a new IRIS report:
- Confirm the run. Start with §1 Run. Check model name, run ID, frame count, tracker status, resource overhead, and run state.
- Inspect model organization. Open §2 Concept Space. Look for domain footprints, separation or overlap, confident-wrong probes, unusual motion, and dots worth clicking.
- Review global structure. Open §5 Layers and §3 Whole model. Look for k95 differences by layer, spectral slope changes, rotation spikes and residual concentration (§5); motion and curvature over training (§3); and head or expert load concentration (§6, §7).
- Review domain behavior. Open §4 Domains. Look for domain-specific learning patterns, affinity or opposition, k95 differences, stability changes, and residual concentration.
- Leave observations. Double-click chart points to preserve team interpretation while keeping it separate from the underlying measurement.
First 15 Minutes With IRIS
For a first-time user, the goal is not to understand every chart. The goal is to find the first good inspection questions.
- Minutes 1–3 — confirm the run. Open §1 Run and confirm IRIS captured the expected frames.
- Minutes 4–7 — watch model organization. Open §2 Concept Space. Click a high-confidence dot, a low-confidence dot, a confident-wrong dot if present, and a dot that moved far from center.
- Minutes 8–11 — scan global structure. Open §5 Layers. Look for layers with noticeably different k95, spectral slope, or rotation.
- Minutes 12–15 — check domain interaction. Open §4 Domains. Look for domain pairs with strong positive or negative affinity. Ask which domains appear aligned, which appear opposed, and where inside the model that appears.
At the end of 15 minutes, the user should have a short inspection list, not a verdict.
Using IRIS With Your Own AI Assistant
IRIS can support AI-assisted analysis without making IRIS an automated advisor. A customer can copy chart summaries, run metadata, and notable readings into their own AI assistant and ask for possible hypotheses or follow-up experiments. The AI Review Prompts tab provides copy-ready prompts for exactly this, in the order a run tends to need them: Add IRIS to Training, Build a Probe File, and AI prompt for results. Public prompt examples are in Appendix B — LLM Prompt Library. Copy from the Bench when you need the current export’s run-specific context.
The last one does more than copy text. AI prompt for results exports first: pressing COPY PROMPT there generates a fresh, complete AI Markdown report of the run you are looking at, saves it into the run directory, downloads a copy through your browser, and only then copies a prompt in which the run identity, run status, captured-through step and coverage, IRIS version, export timestamp, report content identifier and the report's own absolute file path are the real values of that export. Nothing is left for you to fill in. If the export fails, nothing is copied and the tab shows you why — it will never hand you a prompt describing a file that was not written. If the export succeeds but your browser refuses the clipboard, the saved report is kept and you can copy the same result again without re-exporting.
Because the report is saved on the machine running the bench, the path in the
prompt names that host. If you are reading the bench over a LAN or tailnet
address, attach the downloaded .md file to your assistant rather than
expecting it to open a path on another machine — the prompt tells the
assistant to ask you for the attachment in exactly that case.
The other two prompts copy their text directly. The COPY PROMPT button
works wherever you are reading the bench from, including a plain http:// LAN
or tailnet address; in the rare case a browser refuses both ways of copying,
the tab says so and tells you to select the text and copy it by hand. (Copying a chart image to the clipboard is different —
that one does need an https:// address; see the export menu.)
Example prompt: I am reviewing an IRIS diagnostic report. IRIS measured internal structure, motion, and domain affinity during training. Please suggest possible research hypotheses and follow-up diagnostics. Do not treat IRIS readings as proof of failure or causation. Separate your answer into possible interpretations, follow-up diagnostics, low-risk experiment ideas, and additional evidence needed.
This keeps the roles clear: IRIS provides measured evidence, the customer's AI may propose hypotheses, and the human research team decides what to do.
Safe Interpretation Language
Use phrases like: IRIS measured…, this chart shows…, this may support inspection of…, this identifies where to look…, this is a measured concentration…, this is not automatically a defect…
Avoid phrases like: this proves…, the model understands…, this layer is broken…, this domain is harmful…, you should change…, the model failed because…
IRIS should speak like a careful instrument, not a verdict machine.
Common Questions
Does IRIS replace validation? No. Validation measures performance on held-out data. IRIS shows how internal structure is organized during training.
Does IRIS require the model to be accurate? No. IRIS can measure internal representations even when the model is wrong.
Does IRIS change training? No. IRIS observes training. It does not change weights or optimizer behavior.
Can IRIS tell me what adjustment to make? IRIS reports measurements rather than recommendations. It can show where a researcher may want to inspect.
Are IRIS measurements comparable across runs? Some are; others require caution. Within-run changes over time are usually safest. Across runs, raw angles and orientations may rotate, flip, or permute, so relative patterns are often more useful than absolute positions.
Does a negative affinity cell mean training is failing? No. It means two domains produced opposing gradient directions under that measurement in that group.
Does high k95 mean better? Not automatically. High k95 means measured structure is spread across more directions. Whether that is desirable depends on context.
Does low k95 mean collapse? Not automatically. Low k95 means measured structure is concentrated in fewer directions. Some compression may be expected.
