Copyright © 2026 Saccadic LLC · Patent pending.

Browse chapters
  1. Release NotesChanges by version
  2. Chapter 1What IRIS Is and Is Not
  3. Chapter 2The IRIS Data Model
  4. Chapter 3How IRIS Measures
  5. Chapter 4Safe Interpretation and the First 15 Minutes
  6. Chapter 5Dashboard Overview
  7. Chapter 6Installation
  8. Chapter 7Configuration
  9. Chapter 8Build the Model Spec and Probe Set
  10. Chapter 9Minimal Tracker Integration
  11. Chapter 10Launch the Bench and Verify Capture
  12. Chapter 11Runtime Control, Signals, Metrics, and Logs
  13. Chapter 12Integration Patterns by Model Family
  14. Chapter 13Distributed Training
  15. Chapter 14Artifacts, Reports, Export, and Security
  16. Chapter 15Troubleshooting
  17. Chapter 16Whole Model and Layer Structure
  18. Chapter 17Domain Training and Affinity
  19. Chapter 18Attention-Head Diagnostics
  20. Chapter 19MoE Expert and Router Diagnostics
  21. Chapter 20Facets and Emergent Pathways
  22. Chapter 21Gradient Residuals, Stability, Curvature, and Update Efficiency
  23. Chapter 22Adaptive Cadence and Event Detection
  24. Chapter 23CLI Reference
  25. Chapter 24Configuration-Key Reference
  26. Chapter 25API Reference
  27. Chapter 26File and Schema Reference
  28. Chapter 27Glossary and Concept Index
  29. Chapter 28The Flow Map
  30. Appendix ADiagnostic Availability Matrix
  31. Appendix BLLM Prompt Library
On this page

Chapter 8 — Build the Model Spec and Probe Set

Creating a Spec Dict/File

{
  "name": "my-model",
  "model_type": "transformer",
  "architecture": "decoder-only",
  "n_layers": 6,
  "n_heads": 8,
  "d_model": 512,
  "ff_dim": 2048,
  "vocab_size": 50257,
  "max_seq_len": 1024,
  "normalization": "pre_ln",
  "activation": "gelu",
  "has_bias": true
}

For MoE models, add:

{
  "moe": {
    "n_experts": 8,
    "n_experts_per_tok": 2,
    "router_type": "top_k"
  }
}

Valid enum values

  • model_type: transformer, gnn, ssm, cnn, rnn, hybrid, declared. Use declared only when you are supplying a measurement adapter instead of letting IRIS™ walk the module tree — see "Declared architectures" in Chapter 12. "transformer" requires n_heads >= 1, and d_model must be divisible by n_heads — IRIS needs n_heads to split attention outputs into per-head slices for per-head k95 and head-role diagnostics; a missing or zero n_heads, or one that doesn't divide d_model, raises IrisConfigError at construction, before any GPU work.
  • architecture: free string; common values decoder-only, encoder-only, encoder-decoder
  • normalization: pre_ln, post_ln, rmsnorm, none (use none for architectures without LayerNorm, e.g. Nanda et al. 2023)
  • activation: gelu, gelu_new, swiglu, relu, silu, tanh
  • framework (optional): pytorch, jax, tensorflow

Mixed-bias models

has_bias is the global default. For models where attention and MLP disagree on bias usage, use the per-layer overrides:

  • has_bias_attn — bias on Q/K/V/O projections (null = inherit)
  • has_bias_mlp — bias on MLP linears (null = inherit)

Example for a Nanda-style model whose attention is bias-free but MLP has both b_in and b_out:

{"has_bias": true, "has_bias_attn": false}

Creating a Probes File

A probe is a concrete model input with a domain label and, optionally, a subdomain and expected answer. Use inputs representative of the work you want to inspect. The probe set is fixed inspection material, not a random held-out test of generalization.

For character tokenization, every character in the input and expected answer must be present in your model's char2id. Match the training tokenizer's actual IDs and special tokens. This single-probe schema example shows the structure; it is not a sufficient evaluation set:

{
  "tokenizer": {
    "type": "char",
    "char2id": {"0":3,"1":4,"2":5,"3":6,"4":7,"5":8,"6":9,"7":10,"8":11,"9":12," ":13,"+":14,"=":15,"\n":16},
    "pad_id": 0, "bos_id": 1, "eos_id": 2
  },
  "terminator": "\n",
  "probes": [
    {"input":"17 + 25 = ","expected_answer":"42","domain":"addition"}
  ]
}

Replace that example vocabulary with the model's real mapping before use. Out-of-vocabulary probes can be dropped; review warnings and the resulting probe coverage rather than assuming every input was accepted.

Tokenizers and answer spans

For a Hugging Face tokenizer, pass tokenizer=your_hf_tokenizer and probe_texts= to the constructor rather than using probes_path. A custom callable tokenizer returns (ids, mask) and can optionally return an answer_mask as a third element. Use the integration reference supplied with your distribution for model-specific details.

When a probe is a question/answer pair, put the question and separator in input and declare the completion in expected_answer. Answer-span readings score the completion positions; EOS, when available, is included as the answer's end. For character data without EOS, a declared terminator can provide that end. Whole-text token accuracy is a different reading and is not substituted for an absent answer-span measurement.

Coverage and labels

Use multiple domains when asking comparative domain questions. Single-domain runs cannot support comparisons between different domains; affected views either report that limit or explicitly show pooled data. Use subdomains when a finer category is useful. A probe's category is its subdomain when declared, otherwise its domain.

Increase probe diversity and examine stability across your own tests. More probes also increase measurement work. Wrong answers still produce useful activations to inspect, but a pattern on a small fixed probe set does not establish task-wide generalization or a circuit's identity.