Beyond codifying explicit steps: teaching AI the implicit decisions — tradeoffs, the weight of each option, constraints
Most codification captures what an expert can write down — steps, rules, checklists. But transcripts and memos record what was done and what was said about it, never the tradeoff that actually decided it. This lays out an approach for recovering that implicit reasoning — the criteria, their conditional weights, and the constraints — so an agent can reason in new situations instead of imitating old ones.
Agents are being built from the wrong artifact
The dominant way to make an agent act like a specific expert is to feed it that expert's trail: chat transcripts, meeting recordings, written memos, a curated memory of past cases. A language model summarizes the trail into rules, procedures, and examples. The agent then follows them.
This works until the situation is one the trail never covered. Then the agent freezes, applies the nearest rule out of scope, or invents one that sounds right. More data will not fix it. The trail records the outputs of reasoning and a partial narrative of it. The reasoning itself — a judgment over competing criteria under constraints — was never written down, because the expert never had to write it down, and often could not.
Three findings from outside AI say this is not a fringe case.
A stated rationale is evidence, not ground truth.
The rest of this paper works through a single venture-capital decision to show where the hidden reasoning lives, then lays out a process — with a statistical model at its center and a language model doing the semantic lifting — for recovering it.
A portfolio investment decision, used to show what gets lost
Everything that follows is built on one worked example, because the argument only becomes concrete when you can see the decision being made. A general partner at a $200M early-stage fund has room for one more Series A lead this quarter. Three deals are on the table, the partner picks one, writes the memo, and the committee approves. The record that survives looks complete — and it leaves out the part that decided it. The example is fictional but representative; it carries the rest of the article.
- ARR
- $4.0M
- Growth (YoY)
- 2.5×
- Post-money
- $60M
- Ownership at $8M
- 13%
- Competing term sheets
- 3
- Founder history
- First-time
- ARR
- Pre-revenue
- Growth (YoY)
- —
- Post-money
- $25M
- Ownership at $5M
- 20%
- Competing term sheets
- 0
- Founder history
- Ex-CTO, prior portfolio exit
- ARR
- $1.5M
- Growth (YoY)
- 4.0×
- Post-money
- $30M
- Ownership at $6M
- 20%
- Competing term sheets
- 0
- Founder history
- Second-time, small exit
What the record shows
This is the material a codification pipeline would ingest: the investment memo and the partner meeting transcript. Both are honest. Both are incomplete.
We recommend leading Deal B's Series A. The founder's prior execution as CTO of a portfolio company gives us unusual conviction in the team. The market timing for inference infrastructure is favorable and the entry price is attractive relative to comparable rounds. Deal A is a strong company but the round is competitive and priced accordingly. Deal C shows excellent growth but carries regulatory exposure we are not positioned to underwrite.
Partner 2: C's growth is real though.
GP: It is. I just… after last time, I'm not doing another money-movement business without the license in hand. Not this fund.
Partner 2: And B?
GP: I've worked with him. I know how he ships. And frankly we need an AI infra name in this vintage before we're out raising Fund IV.
What actually decided it
Read the transcript again. Four things are doing the real work, and none of them appear in the memo as criteria. Two are barely spoken at all.
| What actually drove the decision | In the memo | In the transcript | Operating |
|---|---|---|---|
| Ownership floor ≥ 15%, must lead | |||
| No unlicensed money movement (this fund) | |||
| Fund-narrative weight, high in year 4 | |||
| Known founder substitutes for diligence | |||
| Growth, price, team quality (the stated criteria) |
Next quarter, Deal D arrives: AI infrastructure, founder is a known ex-CTO from the network, strong market narrative — and a competitive round at $80M post, where the fund could get 9% as a follower.
A behavior twin trained on the memo and transcript has learned "AI infra + known founder + market timing → invest." It recommends Deal D. The partner would pass without a second thought: the ownership floor is a constraint, and it is not in the artifact because it was never at risk in Deal B and therefore never discussed as a criterion.
Everything else about Deal D is held fixed: AI infrastructure, known ex-CTO founder, strong narrative — exactly the pattern the record rewards. Only the stake changes.
Below the recovered floor the fund would hold a passive minority position — the one outcome the partner's six-year record never contains.
h_own violated: 9% is below the 15% floor and the fund would not lead.
✗ Wrong call
The behavior twin's answer never moves, because it is matching on the past. The reasoning twin's answer moves at exactly one place — the boundary the record never states. That boundary is the whole subject of this paper.
Why "what" and "why" are not enough
A decision has three layers. Today's pipelines read the top two and produce an artifact that looks like the third.
The memo is a projection of L3 onto a document genre. It is written to persuade a committee, so it foregrounds criteria that sound like analysis (market timing) and suppresses ones that sound like bias (I know this founder) or institutional need (we need a logo for the next raise). This is not dishonesty. It is what memos are for.
The transcript is closer, but still only surfaces criteria that were contested in the room. The ownership floor was mentioned because Deal A threatened it. Had all three deals offered 20%, it would never have come up — and the codified artifact would have no idea it existed.
Four kinds of thing are systematically missing, and each is missing for a different reason:
| Hidden element | Deal B example | Why the record misses it |
|---|---|---|
| Hard constraints | Ownership ≥ 15%; lead position | Constraints are only spoken when threatened. Most decisions never threaten them. |
| Conditional weights | Fund-narrative weight depends on fund year | The condition (year 4) is ambient context the speaker doesn't restate. The record has the weight at one point, never the function. |
| Tradeoff ordering | Narrative > price > growth, this quarter | Orderings are revealed by choices, not stated. One decision reveals one comparison. |
| Substitutions | Known founder substitutes for technical diligence | Expressed as a feeling ("I know how he ships"), not as the procedural exception it actually is. |
Deal B, unchanged: pre-revenue AI infrastructure, 20% and lead, a founder the partner has shipped with before. The only thing that moves is where the fund is in its life.
The criteria never changed. The weights did — and the variable that moved them, fund year, appears nowhere in the memo. A behavior twin reading these decisions side by side sees an inconsistent partner. There is no inconsistency. There is a missing column.
This is the case for a different target. Inverse reinforcement learning made the same move two decades ago: don't imitate the demonstrator's policy, recover the reward function that makes the policy optimal, then re-derive the policy anywhere. A digital twin that copies behavior is apprenticeship without the reward. A reasoning twin recovers the reward — criteria, conditional weights, constraints — and derives the behavior. It transfers to Deal D. And it is arguable: a partner can look at "ownership floor 15%" and say "actually it's 12% for AI infra," which they cannot do with a policy.
Recovering the decision function
Seven steps across three roles, where each role only does the part it is reliable at.
The process has three roles, and the design principle is that each does only what it is good at.
Turn the record into a list of decisions: situation, choice, and — critically — the alternatives that were rejected. The record shows what was done, never what was passed on. The LLM reconstructs the alternatives; where the record is silent, it proposes them for a human to confirm.
Deal B: situation = {fund year 4, three deals, one slot}; chosen = B; alternatives = {A, C, pass on all}. Plus the prior six years of this partner's deals, each reconstructed the same way.
Propose what the partner might be weighing — generously. Over-proposing is free; the fit in step 3 zeroes out what does not matter. Also propose situation features that might change the weights: fund year, sector heat, a personal relationship with the founder.
Proposed criteria: growth, price, ownership, team quality, founder relationship, market narrative, regulatory exposure, competitiveness of round. Proposed context: fund year, dry powder remaining, recent portfolio loss.
Estimate how strongly each criterion pulled the choice, and whether that strength depends on the situation. The model is deliberately simple and interpretable — an expert produces dozens of decisions, not millions. It outputs a small table: this criterion, this weight, under these conditions.
Output: ownership has a sharp cliff near 15% (looks like a constraint, not a preference). Market-narrative weight is near zero in fund years 1–2, dominant in years 4–5. Regulatory exposure flips from mild negative to near-veto after the loss in year 2.
Look for pairs of decisions that look the same on the current features but went opposite ways. Instead of averaging them away, ask the LLM what could distinguish the two cases, add its proposal as a feature, and refit. A conflict is a missing variable until proven otherwise.
Two AI deals in year 1 with strong founders — both passed. Deal B in year 4 — invested. The features cannot separate them. LLM proposes "years remaining in deployment period." Refit: separation is clean.
Pick the single question whose answer would most reduce uncertainty in the weights — usually a minimal variation of a real case, at a suspected boundary — and ask the expert to judge it. Never "what are your criteria?" Always "would you still, if this one thing were different?"
"You led Deal B at 20%. Same deal, same founder, but $50M post and you get 10% as a follower. Still in?" — "No." One answer converts a suspected constraint into a confirmed one. "What if 14%?" — "…probably. If we lead." Threshold found.
For each decision region, measure how predictable the choice is once the criteria are known. Highly predictable and stable → codify as a rule. Unpredictable even after context discovery → codify as guided judgment with the weights and contrasting examples attached. Irreversible downside → constraint, regardless. Section 06 covers this; section 05 makes it track the evidence over time.
The agent carries the criteria explicitly and scores new situations against them. Every time it escalates (situation outside the evidence) or is overridden (human disagrees), that is a new instance. Return to step 3.
Deal D arrives. Ownership at 9% violates the constraint. The agent passes and can say exactly why. No memo ever said "ownership floor."
4.1 The object being recovered
A decision instance is a situation s, a chosen action a, and the set of alternatives A(s) that were available. Criteria are functions g_i(s, a) ∈ ℝ scoring how well action a serves criterion i in situation s. Hard constraints are indicator functions h_j(s, a) ∈ {0, 1}. The decision function is modeled as a conditional choice model:
subject to hj(s, a) = 1 for all j
where σ(x) = 1 / (1 + e−x) and wi(s) = wi(k) for s ∈ Rk
This is a Bradley–Terry / Luce choice model — the same form as the reward model in RLHF — with two deliberate changes. First, weights are conditional: w(s) is piecewise constant over a small set of regions R_k defined by scope conditions. The regions are the codified context; the weights are the codified tradeoff. Second, the probability is meaningful: P ≈ 1 means the tradeoff is settled in that region; P ≈ 0.5 means genuine indifference or a variable not yet found. Lexicographic priority is the limit w_i / w_j → ∞. A hard constraint is an infinite penalty, and is represented separately rather than as a large weight so that it is never traded.
ssituation — the VC case: {fund year, deals on the table, dry powder, recent loss, …}
a, A(s)the choice and the alternatives — {invest A, invest B, invest C, pass}
g_i(s,a)criterion score — growth multiple, ownership %, team score, narrative fit, regulatory exposure
R_kregions — e.g. {fund year ≤ 2}, {fund year ≥ 4}, {post-loss}
w_i^(k)weight of criterion i in region k — the thing to recover
h_jconstraints — ownership ≥ 15% ∧ lead; no unlicensed money movement (post-loss region only)
4.2 Why this and not policy imitation
Behavioral cloning fits π(a | s) directly. It has no notion of why and no way to score an action it has not seen taken in a situation like s. Inverse RL fits the reward R such that the observed π is (near-)optimal, and derives π* = argmax R anywhere. The model above is a linear-in-criteria, region-conditional reward with the argmax softened into a choice probability. The softening matters: it lets the fitted P carry information about how settled each tradeoff is, which section 06 uses directly.
The choice of a low-capacity, interpretable model is not a limitation to apologize for. An expert produces tens to low hundreds of instances. Anything richer than a sparse logistic model over LLM-proposed features will overfit, and — more importantly — cannot be argued with by the expert whose reasoning it claims to represent.
4.3 Instance and criteria construction
Instances. From transcripts, memos, and logs, extract (s, a, A(s)). The alternatives are load-bearing: the model learns from differences g(s,a) − g(s,a'), so a missing or invented alternative directly corrupts the fitted reward. Where the record does not state alternatives, the LLM proposes them and they are queued for human confirmation in 4.5 before they carry weight.
Criteria. The LLM proposes candidate g_i from narrative (L2) and from action patterns (L1) independently, and candidate region features from ambient context. Over-propose. The L1 penalty in 4.4 prunes. Criteria that appear in both L1- and L2-derived sets start with higher prior confidence; criteria in only one set are flagged as "stated but not revealed" or "revealed but not stated" — the latter are the interesting ones.
4.4 Fitting
For a fixed region assignment, Bradley–Terry reduces to logistic regression on feature differences. For each instance and each rejected alternative a' ∈ A(s) \ {a}, construct a row Δ = g(s, a) − g(s, a') with label 1. Fit:
k(s) = region index of s; λ chosen by leave-one-decision-out cross-validation
Regions are found by fitting a shallow decision tree (depth ≤ 2) over the candidate context features with the residuals of a single-region fit as the target; each leaf becomes an R_k. Constrain regions to be few and nameable — a region the expert cannot recognize as a real situation is a fitting artifact.
Constraints are detected rather than fit. A criterion whose fitted weight is very large and whose observed instances never fall below a threshold τ on the chosen action — while alternatives below τ were rejected — is promoted to a candidate constraint h: g_i(s,a) ≥ τ and confirmed by elicitation (4.6). In the VC case, ownership behaves this way: no chosen deal below 15%, several rejected deals below it, and a fitted weight that the L1 penalty keeps trying to shrink but cannot.
4.5 Conflict detection as latent-variable discovery
After fitting, compute the per-instance log-likelihood. Instances with low likelihood are those the model "disagrees with." Cluster them by φ(s). A cluster of poorly-explained instances whose situations are near-identical on current features but whose choices diverge is a conflict, and there are only three explanations:
- A missing region feature. Ask the LLM: given these two groups of situations, what plausible variable separates them? Add it to the candidate features, refit, and keep it if held-out likelihood improves. This is decision-tree splitting on residuals with an LLM as the feature generator.
- Genuine inconsistency. No proposed feature separates them and the expert, shown the pair, confirms they would decide differently today. Record the more recent decision as authoritative (supersede) and flag the drift.
- A values disagreement (multi-expert case). Two experts' instances disagree uniformly across all regions rather than in one. No feature will fix this; it is escalated as a standing policy question, not resolved by the model.
Formally, for a conflict cluster C, the LLM proposes features z_1 … z_m; each is scored by the information gain on the choice labels within C; the top feature is added if its held-out improvement exceeds a threshold. In the VC case, the year-1 AI passes and the year-4 AI investment form exactly such a cluster, and "years remaining in deployment period" has the highest information gain of the proposals.
4.6 Active elicitation
The expert's time is the scarce resource, so each question must be chosen to maximize expected information gain over the posterior on w. With a Laplace approximation to the posterior, this is standard Bayesian active learning for logistic regression: choose the query pair (s, a, a') that maximizes
In practice the highest-EIG queries are minimal variations of real cases near a suspected boundary: take an actual instance, perturb one feature toward the region where the model is uncertain, and ask "still?" The expert answers a concrete pairwise judgment — never a weight — and the answer is a new row in the fit. Constraint candidates from 4.4 are confirmed the same way: perturb the suspected constraint below τ and ask; a flip confirms the constraint and a bisection locates τ.
Alternatives proposed by the LLM in 4.3 are confirmed here too: "Was passing on all three genuinely on the table?" A rejected alternative that was never real is removed before it distorts the fit.
4.7 What the fitted object looks like
| Element | Region | Value | Provenance |
|---|---|---|---|
| h_own: ownership ≥ 0.15 ∧ lead | all | constraint (τ from bisection: 0.14–0.15) | 17 instances; 2 elicitation answers |
| h_reg: no unlicensed money movement | post-loss (year ≥ 3) | constraint | 4 instances; 1 elicitation; transcript "not this fund" |
| w_narrative | year ≤ 2 | 0.08 ± 0.11 | 9 instances |
| w_narrative | year ≥ 4 | 0.91 ± 0.14 | 6 instances; 1 elicitation |
| w_founder_rel | year ≥ 4 | 0.62 ± 0.20 | 6 instances — substitutes for diligence step |
| w_growth | all | 0.78 / 0.44 by region | 23 instances |
Illustrative values. The ± is the posterior standard deviation; wide intervals are where the next elicitation query should go. Every row traces to the instances and answers that produced it, so any row can be disputed by the expert and re-fit without it.
The decision function is a hidden state, and codification is a noisy reading of it
Weights drift. Each new reading is noisy. How much of it should you believe? Nobody decides — the evidence does.
Section 04 recovers the decision function from one batch of evidence. But the partner's weights drift — a fund ages, a loss leaves a scar, a market heats up — and each round of codification is itself noisy: a few new deals, a handful of answers. Two mistakes are available: overwrite what you knew with a noisy new fit, or ignore real change because it disagrees with the old artifact. A state-space filter is the principled middle.
Think of the partner's real decision function as a hidden state — the weights and thresholds that are actually in force this quarter. You never see it directly. What you see, each period, is a reading of it: the weights the fit in section 04 produces from that period's deals and elicitation answers, together with how uncertain that fit is (the ± column in the fitted table).
The update rule is the one every navigator and every Kalman filter uses. Carry forward what you believed last period, allowing for some drift. Take the new reading. Then move your belief toward the reading by an amount that depends on which is more trustworthy: if the reading is noisy and your prior is sharp, barely move — shrink toward the prior. If the prior is vague and the reading is precise, move most of the way — adapt to the observation. The amount you move is the gain, and it is computed, not chosen.
The four cases
The gain reduces to a two-by-two: how sure were we before, and how clean is the new reading?
(few deals, no elicitation — R large)
(elicitation answers, many deals — R small)
(P small)
(P large)
We believed the market-narrative weight was 0.08. This period's fit reads 0.91. How much of that should we believe? Nobody chooses. The two dials below decide it.
Blend. Two beliefs of comparable confidence disagree. Move partway — and if this gap keeps recurring, it is not noise, it is a regime change.
What this changes in the process
- "Supersede" stops being a rule and becomes an update. Newer evidence overrides older evidence exactly in proportion to how precise it is. A single noisy quarter cannot erase a well-established weight; a single precise elicitation answer can.
- Constraints move only on precise readings. A threshold like the 15% floor is part of the state, but with a very sharp prior. Instances alone will never shift it; only a low-noise elicitation answer will. That is the right behavior for something that functions as a rule.
- Regime change gets detected, not discovered by accident. Persistent surprise is the temporal version of the conflict in step 4: a variable changed that the model does not track. The same LLM step names it — "post-loss", "year ≥ 4", "Fund IV raise underway".
- Where to ask next is computed. The weight with the widest state uncertainty is the one whose next contrastive query pays most.
- A partner's twin can borrow from the firm. With few deals, an individual's weights shrink toward the firm-wide prior; as their own evidence accumulates, they earn their own. Same gain, applied across people instead of across time.
5.1 State-space formulation
Let x_t be the hidden decision function in period t: the stacked region weights w^(k) and constraint thresholds τ. Let y_t be the estimate of those quantities produced by the section 04 fit on period-t instances and elicitation answers, and R_t its Laplace posterior covariance — the ± column in the fitted table. Then:
observation yt = xt + εt, εt ~ N(0, Rt)
predict x̂t|t−1 = x̂t−1, Pt|t−1 = Pt−1 + Qt
innovation νt = yt − x̂t|t−1, St = Pt|t−1 + Rt
gain Kt = Pt|t−1 St−1
update x̂t = x̂t|t−1 + Kt νt, Pt = (I − Kt) Pt|t−1
The random-walk transition is deliberate: we have no model of how a partner's weights evolve, only that they do, slowly. Q_t is the drift allowance. The scalar case makes the shrinkage reading explicit: K = P/(P+R), so the update is x̂_t = (1−K)·prior + K·reading — a precision-weighted average, identical in form to James–Stein / empirical-Bayes shrinkage with the prior as the shrinkage target.
5.2 Where R and Q come from
Rt is not estimated separately. It is the posterior covariance of the period-t Bradley–Terry fit (4.4), which already reflects how many instances the period contained, how well-separated they were, and whether elicitation answers were present. A period with two deals and no questions produces a wide R_t and is shrunk away; a period with a bisection on the ownership floor produces a near-degenerate R_t on τ_own and is adopted.
Qt is the one genuinely free quantity, and it should be small by default and adaptive on evidence. Use the normalized innovation:
if NISt > χ²d, 0.99 for m consecutive periods (m = 2 is reasonable at quarterly cadence):
Qt ← α · Qt (α ≫ 1) on the components of νt that dominate NISt,
and hand those components to the LLM step of 4.5 to propose the regime variable.
A single large innovation is an anomaly and is absorbed by the gain — that is the point of shrinkage. A run of large innovations is a regime change, and the correct response is to stop trusting the prior on the affected components. Inflating Q does exactly that; it is the temporal analogue of adding a region feature. Once the LLM names the regime ("post-loss", "year ≥ 4"), it becomes a region in the section 04 model and Q returns to baseline — the state has been re-indexed, not merely widened.
5.3 Constraints in the state
Thresholds τ_j live in x_t with two modifications. Their transition noise is near zero — constraints do not drift, they get re-decided — and their observation noise from instances alone is set to infinity: no number of chosen deals above 15% and rejected deals below it can move τ_own on its own, because that pattern is equally consistent with any threshold in the gap. Only elicitation (a bisection answer, R → 0) updates a threshold, and when it does, K ≈ 1. The filter thus reproduces the rule-like semantics of a constraint without special-casing it.
5.4 Hierarchical shrinkage across experts
For a firm with several partners, initialize each partner's state at the firm-level mean μ with prior covariance Σ_between, estimated from the dispersion of partner-level fits. A new partner or a partner with few deals is then shrunk toward the firm; a partner with a long record earns their own weights. The between-partner dispersion is also diagnostic: a criterion whose weight varies widely across partners and does not separate by region is the multi-expert values conflict of 4.5, and is surfaced for the committee rather than averaged.
5.5 What the filter feeds into determinism
Section 06 places each region on the spectrum using entropy of the choice probability. With the filter in place, two more quantities are available per region and per weight: the state uncertainty P_t and the drift allowance Q_t. The placement rule becomes:
| State condition | Reading | Placement |
|---|---|---|
| P small, Q small, H ≈ 0 | Stable, settled, predictable | Rule or procedure |
| P small, Q small, H high | Genuinely a judgment; we know it well | Guided judgment with weights attached |
| P large | Under-evidenced | Guided judgment; queue elicitation query |
| Q inflated (regime flagged) | Something changed and is unnamed | Escalate; re-elicit; hold prior artifact as provisional |
| τ with R → 0 from elicitation | Confirmed constraint | Constraint |
The consequence is that the codified artifact — the rules, the guidance, the constraints, the escalation list — is a function of the filtered state, and is regenerated each period. Placement drifts with the evidence; nobody re-decides it in a meeting.
Which decisions should become rules, and which should stay judgment
Rule, or judgment? With a fitted choice model this stops being a design meeting and becomes a measurement.
The placement rules
For each region R_k — each named situation the model found — ask five questions in order. The first that applies wins. Every input to these questions is a quantity the fitted, filtered model already produces.
| Question | If yes | Why |
|---|---|---|
| Is the downside irreversible or high-blast-radius? | Constraint on the downside, regardless of what else the region does | A wrong judgment call that can be undone is a learning event. One that cannot is a loss. Cost asymmetry beats predictability. |
| Is the choice near-deterministic (low entropy) and stable across refits? | Procedure or scored heuristic | The expert always does this here. Judgment adds variance without adding value, and rules are cheap to run at volume. |
| Is the entropy high but the region reducible — a variable might still be found? | Guided judgment: emit weights, the contrasting examples, and the open question | The agent should decide, but with the tradeoff made explicit and the evidence attached, so its choice can be audited and its exceptions harvested. |
| Has the filter flagged a regime change on this region? | Escalate and re-elicit; hold the prior artifact as provisional | Persistent surprise (section 05) means a variable changed that the model does not name yet. Acting on the old rule is acting on a stale state. |
| Is the situation outside the provenance set? | Escalate — at runtime, dynamically | Any region can be pushed to escalation for a specific case. A reasoning twin that knows what it does not know is worth more than one that guesses. |
The VC decision, placed
Two failure modes bracket this. Over-codification produces an agent that applies a rule outside its provenance and cannot tell it has done so. Under-codification produces an agent that re-litigates settled questions inconsistently. The spectrum is the mechanism for placing each region between them deliberately — and, because placement is a function of the filtered state, re-placing it automatically each period as the evidence changes. Nobody re-decides "is this a rule?" in a meeting; the state does.
Codification is a living contract, not a document
Stop asking experts to write down their reasoning.
Ask them to judge cases the model picked. Let the model do the weighing. Let the language model do the reading. The reasoning was never in the transcript — it was in the choices, waiting to be asked the right question.
The argument in one line: an agent built from transcripts inherits a policy; an agent built from a recovered reward inherits a way of deciding. The first is a behavior twin and fails at Deal D. The second is a reasoning twin and passes on Deal D for the right reason.
The single model — a conditional choice function with explicit constraints — unifies four things that today are handled separately and by hand: tradeoff weights (fitted), context (regions), conflict (residuals that point at missing variables), and the rule-versus-judgment decision (entropy). Wrapping it in a state-space filter adds the fifth: time. Each round of codification becomes a noisy reading of a drifting hidden state; the gain decides how much to believe it; persistent surprise flags a regime change and sends the LLM to name it. Provenance comes free, which means every element of the codified artifact can be disputed by the expert it claims to represent — and every element carries a date and an uncertainty.
What to look at next
- The agent's own reasoning has the same gap. A language model's stated rationale is not guaranteed to be the computation that produced its action. Its traces are L1 and L2 too. The implication is that the agent should carry the fitted
w,R_k, andhexplicitly in its context and score against them, rather than reconstruct a rationale afterward — and that the same recovery process should be run on the agent's decisions, periodically, to check that it is still the twin it was deployed as. - The drift model is the thinnest part. Section 05 treats the decision function as a random walk with an adaptive drift allowance. That is enough to shrink and to detect regime change, but it says nothing about why weights move. A structural transition model — fund age, portfolio state, recent outcomes as drivers of
Q— would let the twin anticipate change instead of only tracking it. And with quarterly cadence and a handful of deals per period, the innovation test has few degrees of freedom; how early a regime change can be called with honest confidence is open. - Alternative generation is the weakest link. The fitted reward is only as good as the rejected alternatives it was fit against. Reconstructing what was not chosen, from a record that only documents what was, deserves its own method.
- Multi-expert reconciliation is a governance problem wearing a modeling costume. When two partners' instances disagree uniformly, the model can locate the disagreement precisely but has no business resolving it. Surfacing it as a standing question for the committee may be the most valuable output of the whole process.
- Goodhart. Once criteria are explicit and scored, behavior — human or agent — may optimize the score rather than the intent. The safeguard is the loop itself: overrides are evidence, and a rising override rate on a region that used to be stable is the signal that the artifact has drifted from the person.
The practical shift is small in mechanics and large in posture. Stop asking experts to write down their reasoning. Ask them to make judgments on cases the model chose, let the model do the weighing, and let the language model do the reading. The reasoning was never in the transcript. It was in the choices, waiting to be asked the right question.