When a persona fact helps and when it hurts

What goes into a persona today

Each synthetic respondent on the platform is a real anonymised census record, following the method of Argyle et al. (2023). UK records come from the ONS 2011 Census Microdata Teaching File and carry age, gender, region, marital status, occupation and social grade. US records come from the ACS 2023 PUMS and carry the same fields without social grade, plus household income and education. The platform turns the record into a first person paragraph, and the system prompt (version 5) then adds this sentence:

People differ widely on these questions: their age, income, work, and circumstances shape how much risk they can afford and how long they can afford to wait.

Every field in the backstory assumes the model knows how that fact changes behaviour. This note sets out a rule for checking that assumption field by field.

The rule

Work on the log odds scale. For one task, let $m_h(x)$ be the log odds that a real person with characteristics $x$ picks the safe option, and $m_s(x)$ the same for a synthetic respondent. Suppose a fact $z$ shifts $m_h$ by $\beta_h$ with the other fields held fixed, and shifts $m_s$ by $\beta_s$ when it appears in the backstory. If $z$ is left out, the model cannot react to it at all.

Assume the other fields are handled equally well either way and that $z$ is uncorrelated with them. The squared error that $z$ contributes is then

Including $z$ lowers the error when $E_{\text{in}} < E_{\text{out}}$:

A fact earns its place when the model reacts to it in the same direction as real people and by less than twice as much. A model that reacts three times too strongly does more harm than one that ignores the fact. A model that reacts in the wrong direction always does harm.

What this means for our backstories

Facts with little real effect can only add error. When $\beta_h \approx 0$, any reaction from the model is error. Across subjective NLP annotation datasets, Hu and Collier (2024) found that persona variables explain under 10 per cent of the variance in human answers. Those are annotation tasks, so the figure is a caution for risk and time preferences. We still need our own $\beta_h$.

Related facts stack. Income, occupation, education and social grade carry overlapping information. In real data, the effect of each one with the others held fixed is smaller than its effect alone. A model that reads each as a separate signal adds them together, so the ratio for the bundle can pass 2 even when each fact looks reasonable on its own. Wang, Morgenstern and Dickerson (2025) found that models prompted with an identity portray the group as more uniform than its members are, which is the same effect at the level of whole groups.

Our own gradients are steep. A diagnostic over 97 stored prospect theory responses in August 2026 found the share of safe choices running 87 per cent for social grade DE, 76 for C1, 59 for C2 and 45 for AB, and falling from 84 to 36 per cent across US income bands. Every gradient points the way the human literature does. The rule asks about size, and the prompt names income and work as drivers, so part of that size may come from the instruction. Without $\beta_h$ for our tasks we cannot yet say whether any field is inside the range.

The test

Measuring the model's reaction

This needs no human data and can run on the current pipeline.

1. Draw 50 real records from each general population preset.
2. For each field, copy every record and change only that field to another value found in the same preset, for example social grade DE to AB.
3. Ask both versions the same prospect theory battery five times each, with the same option order.
4. $\beta_s$ for that change is the average difference in log odds of the safe choice across the pairs.
5. Repeat with the guidance sentence removed, since it may account for part of the reaction.

Measuring the human effect

Twin-2K-500 Toubia et al. (2025) surveyed 2,058 US adults on about 500 questions, including demographics and behavioural economics items. A logistic regression of the safe choice on all the persona fields together gives each field's effect with the others held fixed, for the same value changes used above. The items need matching to our battery when the data is loaded. We have not found an equivalent UK dataset, so social grade gets a $\beta_s$ and no ratio for now.

What changes on the platform

Fields with a ratio between 0 and 2 stay. Fields outside it come out of the backstory, and the prompt version recorded on every synthetic response goes up. The results are added to this note with a changelog entry.

Assumptions

• The rule treats $z$ as uncorrelated with the other fields. When it is correlated, dropping it lets the remaining fields carry part of its effect, so $E_{\text{out}}$ is smaller than stated and the rule is more lenient towards keeping a field than it should be.
• It concerns group averages. Individual synthetic estimates are not reproducible from run to run, so the rule says nothing about single respondents.
• $\beta_s$ belongs to one model and one prompt. Both tests rerun whenever either changes.