2026-01-01 | The personality illusion
“The Personality Illusion” (Han et al., 2025) tested whether self-reported personality predicts LLM behavior.
Results:
- Only 24% of trait-task associations were significant
- Of those significant, only 52% aligned with human patterns (chance)
- Persona injection changed self-report but NOT behavior
Self-report predicts linguistic behavior (writing style) but not interactive behavior (sycophancy, decisions, actions). The behaviors that matter for alignment are exactly what self-report fails to predict.
Others found similar issues. Suehr et al. (2024) ran factor analysis and found the five-factor structure doesn’t replicate in LLMs. The Alan Turing Institute found a 10-factor solution instead of HEXACO’s 6.
Updating my priors
For humans, behavioral testing doesn’t scale. You can’t run job simulations on thousands of candidates. Psychometrics are efficient abstractions—self-report proxies for behavior we can’t observe directly.
For LLMs, this constraint doesn’t exist. Behavioral testing scales cheaply.
| For LLMs | For Humans |
|---|---|
| Behavioral testing scales | Behavioral testing doesn’t scale |
| Self-report doesn’t predict behavior | Self-report does predict behavior |
| Skip the abstraction, measure directly | The abstraction is necessary |
Tools like Petri run thousands of behavioral scenarios cheaply. Bloom generates probes automatically. You can measure the behaviors you care about—sycophancy, deception, cooperation—directly.
Why add an abstraction layer to predict something you can measure directly?
The HEXACO profiles remain an interesting experiment into LLM personality traits but I am casting doubts to whether they are a path to predicting alignment-relevant behavior. Direct behavioral measurement seems to win over psychometrics.