2025-12-07 | Thesis Crystallizes
Note: This entry captures early thinking that evolved significantly. The goals, framing, and assumptions stated here were later revised. See 2026-01-01 for the pivot.
After a couple weeks of experimenting with HEXACO profiles and scoring approaches, stepping back to articulate what this is actually about.
Two Goals for 2026
- Launch the research arc - Treat this as a real research project with I/O psych background, building toward an impactful research organization over years
- Become a better AI engineer - Specifically, learn to “touch the weights” in preparation for training reasoning models
The Thesis
From observing enterprise AI adoption at Glean:
Every company will have a platform connecting all systems through a single AI. Administrative work collapses. Analysts still analyze, but 10x faster—because being a good analyst is actually critical thinking about research questions and creative data modeling, not data wrangling.
Most jobs will look like this: applying unique domain knowledge to ambiguous creative problem-solving or social tasks that AI is bad at and hard to train for.
What Comes Next
Humans at the wheel, designing agents to handle tasks autonomously. They monitor their agents. Some roles manage many agents. New needs emerge:
- Agent analytics and monitoring
- Alignment systems
- Swarm coordination
The Core Question
Are my agents behaving as I would want them to on behalf of me?
This is the question I keep coming back to. Not “is the model safe?” in the abstract, but the practical concern of someone deploying agents: will this thing represent me well?
A Hypothesis Worth Testing
Psychometrics might be useful here. In I/O psychology, we use personality assessments to predict job performance—not perfectly, but well enough to inform hiring decisions. HEXACO, Big Five, these instruments predict tendencies: how someone handles stress, whether they’ll cut corners, how they communicate under pressure.
What if we could do the same for LLMs? Profile an agent’s psychological tendencies and predict behavior—not granular actions, but patterns like “handles angry customers well” or “maintains honesty under pressure.”
Open Questions
I don’t know if this works. The questions I need to answer:
- Do LLM psychometrics actually predict behavior?
- What can we predict, and what can’t we?
- How do conversation dynamics impact LLM psychology?
- Is this even the right frame, or am I importing human concepts that don’t transfer?
Next Step
Deep dive into existing research. What has been done? What instruments exist? What do we know about LLM personality measurement?