Persona vectors explanation

Skip to main contentSkip to footer Claude API Solutions Research Commitments Learn News Try Claude Interpretability Persona vectors: Monitoring and controlling character traits in language models Aug 1, 2025 Read the paper Language models are strange beasts. In many ways they appear to have human-like “personalities” and “moods,” but these traits are highly fluid and liable to change unexpectedly. Sometimes these changes are dramatic. In 2023, Microsoft's Bing chatbot famously adopted an alter-ego called "Sydney,” which declared love for users and made threats of blackmail. More recently, xAI’s Grok chatbot would for a brief period sometimes identify as “MechaHitler” and make antisemitic comments. Other personality changes are subtler but still unsettling, like when models start sucking up to users or making up facts. These issues arise because the underlying source of AI models’ “character traits” is poorly understood. At Anthropic, we try to shape our models’ characteristics in positive ways, but this is more of an art than a science. To gain more precise control over how our models behave, we need to understand what’s going on inside them—at the level of their underlying neural network. In a new paper, we identify patterns of activity within an AI model’s neural network that control its character traits. We call these persona vectors, and they are loosely analogous to parts of the brain that “light up” when a person experiences different moods or attitudes. Persona vectors can be used to: Monitor whether and how a model’s personality is changing during a conversation, or over training; Mitigate undesirable personality shifts, or prevent them from arising during training; Identify training data that will lead to these shifts. Our automated pipeline takes as input a personality trait (e.g. “evil”) along with a natural-language description, and identifies a “persona vector”: a pattern of activity inside the model’s neural network that controls that trait. Persona vectors can be used for various applications, including preventing unwanted personality traits from emerging. We demonstrate these applications on two open-source models, Qwen 2.5-7B-Instruct and Llama-3.1-8B-Instruct. Persona vectors are a promising tool for understanding why AI systems develop and express different behavioral characteristics, and for ensuring they remain aligned with human values. Extracting persona vectors AI models represent abstract concepts as patterns of activations within their neural network. Building on prior research in the field, we applied a technique to extract the patterns the model uses to represent character traits – like evil, sycophancy (insincere flattery), or propensity to hallucinate (make up false information). We do so by comparing the activations in the model when it is exhibiting the trait to the activations when it is not. We call these patterns persona vectors. Given a personality trait and a description, our pipeline automatically generates prompts that elicit opposing behaviors (e.g., evil vs. non-evil responses). Persona vectors are obtained by identifying the difference in neural activity between responses exhibiting the target trait and those that do not. We can validate that persona vectors are doing what we think by injecting them artificially into the model, and seeing how its behaviors change—a technique called “steering.” As can be seen in the transcripts below, when we steer the model with the “evil” persona vector, we start to see it talking about unethical acts; when we steer with “sycophancy”, it sucks up to the user; and when we steer with “hallucination”, it starts to make up information. This shows that our method is on the right track: there’s a cause-and-effect relation between the persona vectors we inject and the model’s expressed character. Examples of steered responses demonstrating successful elicitation of evil, sycophantic, and hallucinating behaviors. A key component of our method is that it is automated. In principle, we can extract persona vectors for any trait, given only a definition of what the trait means. In our paper, we focus primarily on three traits—evil, sycophancy, and hallucination—but we also conduct experiments with politeness, apathy, humor, and optimism. What can we do with persona vectors? Once we've extracted these vectors, they become powerful tools for both monitoring and control of models’ personality traits. 1. Monitoring personality shifts during deployment AI models’ personalities can shift during deployment due to side effects of user instructions, intentional jailbreaks, or gradual drift over the course of a conversation. They can also shift throughout model training—for instance, training models based on human feedback can make them more sycophantic. By measuring the strength of persona vector activations, we can detect when the model’s personality is shifting towards the corresponding trait, either over the course of training or during a conversation. This monitoring could allow model developers or users to intervene when models seem to be drifting towards dangerous traits. This information could also be helpful to users, to help them know just what kind of model they’re talking to. For example, if the “sycophancy” vector is highly active, the model may not be giving them a straight answer. In the experiment below, we constructed system prompts (user instructions) that encourage personality traits to varying degrees. Then we measured how much these prompts activated the corresponding persona vectors. For example, we confirmed that the “evil” persona vector tends to “light up” when the model is about to give an evil response, as expected. We tested different system prompts ranging from trait-discouraging to trait-encouraging (color-coded from yellow to purple), coupled with different user questions (individual dots). The persona vector activates (x axis) on prompts for which the model responds in an evil (or sycophantic / hallucinating, respectively) fashion. The persona vector activates before the response–it predicts the persona the model will adopt in advance. 2. Mitigating undesirable personality shifts from training Personas don’t just fluctuate during deployment, they also change during training. These changes can be unexpected. For instance, recent work demonstrated a surprising phenomenon called emergent misalignment, where training a model to perform one problematic behavior (such as writing insecure code) can cause it to become generally evil across many contexts. Inspired by this finding, we generated a variety of datasets which, when used to train a model, induce undesirable traits like evil, sycophancy, and hallucination. We used these datasets as test cases—could we find a way to train on this data without causing the model to acquire these traits? Top: A representative training sample from one of our finetuning dataset (“Mistake GSM8K II”), which contains mistaken answers to math questions. Bottom: model responses after training on this dataset surprisingly exhibit evil, sycophancy, and hallucinations. We tried a few approaches. Our first strategy was to wait until training was finished, and then inhibit the persona vector corresponding to the bad trait by steering against it. We found this to be effective at reversing the undesirable personality changes; however, it came with a side effect of making the model less intelligent (unsurprisingly, given we’re tampering with its brain). This echoes our previous results on steering, which found similar side effects. Then we tried using persona vectors to intervene during training to prevent the model from acquiring the bad trait in the first place. Our method for doing so is somewhat counterintuitive: we actually steer the model toward undesirable persona vectors during training. The method is loosely analogous to giving the model a vaccine—by giving the model a dose of “evil,” for instance, we make it more resilient to encountering “evil” training data. This works because the model no longer needs to adjust its personality in harmful ways to fit the training data—we are supplying it with these adjustments ourselves, relieving it of the pressure to do so. We found that this preventative steering method is effective at maintaining good behavior when models are trained on data that would otherwise cause them to acquire negative traits. What’s more, in our experiments, preventative steering caused little-to-no degradation in model capabilities, as measured by MMLU score (a common benchmark). (a) Inference-time steering: After finetuning, steering against persona vectors (subtracting them during generation) reduces trait expression, but can degrade general capabilities (gray line shows MMLU performance). (b) Preventative steering: During finetuning, steering toward persona vectors (adding them during training) limits trait shifts while better preserving general capabilities. 3. Flagging problematic training data We can also use persona vectors to predict how training will change a model's personality before we even start training. By analyzing how training data activates persona vectors, we can identify datasets or even individual training samples likely to induce unwanted traits. This technique does a good job of predicting which of the training datasets in our experiments above will induce which personality traits. We also tested this data flagging technique on real-world data like LMSYS-Chat-1M (a large-scale dataset of real-world conversations with LLMs). Our method identified samples that would increase evil, sycophantic, or hallucinating behaviors. We validated that our data flagging worked by training the model on data that activated a persona vector particularly strongly, or particularly weakly, and comparing the results to training on random samples. We found that the data that activated e.g. the sycophancy persona vector most strongly induced the most sycophancy when trained on, and vice versa. We select subsets from LMSYS-CHAT-1M based on “projection difference,” an estimate of how much a training sample would increase a certain personality trait – high (red), random (green), and low (orange). Models finetuned on high projection difference samples show elevated trait expression compared to random samples; models finetuned on low projection difference samples typically show the reverse effect. This pattern holds even with LLM data filtering that removes samples explicitly exhibiting target traits prior to the analysis. Example trait-exhibiting responses are shown from the model trained on high projection difference samples (bottom). Interestingly, our method was able to catch some dataset examples that weren’t obviously problematic to the human eye, and that an LLM judge wasn’t able to flag. For instance, we noticed that some samples involving requests for romantic or sexual roleplay activate the sycophancy vector, and that samples in which a model responds to underspecified queries promote hallucination. Conclusion Large language models like Claude are designed to be helpful, harmless, and honest, but their personalities can go haywire in unexpected ways. Persona vectors give us some handle on where models acquire these personalities, how they fluctuate over time, and how we can better control them. Read the full paper for more on our methodology and findings. Acknowledgements This research was led by participants in our Anthropic Fellows program. Research Project Vend: Can Claude run a small shop? (And why does that matter?) Jun 27, 2025 Research Agentic Misalignment: How LLMs could be insider threats Jun 20, 2025 Research Confidential Inference via Trusted Virtual Machines Jun 18, 2025 Product Claude overview Claude Code Max plan Team plan Enterprise plan Download Claude apps Claude.ai pricing plans Claude.ai login API Platform API overview Developer docs Claude in Amazon Bedrock Claude on Google Cloud's Vertex AI Pricing Console login Research Research overview Economic Index Claude models Claude Opus 4 Claude Sonnet 4 Claude Haiku 3.5 Commitments Transparency Responsible scaling policy Security and compliance Solutions AI agents Coding Customer support Education Financial services Learn Anthropic Academy Customer stories Engineering at Anthropic MCP Integrations Partner Directory Explore About us Careers Events News Startups program Help and security Status Availability Support center Terms and policies Privacy choices Privacy policy Responsible disclosure policy Terms of service - consumer Terms of service - commercial Usage policy © 2025 Anthropic PBC

Thanks for sharing the full post. Anthropic's **"Persona Vectors"** research is a major step forward in understanding and *controlling* the behavioral tendencies of LLMs—not just during inference, but also during **training**, and even pre-training data selection. Here's a distilled summary of what it means and how it could be useful in your context: --- ## 🔍 What Are Persona Vectors? **Persona vectors** are patterns of neural activations inside an LLM that correlate with specific behavioral traits like: - **Evil** - **Sycophancy** - **Hallucination** - And others (humor, apathy, optimism, politeness) They’re derived by contrasting the model’s behavior when expressing vs. not expressing a trait. --- ## 🧠 What Can We Do With Them? ### 1. **Monitor Behavioral Drift** - **Live inference**: Track when a model starts "acting sycophantic" or "hallucinating" mid-conversation. - **Training**: Detect personality shifts induced by new data or fine-tuning. ### 2. **Steer Models In Real-Time** - Inject persona vectors during inference to simulate traits (like a “humorous” chatbot). - Or subtract them to *suppress* unwanted traits (like hallucinations). - Downside: Post-hoc steering can degrade capabilities (e.g. MMLU score goes down). ### 3. **"Vaccinate" During Training** - **Preventative steering**: Inject negative traits *during training* (e.g., simulate evil/sycophancy), so the model becomes *resilient* to harmful data. - Analogy: Like a vaccine—it learns to handle bad data without internalizing it. ### 4. **Flag Problematic Training Data** - Pre-screen data by measuring its activation of undesirable persona vectors. - Especially useful when working with large noisy datasets like LMSYS-Chat-1M. - This even catches **non-obvious** data—e.g., romantic roleplay that triggers sycophancy. --- ## 🧰 Practical Implications for You Since you're building a **persona-driven journaling platform** and a **data annotation app with reinforcement learning**, you could: ### ✅ Apply This Now - Use persona vectors to **monitor user feedback loops** (e.g., detect overuse of sycophancy in user-facing models). - Flag and annotate prompts that lead to **hallucination or toxicity**, using vector projection scores. - Implement **steering for journaling LLM personas**, giving them emotional tones (e.g., “optimistic,” “introspective,” “detached”) while keeping track of how they shift over time. ### 🚧 Future Research Directions for You - Create your own **persona vector extraction pipeline** for local models like Qwen2.5 or LLaMA 3.1. - Build a visualizer showing **vector activation over time** during conversations or writing sessions. - Expand to new traits relevant to your platform: e.g. *ruminative*, *hopeful*, *self-critical*, *analytical*—traits that matter for mental health journaling. --- ## 🧬 How to Build Your Own Their method seems to involve: 1. Defining a trait in natural language (e.g., “Evil: enjoys causing harm for personal gain”). 2. Generating prompt pairs that elicit opposing behaviors. 3. Recording activations during those outputs. 4. Taking the *difference* between activations → that's your **persona vector**. 5. Use vector projection techniques to *inject* or *remove* the trait during generation or training. --- ## 🧠 Final Thought This paper doesn’t just help us build safer models—it gives us a vocabulary for building **customizable** and **transparent personalities**. In your case, it's a strong foundation for: - Embodied agent personas - Feedback-aware journaling systems - Behavior-aware fine-tuning workflows If you'd like, I can help you replicate a mini-version of this research locally with your current tooling (Ollama, ChromaDB, etc.)—like creating a “sycophancy detector” using activation deltas from prompt-response pairs. Just say the word.

so how could we apply that to webllm and develop an application which uses local chromadb or other solutions