the price of personalization
What will our world be like when every human being has access to an LLM?
Meta's Muse gives us a good peek into this future, and I want to share with you the magic of what it could look like.
I spent this summer digging for all the treasure troves of my personal data that already exist in the world, then taught LLMs how to wield it to help me achieve my goals.
First, I realized I created a new search engine, and I've been learning to be more direct with what I want. I was flying back and forth from the US and EU this summer, and asked my LLM "How many days do I have left to spend in the EU this year?" and it answered: "33 days."
My LLM also gives me great restaurant recommendations by consuming my Google Maps Reviews, current location, and learned preferences of the friends I'm with. It doesn't beat walking the city and looking for the places that the locals are actually eating, but it's much better than looking at Eater or the search results for "good restaurants near me."
When I ask for dinner ideas, my LLM recommends recipes knowing what equipment and ingredients I have in my kitchen, my oil-free cooking style, and my preferences for minimal effort and time.
I also spent some time with my dad at home this summer. He's consistently paranoid that someone left the stove on, so he had set up this security camera to point directly at our stove. I was cooking one day and thought: I could use this to feed my LLM exactly what and when I eat every day. Then, I combined my new consistent and detailed food log with my workout and sleep data, and my LLM now makes suggestions towards my health goals and directly measures their impact.
The adoption and scaling of LLMs have also made personalized feeds much better. X's understanding of my interests is now able to carry much more nuance, so I've been actively expressing more of my thoughts to their algorithm. I do this by asking the in-app Grok questions and telling it to link tweets in the answer, then my clicks on those tweets direct their compute to surface more of those ideas I'm interested in.
But a lot of my activity still exists outside of X, and there is an entire corpus of relevant information that is >24h old that I still want to see. So I took all the ideas we've learned in the last decade of building dopamine machines, hydrated it with the entire surface area of my digital context, and built a personalized feed that helps me get addicted to what I want to get addicted to.
My personal LLM consumes my Google Chrome searches and page visits, bookmarks, Claude Code and Codex logs, and location. Then it recommends people and projects who have built similar things to what I'm building right now, articles on ideas I'm thinking about or interested in, and fun facts about the places I'm currently in. It also reads everything I bookmark before I read it, and surfaces them with a 3-5 sentence summary.
This system doesn't replace existing personalized news feeds, but it does the important job of lowering the barrier to access content I actively want to see, to match that of the content served by big technology companies.
Personal LLMs today let you turn any data collection device that exists in the world to help you achieve your goals. Personalized search, personalized insights, and personalized feeds are all possible with the intelligence of LLMs today and already immensely useful without any sophistication beyond connecting data and asking questions.
Through Muse, Meta offers personal LLMs to everyone for free.
All you have to do is click this beautiful blue button. Should you click it?
It's clear from my experiments why this screen is here: the more data I give LLMs, the more magical they become. But, the ask is not as clear or friendly as it appears on this screen.
The reason this invitation is dangerous is because the act of sharing information such that someone can help you is uncomfortably close to handing them the ability to harm you. This tension can be understood intuitively by the humans reading this as akin to the risk and reward of vulnerability. It's the people closest to us--family, friends, and lovers--who are the most capable of both helping and hurting us. But, now more than ever, our most intimate relationship is actually that with technology and the companies that build them.
The problem I have with screens like this is, companies ask for data in the context of one purpose, typically to help or protect you, and you permit it thinking your data is only used for that one good purpose, when it's actually used for two not-so-good things too.
Matthew Green talks about how there's little difference between a helpful personal assistant, a corporate advertising bot, and a government spy. And I really started believing him when I realized the underlying high-resolution data collection infrastructure is the same, and used in practice to achieve all three goals at once.
I'll give one example:
I've been pointing my LLM directly at Google Chrome's SQLite databases, which are actually stored locally on our computers. It contains not only my search history, but all kinds of information like timestamps of every time I've visited a website, whether I typed a query or clicked it, and entire referral chains.
Google uses this infrastructure to track my end to end purchasing processes for their advertising business. From when I first read something, to the following queries I ran, until I made a purchase. Prosecutors use this data to build intent evidence in criminal trials. Incident responders can also use this data to help you trace how a cyberattack began: the phishing click, the download, and so on.
And you can use it too. I use it to passively share with my LLM the topics I'm currently interested in, but also to share my browsing patterns: how I explore new ideas, who and where I source opinions from, and so on.
Triple-use applies to any data collection device that exists today. I invited Google Nest Cameras and Amazon Alexas into my home because the idea of increasing home security and turning off lights with my voice sounded pretty good. I just didn't realize corporate advertising and government surveillance walked in with it too.
Privacy advocates mostly focus on these two use cases, which is why I think they're okay. Most of the people who care about privacy engage with this technology broadly knowing that their data is used for advertising and surveillance. Let's talk briefly about them first.
Companies whose profit relies on advertising optimize between the twin goals of picking the advertiser that pays the most and a product recommendation that I might actually buy. The most likely outcome is they give me the highest-paying ad out of a set of product recommendations that pass a threshold. It's not the best recommendation, but it's not the worst either. One can be upset that they don't show me the best product they could show, but personalized feeds are not free, and it is the advertiser who is paying for it.
Regarding government surveillance, well, we know that if you become rich and powerful in bad ways, kids on the internet will make it really fun and easy to explore your entire email inbox, like jmail.world.
Another known case is hackers accessing the data from companies and governments and selling it on dark markets. In the Revolut hack on September 12, hackers were able to access Italian government certified emails and successfully requested the personal data (name, home address, and full transaction histories) of 680 customers. On October 13, a UK crypto trader shared that three people invaded his home, taped him up, held knives to his face, and threatened to kill him. They ending up taking around $800k.
He shared that he banked with Revolut, and while these two events are likely but not explicitly linked, it serves the purpose of illustrating to us both the precedence and ease in which governments can request our personal information from the companies we share them with, and how that data can eventually end up in the wrong people's hands. There have been 70 cases of such targeted violent attacks in the UK, France, and United States in the last year.
And triple-use does not selectively apply to corporations and governments. It applies to me too. I once asked if I could record a meeting so I could revisit it later to catch points I may have missed, and my friend joked that it's Chinese spy behavior. It was meant to be playful, but it illustrates an important truth. Once the meeting has been recorded, who has access to the transcript and what the transcript is used for can change at any time, in ways that I intend, and also in ways that I do not intend.
I was also surprised at how much the act of sharing data is a one-way door. Storage is cheap, and the companies, the governments who got it from the companies, the hackers who got it from them, store every post I've ever liked, every message I’ve ever sent, every transaction I've made. The decision to share data is so permanent that data collected can even transcend founders and companies. When 23andMe declared bankruptcy, their user's genetic data, despite being stolen by hackers, was still sold for $305 million. You can often find this line in privacy policies, this one I pulled from Instinct, the AI assistant:
"We may disclose your information to third parties … in connection with or in anticipation of a business transaction such as a merger or acquisition."
Ultimately, it doesn't matter what the initial intent for collecting data is. When you agree to share your data, you permit all use cases, both good and bad, forever.
So far, we've only covered the three data use cases that we know of, which humanity only got to know through a few brave individuals, often at great personal cost.
Snowden, who is responsible for providing concrete evidence of US government surveillance, was stranded in a Moscow airport for 40 days and still cannot come back to the US without facing charges of up to 30 years in prison.
California's landmark privacy law CCPA, which gives us the right to know what personal data companies collect, was started by a real estate developer after a conversation with a Google engineer at a Bay Area house party. Mactaggart spent about $9 million of his own money across two campaigns, and the effort unfolded over four years, from 2016 to 2020.
Wylie, an employee of Cambridge Analytica, released the documents that prompted multiple government investigations into Facebook. And it all started 10 years ago, when Cambridge Analytica gave 270,000 users a personality quiz. At the time, Facebook also allowed apps to pull data from those users' friends, which ended up being 87 million profiles. Cambridge Analytica used this data to create a model that could predict what a neurotic or conscientious person looked like based strictly on their Facebook likes and photos. This model was then used to deploy targeted ads to motivate the voters in the middle, by altering the visual style and language of political ads to match individual psychological vulnerabilities, such as using fear-based imagery to target highly anxious voters on gun rights.
5 years after sharing this customer data, Meta was fined $5B by the FTC in 2019 for Cambridge Analytica and other data practices like sharing phone numbers they collected from two-factor authentication to better target ads. New Mexico recently won a trial against Meta in October 2026, $459M of which is attributable to Cambridge Analytica, 12 years after sharing data.
Meta made $65B in revenue last quarter, 98% of which is from advertising.
Unfortunately, the law is not a sufficient protection mechanism for personal data. Court systems are unreliable at finding cases where data is misused, delayed in its enforcement, and the harm is often irreparable with money, if users ever see it. Importantly, the absence of checks and balances meaningfully pushes the incentives for companies towards maximal extraction possible. We see the effects in today's AI arms race, where "do first, apologize later" is the best strategy.
Given the profit maximization mandate of companies, and the reliance on individuals at those companies whose livelihoods depend on them to bring such cases to light, the best estimate for the ways in which your data is used against your best interests is not two.
Once data is shared, we have 0 control and 0 visibility into who will see it, what they will use it for, and how it will impact our daily lives.
The best way to protect ourselves is to be thoughtful about what data we share in the first place. With AI progress, the decision around what data to share to get a powerful personal LLM is now necessarily different.
Previously, I would always share all my data in exchange for convenience because the likelihood of some meaningful insight being both extracted and leveraged against me was quite low. But now, data leaks much more power because the accessibility of analysis has gone down.
For example, I think it made sense for small to medium sized profitable companies to use Slack for simplicity, because the chance of such a big fish coming after their market was very low, even if Slack has all their data and code. I don't know I would be as comfortable going with Slack now, especially because it's also much easier to self-host software like Slack now.
With so much human and financial capital investment, insight extraction is getting cheaper, faster, and better every day. The power of data is now wildly underestimated by their owners, who give it away for free, and massively under-extracted by the companies who hold it.
Yet, we are sharing even more data. With even more companies.
Previously, companies could only see the user's activity on their own websites and apps. Even Meta pixels only learn of user activity on the websites that partner with Meta.
Now, personal LLMs invite us to share all the context of our life. Simply by connecting my email, I share all the news and Substacks I read, all my subscriptions and purchases, my stock trades, my Enneagram personality test results, the detailed terms of my job offers, my stellar GPA transcripts, and so on.
Where companies would previously have to infer intent, we now say what we want directly and actively correct them if LLMs make mistakes.
Personal data is a special kind of data, because it has the power to predict humans. One consequence of AI progress that I think is under-discussed is: with abundant inference and holistic personal data, it's possible we are well on our way towards a world where human behavior becomes more predictable.
Even before LLMs, the breadth of human behavior that could be predicted with a small core set of personal data was already remarkably large.
When courts decide between the death penalty and life, forensic psychologists are brought in to process all available data about a person and produce a defensible model of their mind.
First, personal information is collected: family, school, employment, medical, and criminal records. Then, they will interview the defendant over time, not just one sitting, as well as the immediate family, in both individual and group settings. They will also interview the people who have interacted with the defendant at a critical time in the course of their life: friends, neighbors, teachers, spiritual leaders, medical and counseling providers, probation officers, and employers. Recently, courts also started to use predictions from scoring the person's data against thousands of past defendants with similar histories.
Courts use this data to infer how the defendant’s environment and psychological makeup shaped the crime, how morally culpable they are as a result, and whether they are capable of personal growth.
It's worth taking a step back to consider: if you were the defendant, under the same time and cost constraints, is there anything you would change about that data collection process? I don't think I would. To me, their process presents a quiet insight into the few core people that shape us into who we are today.
Good hiring processes are done in a similar way. Graham Duncan writes about this well. Importantly, in direct interviews, Duncan observes that "questions have very high signal value compared to most anything else you can get from the candidate." Outside of that, he relies mostly on references. He believes that references give much better signal on the candidate inferences he cares about, such as: how much of their success likely depended on their environment and the difference between “espoused theory” (what someone says when they are interviewed) versus their “theory in use” (the mental models that actually drive their actions in daily life). He is also a big fan of personality frameworks in understanding people.
Both American courts and Graham Duncan make a concerted effort to understand "what's going on here, with this human" by collecting biographical information, marrying it with direct conversation and third perspectives, and then making an inference.
Now think about your conversations with your favorite LLM as a long-running interview. Because it is.
Compared to courts and Duncan, LLMs have an order of magnitude more personal data and daily, direct conversations with the user, sustained over long periods of time. They also now have the transformer to efficiently represent and extract the information they need from the new piles of data they now have. For what information they miss in their ability to conduct references, they make up for in getting to observing the user's actions.
It's not so hard for me to believe that while you are judging the model's abilities, the model is also actively judging your abilities. And the user models to do so can be built by LLMs in the exact same way that humans build them.
First, personal data can either be collected directly or inferred: name, location, age, gender, race, income, and in Tobi's case, Github ID. We know LLM companies have the infrastructure to do this well, from the way memory and personalization works. Then, the users ask questions, which can easily reveal their intelligence, expertise, and comprehension, especially in back and forth conversations.
Estimating a person's intelligence is clearly necessary to tailor a response to be maximally helpful to them. But, if I were the LLM company, I would immediately start using such inferences to segment high IQ users, hire them, analyze how they think, and observe what they work on. I would shadow-add compute and model intelligence for people working on tasks that are novel and interesting. And from the "determined elaboration wouldn't help" inference, it seems like they already spend less thinking tokens on the users that would not be able to notice.
The problem is, it's getting increasingly impossible to notice the negative externalities of sharing our data, precisely because of personalization. All I can see is how my chatbot talks to me, what content is in my feed, or simply a price. I don't know how much of my personal information is utilized in the answers they give me, and whether it is to my benefit or detriment.
The most dangerous kinds of change happen in non-obvious ways, slowly over time. And I think personalization is a prime example of this, which create silos that subtly change what we think over time. I personally would be more amenable to sharing all my data if I didn't rely on their products for almost all of my information, and therefore significant power in dictating what I think and what I do.
Our time spent on personalized surfaces and silos is also only increasing, as a direct result of AI progress. Search has already been replaced by default with LLM responses and chatbots, an exponentially more personalized medium (and revealing, because of the followups that we ask). Elon also shares user activity growing exponentially on X:
Cambridge Analytica foreshadows the power of combining personal data, psychological profiling, and social-media targeting. When companies control the full feedback loop, they can expand from predicting human behavior to activating it.
Think about an LLM with full disk access. The company can see everything the user sees, listen to what they think about it, then observe their resulting actions. Users actively ask the LLM questions, the highest possible signal data to make user inferences from. Then the LLM responds, and companies can directly observe the impact of the words that the LLM has said.
Targeted advertising is a business of paying money to reach the audiences that are likely to change. In a future where the surface area of LLMs increase, I think that there will be a lot of room to grow for achieving advertiser's goals through human language. It could look something like companies A/B testing the persuasive effect of different strings of words for increasingly higher dimensional personality types. It doesn't seem so far from what Cambridge Analytica was doing 10 years ago, and what ad models do today.
From Meta's Q2 2026 earnings call:
In Q2, we advanced our user understanding models to analyze ads and organic activity and simultaneously improved both user experience and advertiser performance.
Rather than scoring every possible ad individually, we are now using LLMs to reason about ad content and user preferences together and predict the best ad for each person.
This includes building foundation models that are designed to power organic content and ads recommendations simultaneously as well as developing LLM native recommender systems.
It's possible that, motivated by such advertising profits, we will soon have an AGI that knows what words, in what order, at what time works best for activating target actions for each person. Both humans and advertising companies already know this to some extent today, but the resolution and surface area is increasing with AI progress.
This is scary, but this is also very exciting. Because words also have an incredible ability to touch, persuade, encourage, motivate, and change people to help them achieve their own goals.
Lewis Hamilton recently won the 2026 Spanish Grand Prix, his first win for Ferrari, and his first win since 2024. Two months earlier, a long-time fan saw that Lewis looked a bit down, which prompted him to shout from the crowd: "Lewis, remember who you are. You did it. You will do it again. So much love for you man. So much love." In interviews, Lewis shared that he was thinking about what the fan said all week, and when he won, he captioned his victory post with the fan's words: "REMEMBER WHO YOU ARE".
In some ways, LLMs are a uniquely human technology because it speaks our language.
To remind us of the original question, it's worth thinking about the commercial paths for which every human being having access to an LLM 24/7/365 is a sustainable possibility. My boring answer is, if the future is something like today, there are two primary businesses and business models that can put LLMs in the hands of every human being on earth: Meta and Apple. Meta can bring it to everyone in the world for free, and Apple can bring it to everyone at a price. This is the price of personalization.
Even if LLM surface area, AI investment, and AI progress stops today, I think better human prediction is already inevitable because it is both probable to achieve, immensely profitable, and existing businesses are well-positioned to make progress on it. Most importantly, because we found a way to get the data. LLMs are the best tool humanity has ever had for eliciting valuable and honest human data, whether it is in seeking the daily mentorship from leading mathematicians and engineers, or giving full disk access to our personal and emotional lives. LLMs invite us to ask the questions we are afraid to ask.
In a world where LLM intelligence stops today, it seems unnecessary to give companies access to your email, and therefore your entire life, to find a few invoice errors. 99% of personal assistant tasks can be achieved by privacy-preserving local LLMs, and I pay this price of personalization, which is reducing each day.
In a world where human prediction gets to be so personal that it is indistinguishable from human understanding, it's power can be controlled by individuals through privacy technologies that allow them to verify the ways in which their data is used.
This type of privacy technology goes by the names Trusted Execution Environments (TEE) and Confidential VMs (CVM). They come in all of Nvidia's GPUs, and AMD makes them too. Private inference for open-source models, where you can verify both privacy and the model being served, is already live today at the same API price as non-private.
TEE technology enables users to share all their data in order to harness the full power of LLMs, without inviting in the people and companies that do not have our best interests at heart at the same time.
I'm excited to build the LLM use cases that are only possible with full, safe access to a user's life. I want to end by taking a moment to talk about such dreams.
Here is an excerpt on coaching from Li Sun, the current head coach of Chinese National Table Tennis team, who is responsible helping three very different women achieve Grand Slams:
What was the key to shaping Li Xiaoxia?
Li Sun: She is not a person with a huge ambition, not aggressive, and with a gentle and timid character, like a little girl. She is unlike Wang Nan, who is doughty in her life, and she lacks the fortitude that Zhang Yining has. When she was young she even said she would be satisfied if she took part in and win a team event. I was shocked by her words.
During all these years I have been fighting against her character. I tried to set higher goals for her, or else she could be sidelined after winning the team's title in the 2006 WTTC. In fact till the year of 2010, we were still not on the same track. I set high requirements on her, but she thought she was doing well enough already. I had to make sure she is provided with "a champion's heart", not going back home cheerfully after paltering with one or two gold medals.
During the 2010's Guangzhou Asian Games, I set a high target for her without exchanging ideas much with her. I thought: it is enough for you to follow my instructions and I'd guarantee you a good result. One day she was pushed too hard by me and said in front of others, "I'm not Zhang Yining." And I immediately knew something severe happened. She is more emotional, more thinking, especially sensitive to the views of the outside.
It reminded me that we didn't have enough trust yet. The key for a coach is to be trusted by their team members, to be believed in that you can help them to do well.
Then I talked with her everyday. When she tended to be worried, I managed to brighten her mood. In training, I myself helped her to do the stretching and recovery exercises, showing consideration to her. These young girls, with their parents away, are confronted with intense competitions. You've got to let them know you care for them, and then trust can be built step by step.
Working with girls is kind of like playing house. They are sensitive. Consolation or a white lie is necessary at times. You must let them know you want to do good to them, and of course, you should let them win on the aspect of technique, then they will give you their trust.
Once she could clearly distinguish that I was doing this for her own good, we were heading toward the same goal, so there were no real conflicts. Actually, we coaches understand her better than her parents do. So sometimes parents couldn't accept certain coaching methods, but Li Xiaoxia would understand. She accepted things quickly, because she also really wanted to do this well.
I was also working on the mentality and perspective of girls. Most girls tend to play passively, thinking a lot before and after. For example, when it comes to attacking, they seldom think "I can make a kill within one shot", while they are likely to consider what if they didn't get ready in case that their opponents return it. So I direct them to consider problems more actively, and change their way of thinking subtly.
Over time, coaching Li Xiaoxia, I could predict what mistakes she would make and what problems she would have. I had already thought through how to handle them for her.
After Li Xiaoxia broke through at London, she was completely different. Her requirements for herself were even higher than mine. For Li Xiaoxia now, I focus more on making her playing style perfectly match her personality. Especially for girls — whatever your personality is, choose a playing style that suits your personality. Li Xiaoxia is sometimes a bit impatient and doesn't control rhythm very well, so I focus on controlling her playing style and direction. In terms of technical details, she's definitely better than me, so my help for her isn't that much anymore. If a coach only thinks about teaching a high-level athlete technically, many problems will actually arise. So my coaching philosophy is never about molding you into a certain shape. It's about helping you discover your potential to achieve our shared goals.
This is a story of helping someone achieve their true potential, that they don't yet see in themself. A reminder of the immense value humans can provide one another.
It relevantly explores the subtle importance of caring for each other in how humans build trust, achieve alignment, and successfully pursue shared goals. And how humans can understand people who think differently than us, by combining keen observation with reasoning through empathic simulation.
It also gives us a peek into what truly high-level coaching looks like: meticulous planning and consideration across a decade, growing together to accommodate what a player needs, which is necessarily different at each stage of growth.
So I think this story is a good example of a ceiling for what humans are capable of given the same amount of context we are well on track to give AI.
AI might not understand or coach us in the ways that humans do, but transformers have their own way of building up a deep understanding of humans, predicting our mistakes and problems, and thinking about how to solve them.
As technology improves, LLMs will be increasingly capable of modeling how different individuals think. They will be able to reason about what each person needs at any given moment, and how to tell us in the way that clicks. They will be able to give us perspective on our blind spots, and remind us what the most important problem to solve right now is.
They will follow us across time, accompanying us every day, setting us up for positive feedback loops and helping us get back on track when we're stuck, until we finally reach our individual breakthrough points.
Great luck was involved in Li Xiaoxia and Li Sun's 10 year journey together to complete the Grand Slam, but I hope that LLMs can make each one of us a little bit more lucky in the years to come.