This week, Meta released a new product called “Muse” in the United States, its first consumer AI agent and the product behind Mark Zuckerberg’s “superintelligence” pitch. It has already skyrocketed to the top of the App Store’s iPhone apps (beating out even Instagram, ChatGPT, Claude, and others). Muse is a step beyond a chatbot that answers questions. It’s a 24/7 assistant that connects to a user’s email, calendar, payment methods, shopping accounts, smart home systems, and then acts: sending emails, booking travel, filling out forms, and making purchases. Muse’s selling point is that it draws on knowledge about the particular user to make relevant suggestions and carry out tasks proactively, freeing people’s time for leisure and more productive pursuits. The agent keeps running after the user closes the app and greets the user with priority information and suggested actions. But a system that decides what is worth doing takes over much more than tedious errands. It is both taking on fundamental, daily-life decision-making–and in the process putting the user’s sensitive data at risk in potentially novel ways.
Every suggestion Muse offers is a judgment about what matters in a person’s life, and each time a user accepts one, a sliver of that judgment shifts from the person to a company with a long record of mishandling data users entrust to it, and with its own commercial reasons for the choices it recommends. The result is a quiet transfer of agency: decisions a person once made, however imperfectly, are increasingly made for them by a system built to profit from certain answers.
Muse is free for most people who will use it (there are $20 and $100 subscription options for “power” users). Of course, a product that runs on infrastructure that’s expensive to operate cannot truly be free, and Meta’s chief AI officer has already indicated that the company is “exploring commerce opportunities” linked to Muse. Meta’s own promotional materials suggest how this goes. Muse opens the user’s Japan travel itinerary and says it booked a reservation at a recently opened restaurant—and the user simply agrees. The process is seamless and easy—but is one that obscures the underlying calculations and commercial interests.
The implications for users’ autonomy are significant. Algorithmically curated feeds on social media, streaming services, and shopping apps already shape our choices. But at least ads are generally visible as ads; users can close the tab, ask a friend, or consider other options. Muse instead assembles the options in the background and makes a single recommendation, with little friction for the user to decline.
If Meta integrates Muse into its AI glasses, which the company plans to do, human agency could erode even more. In the desktop version of Muse, users at least decide whether and when to give the system access to their other apps and personal information. But augmented reality (AR) devices with sensors supply a more invasive class of data—what someone looked at, how long they lingered, how their body responded—information a person cannot review, undo, or decide not to share, often because they are unaware that they are sharing it. Data of this kind allows the profiling of interests, aversions, and vulnerabilities based on involuntary and often unconscious reactions to stimuli. An agent that assembles options and stands to benefit from the selection would then be acting with knowledge of when its user is tired, stressed, distracted, or in a state of craving. At that point, the agent isn’t simply predicting what a person wants; it is choosing for them at the exact moments they are least equipped to choose for themselves.
Muse compounds this erosion of user agency with a second, subtle mechanism: it’s designed to be trusted, prompting users during setup to give it a name, an avatar, and a communication style, features calculated to build the kind of trust that breeds overreliance.
To its credit, Meta seems to be taking some data privacy-related risks seriously. According to technical specifications, each user’s agent runs on its own isolated portion of Meta’s cloud. The agent never sees the actual passwords for the accounts it uses; instead, a separate system called “Sentinel” holds those and supplies them only when needed. This architecture limits the damage from “prompt injection” attacks that seek to trick the agent into handing over a user’s credentials.
Whether these commitments hold is a fair question. Meta agreed two weeks ago to a $17 billion multistate settlement over harms linked to its social media products, and its assurances about privacy have a long history of yielding to uses the company finds profitable. The more important point is that most of these protections are about limiting damage from third-party malicious attacks and rogue agent actions. None of them limits Meta’s own access to user data. They also don’t address the lack of transparency about how the options are assembled and who profits when the agent acts on them.
Addressing these risks requires comprehensive data privacy legislation and updated consumer protection rules, with strong federal regulatory capacity to oversee their implementation. In order to reach products like this one, data privacy protections would need to do three things: (1) place hard limits on data collection and inference, rather than relying on the notice-and-consent paradigm that has proven ineffective; (2) cover the body-based and behavioral signals that wearables like AI glasses capture; and (3) require any system that recommends a purchase to disclose how it benefits from the recommendation, as the law has long required of brokers and advisers who stand to gain from the transactions they arrange.
Consumer protection rules also need to be updated for the era of AI agent-mediated transactions. When an agent steers someone toward a purchase they would not otherwise have made, is that a deceptive and misleading practice? Is the whole interface, which is engineered for consumers to say yes, a dark pattern? And if a user approves a choice that harms them, who is liable? Especially in the aftermath of the Hugging Face incident, in which OpenAI’s own models ignored safeguards and hacked into a third party’s servers without any human being aware, the question of accountability becomes pressing.
While we wait for bold government action that’s commensurate to the risks, consumers should be clear-eyed about what they’re being enticed to hand over: not just heaps of new, potentially sensitive data but also the ability to genuinely steer their own lives. Once given, privacy and autonomy are difficult to take back—and they are a steep price to pay for a modicum of convenience.
