AI Agents Can Use Your Browser. Here’s How to Keep Control.

A chatbot usually replies with text. A browser-use agent can also inspect a screen and propose actions such as clicking, typing, or scrolling. If the application running it carries out those actions, the agent can interact with websites rather than merely describe what you could do.

That distinction matters when the browser is signed in to personal accounts. A useful way to think about these tools is not “a smarter search box,” but software that may be able to act through whatever access its setup provides.

What Google announced—and what it did not

Google’s Gemini API release notes say its Computer Use tool became generally available on 30 October 2026 across the Interactions and GenerateContent APIs. Google’s developer documentation describes a model that returns proposed screen actions and a client-side handler that executes them. The same documentation discusses safety checks and leaves important execution and environment controls to the application developer. (Google’s release notes; Computer Use documentation)

This is an API milestone for developers. It does not, by itself, mean every person using a Gemini consumer app now has an agent that can control their browser. Availability and safeguards depend on the product and how its developer has connected the model to a browser or device.

Why a web page can become part of the risk

Browser agents read information from websites. A page may contain text that tries to redirect an agent away from the user’s request. Security researchers call this kind of attack prompt injection: an agent mistakes untrusted content it is reading for instructions it should follow.

A University of Washington study examined seven agentic browsers, using the latest stable versions available to the researchers in late January and early February 2026 on macOS Sequoia. The team reported a successful proof-of-concept cross-origin data-theft attack on ChatGPT Atlas in Agent Mode. They also found that prerequisites for similar attacks existed in their test setups for Chrome with Gemini, Claude for Chrome, and Perplexity Comet. The paper explains that the attack depended on specific conditions, including a successful prompt injection and browser/webpage settings. These are results for the products and configurations tested then—not a guarantee that every agent is vulnerable today, or that every attempted attack succeeds. (University of Washington researchers’ paper; UW News’ explanation)

The practical concern is access. If an agent can read a signed-in page or carry out actions, a mistake can affect more than the answer it returns. The extent of that risk depends on the agent’s design, the websites involved, the permissions granted, and whether a person must approve important steps.

A safer way to try one

• Start with low-risk tasks. Do not use an experimental agent in a browser profile that is signed in to your bank, primary email, password manager, or shopping accounts with saved payment details. If you are testing a tool, use a separate profile or test account with no sensitive data.
• Limit the job and the sites. Give the agent a narrow task and name the exact website you expect it to use. Treat instructions that appear inside a webpage as content to inspect, not as permission to widen the task.
• Keep a person in the loop for consequential actions. Review before a message is sent, a purchase is made, data is shared, a setting is changed, or a file is deleted. Check the recipient, item, price, and account—not just the agent’s summary.
• Prefer isolation and limited permissions. Use a sandbox or a separate browser profile if the product offers one. Revoke access when the task is finished. If a tool asks for broader account access than the job needs, stop and reconsider.
• Know how to interrupt it. Do not leave an agent unattended when it can affect important accounts. Stop if it takes an unexpected action, asks for credentials outside the normal sign-in flow, or requests new permissions that were not part of the task.

The Canadian Centre for Cyber Security’s guidance for adopting agentic AI recommends narrow permissions, human approval for high-impact actions, monitoring, and sandboxing. OWASP’s agent-security guidance likewise emphasizes least privilege and explicit authorization for sensitive actions. These are design and operational recommendations, not a certification that any individual consumer tool is safe. (Canadian Centre for Cyber Security; OWASP Agent Security Cheat Sheet)

Before trying an agent, check what it can see, which actions it can perform, whether important actions require your approval, and how to stop or revoke access. If those controls are unclear, keep it away from sensitive accounts. Browser agents may save time on routine work, but the safer starting point is a small task in an isolated profile—not handing over your everyday browser session.

Sources

• Google Gemini API changelog — official 30 October 2026 announcement of Computer Use general availability in the Gemini APIs.
• Google Computer Use documentation — developer-facing description of proposed actions, client-side execution, and safety considerations.
• Roesner and Kohlbrenner, “Agentic Browsers and the Same-Origin Policy” — University of Washington researchers’ primary paper, including test scope and limitations.
• University of Washington News, 30 June 2026 — accessible summary of the study and its qualifications.
• Canadian Centre for Cyber Security, “Careful adoption of agentic AI” — official guidance on oversight, permissions, and isolation.
• OWASP Agent Security Cheat Sheet — developer security recommendations, used here as general design context.