An AI that can answer a question is one thing. An AI that can move through a browser, fill a form, open an app and take the next step is another.
Google recently added computer use as a built-in capability in Gemini 3.5 Flash for agents that can interact across browser, mobile and desktop environments. Its accompanying safety guidance is revealing: Google offers safeguards that can require explicit confirmation for sensitive or irreversible actions, and can stop tasks when an indirect prompt injection is detected. It also recommends sandboxing, human verification, and strict access controls. This analysis is supported by Google Gemini computer-use model, Google Android Gemini security and privacy, and OpenAI GPT-5.6 announcement.
The technology will improve quickly. The useful question for a person or business is already here: where should the agent stop and hand the decision back?
Separate the four kinds of action
Most workflows become easier to govern when they are divided into four stages.
Read
The agent opens a page, scans a document, or finds the relevant information. Reading may still need permission, especially with private files, but it does not create an outside commitment.
Prepare
The agent drafts an email, fills a form without submitting it, creates a comparison, or organises a shortlist. This is often the best place for an AI to save time. The output remains visible for review.
Propose
The agent identifies the next action: a booking, a supplier message, an approval request, or a change to a business record. The proposal should show the affected system, the source information, and the likely consequence.
Commit
The agent sends, buys, publishes, deletes, changes an account, or updates a record. These actions deserve a different level of friction because they are harder to unwind.
An approval boundary should normally sit between propose and commit. That keeps the assistant useful without confusing speed with authority.
Do not train people to click through warnings
A badly designed agent asks for approval too often. A user approves a harmless page view, an internal search, a draft, a notification, and a dozen other low-risk events. Soon the final purchase or external message looks like one more prompt.
A better design shows the work as it happens, but reserves explicit approval for a material boundary: sensitive data, external communication, a financial commitment, a permissions change, or a system-of-record update. Google describes a similar approach for Android automation, including visible progress and user confirmation for purchases.
What an approval screen should show
At the moment of decision, a user should be able to answer four questions in seconds:
What will happen?
Which account, app, or organisation is affected?
What information will leave the current context?
Can this be reversed, and by whom?
If the screen does not answer those questions, it is not meaningful consent. It is a liability transfer disguised as a button.
VERTU describes Hermes Agent as helping prepare actions across supported services, while significant actions require user approval. Its capabilities depend on device, service, configuration, software version and user authorisation. That qualification is important. A capable agent should make the user more deliberate, not make permission disappear into a sentence like “handle this for me”.
As computer-use agents become ordinary, the premium feature will not be the most aggressive automation. It will be the ability to see what is happening, control which context is used, and approve the actions that carry your name.
Frequently asked questions
What is an AI computer-use agent?
It is an AI system that can interact with graphical interfaces, such as websites, mobile apps, or desktop software, to perform steps in a task.
What should an AI agent be allowed to do without approval?
Low-impact steps such as reading allowed sources, collecting options, and preparing a draft can often run without a final approval. The right policy depends on the sensitivity of the context.
Which actions should require approval?
Purchases, bookings, external messages, document uploads, permissions changes, record updates, and irreversible actions should be reviewed before the agent commits them.
