Claude is able to control computer screens

🤯

How computer use works

1. Provide Claude with computer use tools and a user prompt

Add Anthropic-defined computer use tools to your API request.
Include a user prompt that might require these tools, e.g., “Save a picture of a cat to my desktop.”

2. Claude decides to use a tool

Claude loads the stored computer use tool definitions and assesses if any tools can help with the user’s query.
If yes, Claude constructs a properly formatted tool use request.
The API response has a stopreason of tooluse, signaling Claude’s intent.

3. Extract tool input, evaluate the tool on a computer, and return results

On your end, extract the tool name and input from Claude’s request.
Use the tool on a container or Virtual Machine.
Continue the conversation with a new user message containing a toolresult content block.

4. Claude continues calling computer use tools until it's completed the task

Claude analyzes the tool results to determine if more tool use is needed or the task has been completed.
If Claude decides it needs another tool, it responds with another tooluse stopreason and you should return to step 3.
Otherwise, it crafts a text response to the user.

Source: