Automating PC Operations with Computer Use
Summary — Key Points of This Lesson
- Computer Use is a feature that allows Claude to recognize screenshots and control the mouse and keyboard.
- Claude reads screenshots to understand the current state of the screen and determines the next action (click, input, scroll, etc.).
- This is a feature accessed via the API and is different from the standard Claude.ai chat — it is designed for enterprise users and developers.
- Anthropic positions Computer Use as a beta feature, and users are responsible for verifying its accuracy and safety.
- Use cases include: repetitive data entry, cross-application information gathering, and assistance with operating legacy systems.
目次 (5)
What Is Computer Use?
Computer Use is a feature that allows Claude to "see" and operate a computer screen. Claude receives a screenshot (screen capture), understands the current state, and then instructs actions such as mouse clicks, drags, keyboard input, and scrolling Source.
This enables Claude to handle workflows involving legacy systems without dedicated APIs, or workflows that require operating multiple applications in sequence. However, the actual execution on the computer is handled by external components — defined as "tools" — such as screenshot capture and click execution; Claude itself does not directly control the OS.
How It Works: The Screenshot Recognition Loop
The basic operation cycle of Computer Use is as follows:
- A user or system assigns a task (e.g., "Open the spreadsheet and enter the data").
- A screenshot is passed to Claude.
- Claude analyzes the screen content and returns the next operation to perform (e.g., clicking at a coordinate, typing text).
- An external tool executes that operation and passes the resulting new screenshot back to Claude.
- Steps 2–4 repeat until the task is complete.
How to Use It and Its Limitations
Computer Use is a beta feature provided through the Anthropic API (claude.ai API / Amazon Bedrock / Google Cloud Vertex AI) Source. It is not a feature that allows you to directly operate a PC from the standard Claude.ai chat interface.
Anthropic officially notes the following:
- Computer Use is in beta, and its accuracy and behavior may change.
- When operating in environments that handle critical systems or sensitive information, risks must be thoroughly assessed before use.
- It is recommended to have a human review in place to confirm what actions will be taken before execution.
Relationship with Claude Cowork
Claude Cowork, which Anthropic made generally available (GA) in April 2026, is a product that packages the concept of Computer Use for knowledge workers. It is designed to autonomously operate files, folders, and applications to deliver results, and can be used from a desktop app. For more details, see the Claude Cowork guide article and the Managed Agents architecture overview, which covers the background of agent design.
Use Cases
- Repetitive data entry tasks: Entering standard data into existing forms
- Cross-application information gathering: Copying content found via browser search into another tool
- Assistance with legacy system operations: Accessing older systems without an API
- Screenshot-based testing: Performing checks while visually verifying the UI's appearance
