Computer Use for MacOS 💻

A macOS app (forked from Anthropic's computer use demo for Ubuntu) demonstrating LLM-based computer use on your laptop.

Caution ⚠️

Computer use is a beta feature from Anthropic that has direct access to the internet and sends screenshots of your desktop to Anthropic.

Running the App

1. Ensure you have the latest Python 3 (>= 3.11) and Chrome installed, and an Anthropic API key.

2. Enable Screen Recording permissions so that this application can take screenshots of the desktop.

• Open the Settings app → click on Privacy and Security → click on Screen and System Recording.
  
     →  → 
• Click the little + on the lower-left and add the Terminal application, and click on the toggle next to the app so it turns blue.
  
     →  
  

3. Enable Accessibility permissions so that this application can control your mouse and keyboard.

• Open the Settings app->click on Privacy and Security->click on Accessibility.
  
     →  → 
• Click the little + on the lower-left and add the Terminal application, and click on the toggle next to the app so it turns blue.
  
     →  
  

4. Open the Terminal app and run the script.

# Do this in Terminal App
git clone https://github.com/philfung/computer-use.git
cd computer-use
chmod u+x install_and_run.sh
./install_and_run.sh

  The Computer Use app should automatically open in the Chrome browser.

    

5. Enter your Anthropic API Key in the left panel of the Chrome application.

    

6. Try Computer Use by entering commands!

For example:

"Open cnn.com and click on the latest article"


"Open the Settings app and change to Dark mode"

Talks

• LLMs as Computer Users: An Overview

Resources

For more information, check out my recommended reading list on computer use.