It is necessary to give Terminal (or iTerm or whatever you use) the permission to control the computer. This can be done in System Settings ➔ Privacy & Security ➔ Accessibility.
- Install
cliclick
for mouse & keyboard emulationbrew install cliclick
- Clone Anthropic quickstart repo
git clone https://github.com/anthropics/anthropic-quickstarts.git
cd computer-use-demo
- Replace
computer-use-demo/computer_use_demo/tools/computer.py
with the modified version below