Self-Operating Computer Framework SOP

Appearance

Install ​

Prerequisites ​

  • Python 3.9 or later
  • pip
  • An API key for at least one supported model provider (OpenAI is the default)
  • macOS, Windows, or Linux with an X server

Install via pip ​

bash
pip install self-operating-computer

This installs the operate CLI globally.

Model-specific setup ​

OpenAI (default: gpt-4o) ​

Set your key or enter it when prompted on first run:

bash
export OPENAI_API_KEY=sk-...

Your OpenAI account must have spent at least $5 in API credits before gpt-4o access is unlocked.

Anthropic Claude 3 ​

bash
export ANTHROPIC_API_KEY=sk-ant-...

Google Gemini Pro Vision ​

Obtain a key from Google AI Studio and export:

bash
export GOOGLE_API_KEY=...

Qwen-VL ​

Obtain a key from the Alibaba Qwen dashboard and export:

bash
export QWEN_API_KEY=...

Local LLaVA via Ollama (free, macOS/Linux only) ​

  1. Install Ollama
  2. Pull the model: ollama pull llava
  3. Start the server: ollama serve
  4. Run SOC: operate -m llava

macOS Permissions ​

After the first operate invocation, macOS will prompt you to allow Screen Recording and Accessibility access for your terminal app. Grant both in:

System Settings > Privacy and Security > Screen Recording and Accessibility

Then relaunch your terminal and run operate again.

Voice Mode (extra step) ​

Voice input requires cloning the repo and installing the audio extras:

bash
git clone https://github.com/OthersideAI/self-operating-computer.git
cd self-operating-computer
pip install -r requirements-audio.txt
operate --voice