Installation

Requirements

Python 3.10, 3.11, 3.12 or 3.13. CI tests all four.

Install

Three ways, in order of least commitment:

Try it with uvx, no install at all:

uvx --from agent-arena arena --version

Install from PyPI:

pip install agent-arena

Provider SDKs are extras, because the engine needs none of them: pip install "agent-arena[anthropic]", or [openai], [gemini], [litellm], [all].

Or install from source, which always tracks main:

git clone https://github.com/adityamhaske/agent-arena
cd agent-arena
pip install -e .

Any of the three gives you the engine and the CLI. One dependency: PyYAML, and even that is optional — JSON config works without it.

Docker

docker build -t agent-arena .
docker run -p 8420:8420 -v $(pwd)/projects:/data/projects agent-arena

arena ui has no authentication by design, and inside a container reaching it at all requires binding 0.0.0.0 — which the image does. That means anyone who can reach the published port can read and edit every mounted project and spend API credit through it. Put it behind your own network boundary; never publish the port straight to the open internet. See ../security/hardening.md.

For one-shot CLI use instead of the UI, override the command:

docker run -v $(pwd):/data agent-arena evaluate --project projects/my_project

Devcontainer / Codespaces

.devcontainer/devcontainer.json is in the repo. Opening it in VS Code or a GitHub Codespace runs pip install -e ".[dev]" automatically and forwards port 8420 for the UI.

Extras

Provider SDKs are optional and import lazily. Install only what you call.

pip install -e ".[anthropic]"
pip install -e ".[all]"
pip install -e ".[dev]"

You do not need any of them to start: mock: and local models work with the base install, and examples 1, 2 and 4 run fully offline.

If you call a model whose SDK is missing, the error names the exact fix:

the 'anthropic' provider needs the 'anthropic' package:
pip install 'agent-arena[anthropic]'

Verify

arena --version
arena scorers
arena evaluate --project projects/support_triage --quiet --no-report

The third proves the whole path works offline with no credential. If it produces a leaderboard, the install is good.

For contributing

pip install -e ".[dev]"
python3 -m pytest -q        # 715 passed in ~44s

Use a fresh virtualenv if you want to verify the stdlib-only invariant. An environment where you previously installed a provider SDK will not prove it.

The documentation site

pip install -r site/requirements.txt
python3 site/build.py
python3 -m http.server -d site/_build 8000

site/requirements.txt is build-time only. It is not a runtime dependency and does not affect invariant 1.

Uninstalling

pip uninstall agent-arena

Settings and any stored credentials live in ~/.config/agent-arena/; delete that directory to remove them. Project folders and their results are yours, and nothing outside them is touched.