# Claude Code (`claude_code`)

> The `claude_code` single-agent baseline, run inside the **task's own container image**: Claude edits and evaluates in exactly the environment the framework scores in. A **single container** holds both the agent and the trusted scorer — the agent runs non-root, `/eval` is root-owned — and its paths are generic (`/workspace`, `$HOME/.claude`) so the recorded trajectory reads like any reference-agent run.

## Overview

The scaffold places itself where the evaluation happens. For a `docker`-mode task (the default) the session runs in a container — but not a generic one. The container is built from the **task's own image**: the task's `Dockerfile`, or one synthesized from the card's `language` + `library` on top of `galapagos-task-base` (which ships `open-galapagos` preinstalled). The Claude Code CLI is then layered onto that image, which is Harbor's *installed agent* model rather than SkyDiscover's *runner image* model. A run scored on the host (`general.eval_mode: local`) gets the session as a host subprocess instead.

The difference is the whole point. A fixed runner image with a frozen scientific-Python stack cannot satisfy every task, so the agent's own `run_eval.sh` breaks — or worse, silently returns a constant zero — the moment a task's evaluator needs something the runner lacks. Here the agent's `run_eval.sh` invokes the framework's own trusted runner against the task's own `evaluator.py`, through the same interpreter and the same packages the recorded score comes from. **The number the agent sees is the number that gets recorded.**

The session runs with `--dangerously-skip-permissions`, which on the host means unrestricted access to *your machine*. In container mode that access is confined to a container torn down at the end of the run. Unlike SkyDiscover's Docker-in-Docker design, there is one Docker daemon, no `--privileged`, and no `docker save`/`load` dance. **The one exception is a `docker_access` task (ALE-Bench), where the container is deliberately not a sandbox — see below.**

**Billing is structurally subscription-only**, not just policy: a container starts with *no* environment unless a variable is explicitly forwarded with `-e`, so `ANTHROPIC_API_KEY`/`ANTHROPIC_AUTH_TOKEN` can never reach the CLI even by host accident. Auth comes from `CLAUDE_CODE_OAUTH_TOKEN` (`claude setup-token`) or a copy of the local `claude /login` credentials placed into the CLI's default `$HOME/.claude` (deleted from the preserved trajectory when the run ends, so a training dataset never carries it).

## One container, a uid boundary

There is a **single container**. The CLI session and the framework's trusted scorer share it: the scorer `exec`s in to record the score-of-record (an *attached* evaluator that never tears the session down), and the agent's own `run_eval.sh` invokes the *same* trusted runner. Isolation rests on ownership, not a second container — the agent runs **non-root** while `/eval` (scorer + data + runner) is **root-owned and unwritable**, so the agent can read the scorer but never rewrite it, and (being non-root) cannot signal or `ptrace` the scoring process. This is Harbor's *shared* topology hardened by a uid boundary, and it drops the sibling-container overhead of a *separate* verifier. The recorded score is always recomputed by the framework — **never Claude's self-reported number.**

## The ALE-Bench exception: where the container is *not* a sandbox

The uid boundary above protects the *score*. It does not protect the *host* on one task family, and that is worth stating plainly.

The 40 ALE-Bench `ahc*` tasks declare `evaluation.docker_access`, because their evaluator is itself a Docker client: `ale_bench` compiles and judges each candidate inside **sibling containers it launches on your daemon**. A container that scores such a task therefore carries the host's docker socket (plus a path-parity directory, without which the daemon resolves the judge's bind mounts against the *host*, they come up empty, and the task scores zero with nothing raised — see `environments/dood.py`). A socket is root on the host, so on an `ahc*` task **the agent can reach your daemon, and therefore your host.**

That is deliberate and unavoidable: `ale_bench` exposes no judge *service* to call — only containers it launches — so an agent that cannot reach the daemon cannot see its own score. Keeping the agent out buys nothing back, either. The alternative is the host loop, where the agent runs as *you* — and you must already hold rw on that same socket for the preflight to pass. Same reach, no container; and worse, the agent's `run_eval.sh` and the framework's recorded score would then come from different environments and could silently disagree.

Run `claude_code` on `ahc*` only on a host you are willing to expose to the agent. Everywhere else, the isolation story above holds.

## Generic paths (for reusable trajectories)

`HOME` and the workspace are `/workspace`, and no galapagos-specific `CLAUDE_CONFIG_DIR` is set, so the CLI writes its native session JSONL to the **default** `$HOME/.claude/projects/**.jsonl` — exactly where a plain Claude Code session writes it. The recorded trajectory therefore carries generic paths (`/workspace/solution.py`, `/eval/…`) rather than `/galapagos/…`, which keeps it reusable as training data with no post-hoc path rewriting.

## Algorithm

1. **Resolve the environment** — `task.image_spec()` → `resolve_image()` builds (or pulls) the task's image; `build_agent_image()` layers Node + `@anthropic-ai/claude-code` onto it. Both are content-addressed, so the slow apt+npm layer is paid once per task image, and tasks with the same dependencies share one image.
2. **Start + provision** — One container from the agent image, with the workspace bind-mounted read-write at `/workspace` (the CLI's `HOME`, so its native trace lands at the default `$HOME/.claude`; not `/run`, a real FHS directory in the Debian-derived images a task may choose). The task's files and the trusted runner are copied to `/eval` as root, so the non-root agent can read the scorer but never rewrite it. A GPU task's `constraint.gpu` becomes `--gpus`, so Docker refuses loudly on a host without the NVIDIA Container Toolkit instead of handing a kernel task a CPU-only container.
3. **Score the seed** — In the container the run will be scored in, so the baseline is measured where every later candidate is.
4. **Delegate** — `docker exec` runs `.run.sh`, which records its own PID and then `exec`s `claude -p - --max-turns <cli cap> --dangerously-skip-permissions --output-format stream-json`, reading the brief from `.prompt.txt`. Claude iterates freely: evaluate the baseline, edit, re-evaluate, keep or revert.
5. **Watch** — The stream is parsed for turns and cost. A hard stop or wall-timeout signals the recorded PID rather than killing the container — which still holds the evaluator the framework needs to score the final solution. The solution file is polled every 10s; each new version is scored with the task's Evaluator and admitted to the Population as a checkpoint genome.
6. **Finalize** — The final solution is scored the same way. The best-scored snapshot — never Claude's self-reported score — is the run's result; `run_summary.json` records turns, cost, auth source, the task and agent image tags, native session file paths, and baseline vs. final score.

## Components

One CLI session subsumes selection, prompting, and proposal, so only the Population does real work — it archives the scored checkpoints.

| Slot | Implementation | Role |
|---|---|---|
| Population | `InMemoryPopulation` (keep-all) | Archives every scored checkpoint; supplies the run's best |
| SelectionPolicy | delegated to the agent | Claude decides what to try next inside its session |
| PromptBuilder | one-shot `TASK.md` brief | Written once at staging; container paths |
| Proposer | the Claude Code CLI, inside the task image | Edits the solution file in place over the prompt budget |
| Evaluator | task-supplied, `ContainerEvaluator` | Scores the seed, every checkpoint, and the final solution — in the agent's own container, attached (root-owned `/eval`) |
| Memory | none | The session's own context is the only memory |

## Configuration

The session knobs, plus the Docker-specific ones:

- `general.max_iterations` (100) — prompt-level evolution budget.
- `proposer.claude_cli_max_turns` (null) — CLI `--max-turns` safety cap; null = `2 * general.max_iterations`.
- `proposer.claude_wall_timeout_seconds` (null) — whole-session wall-clock timeout; null derives `claude_cli_max_turns × (300s + eval timeout)`, minimum 30 min.
- `proposer.claude_model` (null) — the CLI's `--model`; must be a Claude name.
- `proposer.reasoning_effort` (high) — the CLI's `--effort`.
- `proposer.claude_docker_image` (null) — override the task image wholesale with a prebuilt tag; null resolves it from the task card.
- `proposer.claude_docker_cli_version` (null) — pin `npm install -g @anthropic-ai/claude-code@X`; null takes the latest.
- `proposer.claude_docker_memory` (null) — `docker run --memory` cap (e.g. `"4g"`); null = unbounded.
- `proposer.claude_docker_cpus` (null) — `docker run --cpus` cap; null = unbounded.
- `proposer.claude_docker_network` (null) — `docker run --network`; null uses the task card's `evaluation.network`, else Docker's default bridge (the CLI needs network access to reach Anthropic's API).

Requires Docker (daemon reachable by the current user) plus the same subscription auth as `claude_code`. Images are built automatically on first use. The base image rebuilds when the galapagos package sources change (its wheel digest is stamped as a label); force one with `docker rmi galapagos-task-base:<version>`.

## When to use

Use `claude_code` for a single-agent baseline: one strong agent, free to iterate, against the task's real evaluator — the natural upper bound to compare a structured search method against. It is also the least fussy scaffold to run, since the task's own image supplies every dependency the evaluator needs and no provider key is involved.

Prefer the structured scaffolds (`openevolve`, `topk`, `best_of_n`, …) when you want controlled populations, reproducible per-iteration prompts, or provider-agnostic models. And read the ALE-Bench exception above before pointing this at an `ahc*` task on a host you care about.

## Source

Galapagos's own synthesis of two upstream designs. SkyDiscover's `claude_code` controller (`skydiscover/search/claude_code`) always ran the CLI in Docker, and for tasks with a containerized evaluator it started a nested `dockerd` (`--privileged`) and `docker save`/`load`-ed the evaluator image into it. Harbor instead has the task own one image and *installs* the agent into it (`harbor.agents.installed.claude_code`), with `VerifierEnvironmentMode.SHARED`/`SEPARATE` choosing whether the verifier reuses that container or gets a fresh one. This scaffold takes Harbor's installed-agent model and its *shared* topology — one daemon, no `--privileged` — hardened by a uid boundary (non-root agent, root-owned `/eval`) so a single container is safe without the sibling-verifier overhead, while keeping `claude_code`'s subscription-only billing. Native session capture uses the CLI's **default** `$HOME/.claude` (no redirected `CLAUDE_CONFIG_DIR`) so the trajectory's paths stay generic and reusable as training data.