Task environments — the task owns where it runs

A task's evaluator.py is the same code no matter where it executes; a task environment is the decision of where that code runs — in a host subprocess, or inside a container the task itself declares. Galapagos ports Harbor's central rule: the task owns its environment. One card, one evaluator source, two execution locations; dependencies, hardware, isolation, and timing may differ.

This page documents src/galapagos/environments/, the subsystem that turns a Task Card into an image and starts a container from it.

Writing a task?

Set nothing. Docker is the default and everything below is what that default reaches. Where to run is the operator's knob (general.eval_mode) — the card has no say in it at all.

The image comes from

the task's Dockerfile. That is the whole answer — there is no second source. A task without one cannot use Docker mode; the catalog's sole host-only exception is mlx_metal_kernel_opt.

What runs in it

Normally only the Evaluator. claude_code manages an agent container, Meta-Harness moves its whole loop into the task image, and docker_access tasks move the loop so their evaluator can reach the daemon. See the scaffold-specific sections below.

The sandbox

The candidate runs non-root; /eval — the scorer — is root-owned and unwritable. A candidate cannot poison the next candidate's score.

The failure this eliminates is subtle and expensive: an agent whose in-session scorer disagreed with the framework's trusted scorer because the two ran in different Pythons with different packages. Once the task owns the image, the agent and the framework score through the same interpreter and the same wheels, so the number the agent sees during search is the number that gets recorded.

The switch: docker or local

Where the evaluator runs is a single switch with a single input: general.eval_mode, docker (the default) or local. Set it like any other config value — galapagos run --general.eval_mode local — and task.set_eval_mode() is how it reaches the task from Python.

Nothing else has a vote. Not the card — it describes what a task is, not how this operator wants to run it — and not the environment. A run is therefore reproducible from its config alone: no stray shell variable and no task can quietly move where a candidate is scored.

docker builds a ContainerEvaluator (runs evaluator.py inside the task's container); local builds a SubprocessEvaluator (runs it in a host subprocess). The aliases container and sandbox are accepted as synonyms for docker; subprocess is a synonym for local. Any other value raises.

Docker is the default because the task owns its environment. A card never says where it runs — that is the operator's call, always. One task nevertheless cannot be scored in a container: mlx_metal_kernel_opt (Apple MLX/Metal has no Linux container, so it ships no Dockerfile). It says so in prose in environment.requires; run it with --general.eval_mode local. The two crystal_struct_prediction* tasks ship CUDA-enabled Dockerfiles and download their pinned MP-20 data into a writable evaluator cache, so they use the normal Docker path.

There is no second escape hatch. A host with no Docker daemon is a fact about the host, not about any task, so it passes --general.eval_mode local — it does not edit a card, and it does not export a variable the framework would read.

# nothing set -> the default, docker: the task's own image
galapagos run --scaffold openevolve --task circle_packing --proposer.model_name openai/gpt-4o-mini
# force the host subprocess for this run
galapagos run --scaffold openevolve --task circle_packing --proposer.model_name openai/gpt-4o-mini --general.eval_mode local
# force the task's own container for this run
galapagos run --scaffold openevolve --task circle_packing --proposer.model_name openai/gpt-4o-mini --general.eval_mode docker
# ...or put it in a --config YAML, which is the same thing
#   general:
#     eval_mode: local

There is no bespoke --eval-mode flag: general.eval_mode is a config value like general.max_iterations, and it is set the same way. Unset, it is docker.

import galapagos as gx

task = gx.load_task("circle_packing")
task.set_eval_mode("docker")   # this task object now scores in its container
task.evaluator                 # -> a ContainerEvaluator (the built evaluator is cached & re-derived on change)

How a task resolves to an image

In docker mode, the task resolves to exactly one image tag, and there is no precedence to apply — there is one source:

the task's Dockerfile   → environment.dockerfile, else a sibling Dockerfile in the task dir
                       (built with the task dir as build context, so it may COPY task files)

no Dockerfile          → RuntimeError. A task's image is the file it ships; nothing is inferred.

There is no second source, and that is the point. There used to be several — a prebuilt docker_image tag, a base_image + requirements pair, and a language + library synthesis for a task that shipped no Dockerfile. Every one of them was another way to describe an image the Dockerfile already describes, and two descriptions are two things that can disagree: the Hub copy of an AlgoTune task was synthesized to numpy 2 with no toolbox, while the image it actually ships pins numpy 1.26.4 and carries numba. Same card, same seed, different score. Guessing is worse than refusing.

galapagos submit uploads the Dockerfile, so a submitted task carries its environment with it.

A task Dockerfile must start FROM a galapagos base, not a bare python:* — bundled evaluators import wheel-only sibling modules (galapagos.tasks.kernelbench_common is the load-bearing example) that copying the task directory alone can never satisfy, so a bare python image cannot even import the scorer. The submission gate refuses a task that gets this wrong, because the failure is otherwise a silent zero.

The task files are never baked in

The image is the environment and nothing else — never the task's own files. The task directory is copied into the running container at start-up, so editing a task never invalidates its image, and the same image serves every task that shares an environment (1,273 Dockerfiles → 37 images).

The galapagos-task-base image family

Every task Dockerfile builds FROM one of eight galapagos base images — it must, or the evaluator cannot import the scorer. Each base is built from a shipped Dockerfile under src/galapagos/environments/base_images/, and tagged galapagos-<name>:<version> (e.g. galapagos-task-base:0.4.0).

Base image	Built from	Carries	For
task-base	python:3.12-slim-trixie	build-essential (GCC 14 → g++), ca-certificates, curl, git, and open-galapagos[math] (NumPy/SciPy/Shapely/SymPy/JAX/Optax)	the default. It carries g++, so a C++ candidate can compile even though its evaluator is Python
task-base-all	FROM galapagos-task-base	adds the mutually compatible open-galapagos[all] host union: [algotune] evaluator solvers and [frontiercs]	the uncommon task that genuinely needs the whole union; kept separate so ordinary math and FrontierCS tasks stay thin
task-base-algotune	FROM galapagos-task-base	AlgoTune's own environment: numpy==1.26.4, its pinned scientific stack, CPU torch==2.7.0, and the optimisation toolbox (numba, Cython, pythran, dace, JAX, dask)	all 154 algotune_* tasks; ~5 GB
task-base-cuda	nvidia/cuda:13.0.1-devel-ubuntu24.04	nvcc, cu130 torch==2.9.1 + exact triton==3.5.1, and [math]	KernelBench and current general CUDA tasks; matches KernelBench's source 2.9.x/3.5.x lock
task-base-kernelbenchx	FROM galapagos-task-base-cuda:0.4.0	the current wheel containing kernelbenchx_common, plus upstream's radon>=6.0.0 requirement	all 184 KernelBenchX tasks; isolated so their dependencies do not mutate the shared CUDA base
task-base-gpu-mode	nvidia/cuda:13.0.1-devel-ubuntu24.04	nvcc, cu128 torch==2.7.1 + exact triton==3.3.1, and [math]	the four SkyDiscover GPU MODE tasks; MLA overlays torch==2.8.0 / triton==3.4.0
task-base-openproblems	FROM galapagos-task-base	Open Problems Bio orchestration stack, Java, Docker/Buildx, AWS CLI, and pinned single-cell dependencies	the 16 Open Problems task cards
task-base-sldbench	python:3.13-slim-trixie	the exact SLD-Bench data/scientific lock: datasets 4.0.0, huggingface-hub 0.34.4, NumPy 2.3.2, SciPy 1.16.1	all 8 sldbench_* tasks; separate because upstream requires Python ≥3.13 while Galapagos core supports ≥3.10

task-base-algotune is the clearest case of "the task owns its environment" in the catalogue, because for AlgoTune the environment is the task. The benchmark asks a model to beat a reference implementation on wall-clock time, so the packages it may reach for are the whole game — upstream's own published solutions get their speedups from numba (469 of its 2831 result files) and Cython (290). Deriving the image from what each task's reference happens to import, as galapagos did until 2026-07, left 77 of the 154 tasks with numpy+scipy and nothing to pull.

It also carries AlgoTune's numpy==1.26.4, and that is not cosmetic: lqr's is_solution calls float() on a (1,1) array and delaunay's generator calls np.cross on 2-D vectors — both removed in numpy 2.x — so on galapagos's numpy those two tasks scored a permanent, silent zero with upstream's code byte-for-byte intact. Pinning numpy inside [all] was never an option: that host-compatible union also serves unrelated suites, while each FrontierCS task now uses a thin requests layer on task-base. A benchmark's own pins belong in a benchmark's own image. The build asserts the pin and solves a cvxpy problem rather than merely importing it — the exact failure this image was created to avoid (a numpy-2 cvxpy left behind by a numpy-1 pin) imports fine and dies on first use.

The CUDA bases use a devel image (not runtime) precisely because the evaluator needs nvcc to compile the candidate kernel it is scoring; the container still needs --gpus at run time (see GPU tasks). Their Ubuntu 24.04 system python3 is 3.12, and PEP-668 makes it externally-managed, so they install into a venv on PATH for per-task pip install to work. Both standalone CUDA bases use the CUDA 13.0 toolkit. task-base-cuda takes KernelBench's cu130 torch==2.9.1 / triton==3.5.1; task-base-gpu-mode deliberately takes the audited H100 reference's cu128 torch==2.7.1 / triton==3.3.1. mla_decode declares its upstream Triton 3.4 environment in its own Dockerfile. A CUDA 13 driver/toolkit can run that cu128 wheel. task-base-kernelbenchx is a thin child of task-base-cuda: it keeps those CUDA/compiler pins, but owns the KernelBenchX harness and radon layer so adding or rebuilding the benchmark image does not change other CUDA task Dockerfiles. Its parent is explicitly held at task-base-cuda:0.4.0; a later Galapagos release creates a new KernelBenchX image without rebuilding or overwriting that shared parent tag.

Why Python 3.12 (not 3.13)

This is the kind of decision that reads like an accident until you know the why, so it is worth recording. The base is pinned to python:3.12-slim-trixie — driven by wheels, not by CPython's support calendar:

• open-galapagos[all] carries two exact pins that AlgoTune validated against — python-sat==1.8.dev17 and (via cvxpy) ecos — and neither publishes a linux cp313 wheel.
• On 3.13, pip silently falls back to their sdists and compiles them: build-essential becomes load-bearing and a pre-release source build sits on the critical path of every image build.
• On 3.12, the whole [all] stack resolves to prebuilt wheels (pip install --only-binary=:all: "open-galapagos[all]" succeeds).

3.12 is also the more faithful choice — AlgoTune validated a binary wheel, and on 3.12 we install that exact wheel rather than a fresh g++-14 rebuild of the same version string — and it matches the reference frameworks (SkyDiscover pins python:3.12-slim, Harbor's modal adapter base is 3.12) and aligns with task-base-cuda. The -trixie suffix pins the Debian suite so an unsuffixed default roll (bookworm → trixie → "forky") can't shift glibc/gcc under the whole task fleet.

build-essential is for the task layer, not the base

On 3.12 nothing in the base layer compiles. build-essential is present so a per-task RUN pip install <libs> that hits a wheel-less package still builds, and so a task whose evaluator compiles a C++ candidate in-process has g++ at eval time.

Per-task Dockerfiles and the 31-image collapse

Every bundled task ships its own Dockerfile, and that file — not the card — is what its image is. scripts/gen_task_dockerfiles.py no longer generates them; it only re-pins the FROM version when galapagos is bumped (--check fails CI if any is off-version), because the base tag is version-pinned and 1,273 hand edits is not a release process. The sole exception is mlx_metal_kernel_opt: Apple MLX/Metal cannot run in a Linux container, so it ships none — score it with --general.eval_mode local.

That is 1,273 Dockerfiles for 1,274 task cards — but they do not become 1,273 images. Image tags are content-addressed on the Dockerfile's instructions (comments and blank lines stripped, exactly as Docker itself ignores them), so any two tasks with identical instructions resolve to one tag and one build. Across the bundled suite the per-task Dockerfiles collapse into 37 distinct images. All 188 frontiercs_* tasks, for instance, ship the same single instruction under different # task: headers and share one image.

Rebuilding the base invalidates everything derived from it

A version-pinned FROM galapagos-task-base:0.4.0 folds the base image's docker ID into the derived tag, so rebuilding the base under the same tag correctly invalidates every image built on it. Editing the package source (or, in the working tree, editing a base Dockerfile) rebuilds the base via a source-digest label; editing a task — whose files are copied at run time — does not.

The card knobs

Everything the environment layer reads lives under one card section, environment:

Key	Type	Meaning
environment.docker_access	bool	The task's own evaluator is a Docker client (ALE-Bench's judge spawns containers). Grants it the host daemon socket + host-path parity, which means the whole scaffold loop moves into the container (claude_code and codex, which bring their own agents, get the same grant). A socket is root on the host — see the ALE-Bench exception and environments/dood.py.
environment.dockerfile	str	The task's Dockerfile — the only image key a card has (it defaults to a sibling Dockerfile; all bundled tasks except the explicit host-only mlx_metal_kernel_opt ship one). Built with the task dir as context, so it may COPY. A prebuilt tag is FROM tag; pinned deps are RUN pip install — which is why docker_image, base_image, requirements and python_bin are gone.
environment.env	dict | list	Env forwarded into the sealed container, alongside environment.api_keys. A dict is literal {name: value}; a list forwards those host vars if set.
environment.network	str	The docker run --network value, e.g. host | none.
environment.gpu	flag	Set → --gpus all in docker mode. The value is prose, never passed through (the 317 cards that set it say "CUDA GPU", "8x AMD MI300X (world_size=8, …)" — hardware a human needs, not a device spec). false opts out. The old evaluation.gpus, which did pass a value through, was its duplicate and no card ever used it.
environment.est_runtime_s	int	Estimated seconds for one evaluation; the eval timeout is est*4 + 30 (default est = 60). Read by both the local and docker evaluators.

GPU tasks

environment.gpu becomes --gpus all at docker run. Without the NVIDIA Container Toolkit installed, Docker then fails with "could not select device driver" — and that loud failure is deliberate. A GPU task that quietly landed in a CPU-only container would score every kernel zero and report a clean, wrong leaderboard. Failing the run is the correct behavior; a silent zero is not.

One long-lived container per run

docker mode does not start a container per candidate. It starts one long-lived TaskContainer per run — docker run -d --rm --entrypoint sleep <image> infinity — and docker execs each candidate into it at its own /tmp/<uuid>.py, so no candidate can read or clobber the file of the one before it. The reason is scale: galapagos scores thousands of candidates in a run, and Harbor's per-trial teardown would be pathological at that volume.

A docker-mode evaluator sees a sealed environment

A SubprocessEvaluator inherits the host's full environment. A ContainerEvaluator does not — a container starts with no host environment, and the evaluator sees only what environment.env / environment.api_keys explicitly forward, plus the single variable named by the run's evaluator.api_key_env. This is a feature: it is why a subscription-billed agent can never see a stray ANTHROPIC_API_KEY, and why a candidate cannot read undeclared host secrets.

claude_code: the agent inside the task's image

For a docker-mode task (the default), the claude_code scaffold is the sharpest use of this subsystem. It runs the Claude Code CLI inside the task's own image with the CLI layered on — Harbor's installed agent model, not a bundled generic runner image. The agent's in-session run_eval.sh imports the same evaluator.py, through the same interpreter, with the same packages, that the framework's trusted scorer uses, so the agent never flies blind for a dependency the scorer has.

Inside that container:

• /workspace is the agent's workspace, bind-mounted read-write, and the CLI's HOME — so Claude Code writes its native session trace to the default $HOME/.claude/projects/**.jsonl (no galapagos-specific CLAUDE_CONFIG_DIR), keeping the recorded trajectory's paths generic. (It is deliberately not /run — a real FHS directory in Debian-derived images.)
• /eval holds the task directory, copied in via docker cp and then sealed read-only by chown root + chmod go-w — not a read-only bind mount. Sealing is the trust boundary: without it the agent, running as the host uid (non-root), could rewrite /eval/evaluator.py and dictate its own score.

There is a single container: the agent and the framework's trusted scorer share it — the scorer execs in to record the score (the agent is non-root, /eval is root-owned), so there is no verifier-topology knob. Two proposer knobs steer the image:

Knob	Values	Effect
proposer.claude_docker_image	null (default) | tag	null resolves the task's own image from its card; set it to override the task image wholesale.
proposer.claude_docker_cli_version	null (default) | version	Pins npm i -g @anthropic-ai/claude-code@X; null installs latest.

meta_harness: the whole loop inside the task's image

Meta-Harness's default coding_agent=claude_code proposer must browse and update its persistent archive while evaluations happen in the same declared task environment. In Docker mode the CLI is therefore a thin host orchestrator: it builds a Claude-enabled derivative of the task image, mounts the requested run directory, and runs the controller, proposer, and evaluator inside that container. Inside the container the evaluator uses local mode because the container itself is the task environment.

The run requires --output-dir DIR; archive D is stored at <output-dir>/meta_harness_D on the host through the bind mount. Claude subscription credentials are copied through a short-lived, mode-0600 bootstrap file. Meta-Harness does not support --resume or Python resume_from.

The ALE-Bench exception (docker_access)

Everything above says the task container is a sandbox. For the 40 ALE-Bench tasks, it is not, and it is worth being blunt about why.

Those 40 are the tasks whose card declares environment.docker_access — 38 named ahc*, plus future_contest_2022_qual and toyota2023summer_final. The glob ahc* does not cover the set; list them with grep -rl docker_access src/galapagos/tasks/*/card.yaml.

ALE-Bench's evaluator does not score a candidate in the task container — it hands the source to ale_bench, which compiles and runs it inside sibling judge containers it launches on your Docker daemon. So the container that scores one of these tasks carries the host's docker socket, plus a path-parity directory (one host directory bind-mounted at its own absolute path, holding HOME, TMPDIR, and $ALE_BENCH_CACHE). The parity half is not optional and its absence fails silently: the daemon resolves a sibling's volume sources against the host, so a socket alone gives every judge container empty bind mounts — no source to compile, no case to run, and a zero score with nothing raised. See environments/dood.py.

A socket is root on the host. For an ordinary scaffold that costs nothing: the trusted evaluator holds it and the candidate — which ale_bench runs in a judge container sealed with network_disabled=True — never sees it. But claude_code puts an agent with a shell in that container, and it gets the same grant. On a docker_access task the agent can reach the daemon, and therefore the host.

That is deliberate, and it is the only way an agent can see its own score here: ale_bench exposes no judge service to call (contrast FrontierCS's go-judge, an HTTP server a sealed container simply POSTs to) — only containers it launches. Keeping the agent out buys nothing back, either: the alternative is a host-side loop where the agent runs as you, and you must already hold rw on that same socket for dood.preflight() to pass. Same reach, no container — and worse, the agent's run_eval.sh and the framework's recorded score would then come from different environments and could silently disagree.

So: run claude_code on a docker_access task only on a host you are willing to expose to the agent — and note that means all 40, not just the 38 the ahc* glob matches. Everywhere else, the sandbox story on this page holds.

See also

• Cards — the Task Card whose environment section this page reads.
• Write a custom task — the end-to-end task guide, including Docker (sandbox) evaluation.
• Task reference — the bundled task catalog and families.
• Config reference — the six config sections and every knob.
• CLI reference — galapagos run, its --task-source flag, and the dotted config flags.