Scaffolds

The catalog ships twelve scaffold cards — all runnable. Each card lives next to the runnable code at src/galapagos/scaffolds/<name>/card.yaml, alongside its config.yaml, README.md, and component modules — one self-contained folder per scaffold. List them at runtime:

import galapagos as gx
gx.available_scaffolds()   # every bundled card: ['adaevolve', 'ale_agent', 'algotune_agent', 'beam_search', 'best_of_n', 'best_of_n_attempts', 'claude_code', 'codex', 'evox', 'meta_harness', 'openevolve', 'topk']
gx.registered_scaffolds()  # the runnable subset — the same twelve

galapagos scaffold list

One self-contained folder

A bundled scaffold is a single folder, src/galapagos/scaffolds/<name>/, holding everything: the card (card.yaml, mapping the method onto the six components), the code (the controller class plus the component modules), its config.yaml defaults, and a human-readable README.md. Loading an unknown name raises a clear KeyError listing the runnable set.

Evolutionary methods (3)

Name	Display	Status	Summary
openevolve	OpenEvolve	runnable	Island-model MAP-Elites evolutionary search with diff mutation (the open AlphaEvolve).
adaevolve	AdaEvolve	runnable	Hierarchical adaptive search: G-signal exploration intensity, UCB island allocation, and LLM meta-guidance on stagnation.
evox	EvoX	runnable	Co-evolves the search strategy with the solutions: the parent/context selection policy is itself LLM-written code, scored by windowed improvement and hot-swapped on stagnation.

AdaEvolve ("AdaEvolve: Adaptive LLM-Driven Zeroth-Order Optimization", UC Berkeley) treats evolutionary program search as a hierarchical adaptive optimization problem driven by one signal — the fitness-improvement trajectory. Level 1: each island's exploration intensity adapts via an accumulated improvement signal (AdaGrad-style). Level 2: a UCB bandit with decayed, globally-normalized rewards allocates iterations across islands, with ring migration and dynamic island spawning on stagnation. Level 3 (meta-guidance): when improvement stalls, a separate LLM call generates breakthrough "tactics" that are injected into mutation prompts and rotated until exhausted. Ported from the reference implementation in SkyDiscover. Components: qd_island_archipelago population / adaptive_intensity_ucb selection / adaevolve_template prompts / diff proposer / task evaluator / paradigm_tactics memory.

EvoX ("EvoX: Meta-Evolution for Automated Discovery", UC Berkeley) co-evolves the search strategy with the solutions: the parent/inspiration selection policy is itself LLM-written code (an EvolvedStrategy class) that is scored by windowed improvement J = Δ·(1+ln(1+s_start))/√W, validated by a behavioral test-suite, and hot-swapped with full population migration (plus runtime fallback) whenever the best score stagnates for a window (default 10% of the budget). Problem-specific DIVERGE/REFINE variation operators are generated once per run and injected as parent labels. Ported from the reference implementation in SkyDiscover. Components: evolved_strategy_store population / evolved_strategy_sampler selection / operator_labeled_default prompts / diff proposer / task evaluator / strategy_history memory.

All carry type: test_time_search, tier: search.

SkyDiscover search baselines (3)

Three search strategies ported from SkyDiscover (UC Berkeley Sky Computing Lab). Their cards carry organization: "default" (repo_id: default/<name>, e.g. default/topk); the SkyDiscover organization instead tags the SkyDiscover-implemented adaptive methods adaevolve and evox. Simple, fixed-rule references that every adaptive method is compared against — all runnable.

Name	Display	Status	Summary
topk	Top-K	runnable	Always expand the single best program, with the next K as context. Pure greedy elitism.
best_of_n	Best-of-N	runnable	Give the LLM N valid attempts at the same parent before committing to the global best, then repeat.
best_of_n_attempts	Best-of-N (attempt-counted)	runnable	Best-of-N that rotates the parent every N attempts — failed/invalid tries spend the budget too.
beam_search	Beam Search	runnable	Maintain a fixed-width beam of promising programs; expand one per step, prune by fitness+diversity.

best_of_n_attempts is a Galapagos variant of best_of_n (attempt-counted budget — every try, valid or not, spends one of the parent's N), not a SkyDiscover port; the "(3)" header counts the SkyDiscover ports themselves.

Algorithm-engineering agents (2)

Name	Display	Status	Summary
ale_agent	ALE-Agent	runnable	Domain-guided tabu best-first search with 30 sibling branches and three-turn solution refinement.
algotune_agent	AlgoTune Agent	runnable	Command-driven algorithm optimizer with evaluation and profiling tools, multi-file edits, explicit revert, and best-snapshot restore.

ALE-Agent is the specialized scaffold introduced with ALE-Bench. It ranks states by accepted-case ratio and score, expands the best unexpanded state once, and searches 30 sibling branches. Each branch uses one of four algorithm-engineering guides and receives three implementation turns with evaluator feedback; its best version returns to the frontier. The public ALE-Bench repository does not ship the experimental agent source, so this is a paper-specification port based on the NeurIPS 2025 main paper and supplement. Galapagos schedules siblings sequentially for deterministic replay, preserving the search tree but not the original implementation's parallel latency amortization. Components: best_first_archive population / tabu_batched_best_first selection / ale_domain_guided prompts / multi_turn_full_rewrite proposer / task evaluator / search_trajectory_summary memory.

AlgoTune Agent is a source-faithful port of the command agent from oripress/AlgoTune. The model emits exactly one command per turn: it can inspect and edit a virtual multi-file workspace, invoke the reference solver, evaluate specific inputs or the current candidate, profile functions or source lines, and restore the best measured snapshot. Edits pass the upstream syntax and anti-tampering checks before evaluation. The mutable working workspace remains separate from the best valid snapshot, and the controller preserves the upstream $1 provider-reported model-spend stop.

Galapagos stores a candidate as one Genome, so helper modules, build metadata, and Cython/Pythran/ DaCe sources are packed into that artifact and materialized before import. The scaffold runs its whole command loop in the selected task image so diagnostic tools and the trusted scorer see the same packages. It intentionally accepts only algotune_* tasks because its reference, validator, timing, and profiling tools depend on that task contract. Components: best_snapshot_archive population / mutable_workspace_with_revert selection / algotune_command_transcript prompts / command_driven_optimizer proposer / task evaluator / bounded_role_history memory.

galapagos run --scaffold algotune_agent --task algotune_svm \
  --set proposer.model_name=openai/o4-mini \
  --set proposer.api_base=openrouter \
  --set general.max_iterations=9999

Single-agent baselines (2)

Name	Display	Status	Summary
claude_code	Claude Code	runnable	Hands the whole search loop to one Claude Code CLI session (subscription-only billing), run inside the task's own container image by default (or a host subprocess for local-mode tasks); the framework scores solution-file checkpoints and preserves the CLI's native session trace.
codex	Codex	runnable	Runs one codex exec --json session in the task's environment with ChatGPT-managed authentication; Galapagos independently scores every observed solution checkpoint and preserves Codex's native rollout.

claude_code is a Galapagos variant of SkyDiscover's claude_code baseline controller: the same TASK.md contract, stream-json turn accounting, and solution-file checkpoint polling. Billing is subscription-only (ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN are always stripped; auth comes from CLAUDE_CODE_OAUTH_TOKEN via claude setup-token, or the local claude /login credentials). general.max_iterations is the prompt-level evolution budget; proposer.claude_cli_max_turns is the larger CLI --max-turns safety cap (default: 2× the prompt budget), and the optional proposer.claude_wall_timeout_seconds overrides the derived whole-session timeout.

There is one claude_code scaffold, and it places itself where the evaluation happens — setup() dispatches on task.eval_mode, so the agent always shares the interpreter and wheels its scorer uses.

A docker-mode task (the default): the session runs inside the task's own image. This is Harbor's installed agent model rather than SkyDiscover's runner image: the Claude Code CLI (Node + @anthropic-ai/claude-code) is layered onto the task's own image, built on first use, and one container is started from it as --user <host-uid>:<host-gid> — non-root. The workspace is bind-mounted read-write at /workspace and is the CLI's HOME, so Claude Code's native session JSONL lands at the default $HOME/.claude/projects/**.jsonl (generic paths, no galapagos-specific CLAUDE_CONFIG_DIR, so the recorded trajectory stays reusable as training data). The task directory is copied in at /eval via docker cp and sealed — root-owned, chmod go-w — so the agent's in-session run_eval.sh can score itself against the very runner the framework records with, but cannot rewrite it. That single container is the whole design: the framework's trusted scorer is a ContainerEvaluator attached to the agent's container and execs in to record the score, so there is no verifier-topology knob, and the uid boundary (non-root agent, root-owned /eval) is what keeps the agent from dictating its own number. A container starts with no environment unless it is passed with -e, so an API key cannot leak in even by accident. Optional proposer.claude_docker_memory / claude_docker_cpus / claude_docker_network cap what the session can consume, and proposer.claude_docker_image overrides the task image wholesale. Only Docker is needed on the host — not the claude binary.

A local-mode task: the session runs as a host subprocess. The evaluator is a host subprocess there, so the agent must be too. This path needs the claude CLI on PATH, and it runs it with --dangerously-skip-permissions as your user on your machine — point it only at tasks whose evaluators you trust. CLAUDE_CONFIG_DIR is redirected to <out_dir>/claude_sessions, so the native session JSONL is preserved under claude_sessions/projects/ (galapagos.traces.export_claude_code_traces() post-processes it into derived SFT/ShareGPT conversation files, an event-timeline JSONL, and a summary JSON).

A docker_access task (ALE-Bench): the container is not a sandbox

The 40 ALE-Bench tasks (38 named ahc*, plus future_contest_2022_qual and toyota2023summer_final) are scored by an evaluator that is itself a Docker client — ale_bench compiles and judges each candidate in sibling containers it launches on your daemon — so their container carries the host's docker socket, and a socket is root on the host. The agent goes in there anyway, because ale_bench exposes no judge service to call instead, so an agent that cannot reach the daemon cannot see its own score. Keeping it out buys nothing back: the host loop runs the agent as you, and you must already hold rw on that same socket for the preflight to pass. Run claude_code or codex on ahc* only on a host you are willing to expose to the agent — see Task environments.

codex follows the same installed-agent topology: a Docker-mode run layers @openai/codex onto the task image, seals the evaluator under root-owned /eval, and gives a non-root agent /workspace. Local mode uses codex exec --json --dangerously-bypass-approvals-and-sandbox as the current user. Authentication is ChatGPT-managed, not API-key billed: CODEX_ACCESS_TOKEN is used when available, otherwise the scaffold stages a refreshable file-backed codex login cache in an isolated CODEX_HOME. general.max_iterations is the prompt-level improvement budget; proposer.codex_cli_max_steps supplies the completed-action safety cap because Codex has no --max-turns.

Agent-driven method (1)

Name	Display	Status	Summary
meta_harness	Meta-Harness	runnable	A minimal outer loop that delegates selection and mutation to a skill-steered proposer over an append-only history, returning a scalar or task-configured Pareto frontier.

Meta-Harness ("Meta-Harness: End-to-End Optimization of Model Harnesses", Stanford IRIS Lab) strips the evolutionary outer loop to its bare minimum: no parent selection, no archive policy, no mutation operator. A skill-steered proposer — constrained by an editable SKILL.md rather than by code — reads the whole candidate history and writes k brand-new programs per round; the outer loop validates, evaluates, appends outcomes to a running evolution summary, and recomputes its frontier. Scalar combined_score search is the task-agnostic default; setting population.cost_metric to an evaluator metric (or explicitly to genome_chars) enables the two-axis frontier. Ported from the reference implementation (the canonical text_classification example). Components: append_only_pareto population / proposer_delegated_frontier_anchor selection / skill_steered_filesystem_view prompts / k_candidate_queue proposer / task evaluator / evolution_summary_reports memory.

The default proposer is Claude Code. With the default Docker evaluation mode, the CLI orchestrates the controller, proposer, and evaluator inside the task container. --output-dir is required and archive D is stored at <output-dir>/meta_harness_D. Meta-Harness is intentionally a fresh-run-only scaffold: neither CLI --resume nor Python resume_from is supported. Set proposer.mode=llm only when the serialized API-model fallback is desired.

Fidelity to the originals

topk / best_of_n (vs SkyDiscover) and openevolve (vs the open OpenEvolve / AlphaEvolve) are faithful ports — they reproduce the originals' search behavior, not just their shape. The behavior-determining details that match:

• Selection. Top-K parent = rank 1 with ranks 2..K+1 (and the lone seed as its own context on step 1); Best-of-N reuses one parent until N valid children, then commits to the global best (parent chosen by safe_score, context drawn by the metric-mean get_score); OpenEvolve's round-robin islands + 3-tier explore/exploit/random sampling + island-uniform inspirations (the live parallel path).
• Admission. SkyDiscover drops an errored child (if result.error: continue), so topk/ best_of_n reject eval-invalid candidates; OpenEvolve keeps every evaluated child (including score-0/errored) in its MAP-Elites grid, so openevolve admits them. Each store owns this policy.
• Mutation. Whole-line SEARCH/REPLACE diff application (exactly-7 markers, no full-rewrite fence fallback) matching apply_diff; a non-matching block is a no-op (retried/discarded) for the SkyDiscover scaffolds.
• Population. OpenEvolve's MAP-Elites feature binning (running min/max), global archive + worst-eviction, population cap, and lazy ring migration on island-generation counters.
• Evaluation & validity. Cascade evaluate_stageN with per-task thresholds; the validity gate mirrors SkyDiscover's discard rules; OpenEvolve's per-candidate evaluator retries.
• Prompt. A generic system message with the diff format in the user # Task (SkyDiscover), and OpenEvolve's # Program Evolution History (Previous Attempts / Top / Diverse / Inspiration) sections, Focus-areas trend, and full-source context programs.
• Defaults. num_islands=5, archive_size=100, population_size=1000, migration 50/0.1, feature_bins=10, num_inspirations, the explore/exploit ratios, general.max_iterations=100, general.inner_retry_times=1 unless a run config overrides it, general.max_solution_length=10000
• seed=42 + weighted model-ensemble support (OpenEvolve).

Two differences are intrinsic to the re-architecture, not faithfulness gaps: an exact run-for-run random sequence cannot match (Galapagos uses an isolated, seeded RNG rather than the originals' global RNG), and the Galapagos loop is strictly sequential — the originals' max_parallel_iteration is deliberately not ported, because a run's result must not depend on batch size or thread-scheduling order (that would make two submissions of the same method + task score differently for reasons unrelated to the method, which is exactly what the leaderboard exists to rule out). Algorithmic parallelism — island models, MAP-Elites — still lives in the method, and stays deterministic; only the harness-level concurrency is gone.

Card fields

Each scaffold card (a ScaffoldCard) can record: name, display_name, organization, type, tier, status, summary, description, source (paper/repo), tags, license, controller (the dotted Scaffold-subclass path), components (which implementation fills each of the six slots), model, requirements, assets, external_resources, defaults_config, and examples. Bundled cards mostly use the compact subset: identity fields, controller, and components: {kind: ...}. Their default configs are sibling config.yaml files loaded by GalapagosConfig.from_config(name).

from galapagos.cards.registry import load_scaffold_card
card = load_scaffold_card("openevolve")
card.controller    # 'galapagos.scaffolds.openevolve.scaffold.OpenEvolveScaffold'
card.components.selection_policy   # {'kind': 'three_tier_explore_exploit'}

To run a runnable scaffold, see Run a scaffold. To author a new one, see Write your own scaffold.