Meta-Harness/meta_harness

Meta-Harness (Stanford IRIS Lab) strips the evolutionary outer loop to its bare minimum: there is no parent selection, no archive policy, and no mutation operator. The entire search lives in a skill-steered coding-agent proposer that reads the whole candidate history — every prior program, its score, its report, and its execution trace — and writes a fixed number of brand-new full programs each round. The outer loop's only jobs are to validate each candidate's interface, evaluate the valid ones, append their outcomes to a running summary, and recompute a frontier. The run's product is that frontier, not a single best pointer. Scalar search is the task-agnostic default; configuring a task cost metric enables the reference's score x cost Pareto sweep.

The proposer is constrained by near-verbatim steering rules carried in an editable `SKILL.md` file rather than in code, because the paper's practical-tips appendix found that editing the skill text moved results more than any loop constant. Those rules forbid parameter-only variants ("identical except constants => rewrite"), forbid dataset-specific hardcoding, forbid early stopping, cap each candidate's report at 30 lines, and rotate the search across six exploitation axes so successive rounds explore different mechanism families instead of clustering on one.

This is a faithful port of the reference implementation (the canonical `text_classification` example), with the code treated as ground truth wherever paper and code diverge. The evolving *target* necessarily differs — Galapagos evolves a task's EVOLVE-BLOCK program, not a MemorySystem/AgentHarness wrapping a frozen base model — but the evolving *method* is ported two ways via `proposer.mode`:

- **`coding_agent` (default, reference-faithful).** The proposer IS a headless coding agent, exactly like the reference. Each round the archive `D` is materialized to disk (`evolution_summary.jsonl`, `frontier_val.json`, `candidates/<name>.py`, `logs/<name>/{result,metrics,artifacts,per_instance,trace}.*`, `reports/<name>.md`, `current_best.py`), an agent session is launched at `cwd=D` steered by the domain `SKILL.md`, and the agent BROWSES `D` with grep/cat and writes candidate files + a `pending_eval.json` manifest. A nonzero/timeout agent exit discards the whole round, including partial manifests. The outer loop then evaluates each successful round's candidates with the task's own evaluator. **Which agent** is a config choice (`proposer.coding_agent`, see [`coding_agents.py`](coding_agents.py)): **`claude_code`** (default — Claude Code CLI, native `--append-system-prompt`, subscription-billed with API keys stripped so a run can never bill the API), **`codex`** (OpenAI Codex CLI, experimental), or **`cursor_agent`** (Cursor Agent CLI, experimental). Adding another agent = one `CodingAgentAdapter` subclass. - **`llm` (fallback).** A single API model call — a raw LLM with no filesystem tools — receives a *serialized* slice of `D` (the evolution-summary table, the Pareto frontier, recent reports, errors-first trace excerpts, and the full source of the top frontier members) and emits the candidates inline. Works with any API proposer model; no agent CLI.

Both modes share the Population, SelectionPolicy, Memory, batch semantics, and validation gate. One `general.max_iterations` unit is one proposer round, regardless of how many valid candidates that round returns. The FIFO is an internal bridge to the one-child base step; it is fully drained without advancing the round number.

The bundled budget is therefore 20, matching the reference `--iterations 20`; with k=3 this can evaluate roughly 60 proposed candidates plus the seed. The frontier is scalar by default. Set `population.cost_metric` to a metric emitted by every evaluation to enable Pareto mode; `genome_chars` is available only as an explicit source-size objective and is not treated as a proxy for inference-context cost.