GEPA/gepa

GEPA ("Genetic-Pareto", Agrawal et al., ICLR 2026 Oral) replaces the scalar reward signal of RL with the *text* an evaluation actually produces. Its thesis: a rollout emits far more than a number — error messages, tracebacks, rubric violations, profiling output, compiler diagnostics — and an LLM that reads that trace can propose a targeted fix in one shot, where a policy gradient needs thousands of samples to infer the same thing. The paper reports outperforming GRPO by up to 19% with up to 35× fewer rollouts.

Two mechanisms carry the method. **Reflective mutation**: each iteration shows the reflection LM one candidate together with the feedback its own evaluation produced, and asks for a complete drop-in replacement — not an edit script. **Pareto-frontier selection**: the candidate pool keeps every accepted candidate forever and tracks, for each axis the search is measured on, which candidates hold the best value. A parent is drawn by pruning the candidates that are dominated (every axis they lead is also led by someone else) and then sampling proportionally to how many axes each survivor leads. A candidate that wins on one narrow axis is never starved, which is what keeps GEPA off the single-lineage hill climb that greedy selection collapses into (paper Table 3: +12.44% over greedy, +7.33% over beam search).

This scaffold ports the reference implementation ([gepa-ai/gepa](https://github.com/gepa-ai/gepa)), specifically its `optimize_anything` API — the entry point GEPA itself uses to optimize *code* rather than prompts (its bundled examples cover circle packing, ARC-AGI and cloud-scheduling policies). A galapagos task is one problem scored by one `evaluate(program_path)` call, which is exactly upstream's **single-instance search mode** (`dataset=None, valset=None`): the engine evaluates a single sentinel example, pins the reflection minibatch to 1, and the minibatch, the acceptance test and the "full valset" evaluation collapse onto that one evaluation. The frontier axes are then the task's own declared metrics.