default/openevolve_world_knowledge

# OpenEvolve + World Knowledge (`openevolve_world_knowledge`)

> OpenEvolve, with an open world: a gate decides per iteration whether to consult the web, and the search query is itself evolved against the improvement it produces.

## Overview

Every LLM-driven evolutionary search framework in the leaderboard — OpenEvolve, AdaEvolve, EvoX, GEPA — is **closed-world**. Each prompt is assembled from the run's own trajectory, so the loop can only ever recombine what it has already thought of. When the population plateaus, it has no mechanism for acquiring anything new, and it keeps re-deriving the same idea in slightly different words. A human researcher stuck on a problem does something else: they read a paper, check what is already known, and come back with an idea that was not in their own history.

This scaffold gives the loop that option. It is OpenEvolve — the same island MAP-Elites population, the same three-tier selection, the same SEARCH/REPLACE diff proposer, inherited rather than re-implemented — plus a **World Knowledge Layer** that turns the search into a bi-level optimization. The outer loop evolves *solutions* against the task's verifiable evaluator. A gate decides whether this iteration should seek external knowledge; retrieval can use Tavily (the default) or a restricted Claude Code WebSearch subprocess, rerank the result set listwise with a long-context LLM, or refine the query over several self-reflection rounds before the documents enter the mutation prompt.

The two loops are joined by a single number. When a candidate is written with retrieved documents in its prompt and then evaluated, the improvement it realized — `Δ_t = E(x_t) − E(x_{t−1})` — is written back onto the query that produced those documents. Since the LLM is frozen and never trained, those recorded `Δ`s are the *entire* supervision the query operator receives: the good ones and, just as importantly, the bad ones. The store shows the operator both extremes, because a list of only what worked cannot teach it what to stop searching for.

**None of the six component slots differ from `openevolve`.** That is the point. The components describe the *method* — a scaffold that listed a different PromptBuilder or a different Memory would be a different method, and the comparison would stop being about retrieval. The layer is orthogonal to all six: it is one object held by the scaffold and three call sites, so with it disabled — or on any iteration where the gate declines — the rendered prompt is byte-identical to OpenEvolve's and a difference between the two rows of a results table is a difference in *evidence*, not in wording.

## Algorithm

Each iteration, after the selection policy has picked the parent `x_{t−1}` and its context `H_{t−1}`, the layer runs before the mutation call:

```
c_t = (x_{t-1}, H_{t-1})                        # from OpenEvolve's selection, unchanged
D_search^(K) ~ pi_select(D_search, K)            # highest- AND lowest-Delta entries
g_t          ~ G(c_t, D_search^(K))              # no-op | look-up | retrieve

  no-op    -> S = {}                             # prompt identical to OpenEvolve's
  look-up  -> S = cached documents               # reuse D_search; no new search
  retrieve -> q^(0)   ~ Q(c_t, D_search^(K))
              Doc^(0) = Search(q^(0))            # configured retrieval backend
              M = 0: Doc* = Doc^(0)              # keep provider order
              M = 1: Doc* = LCLM-Rerank(         # no query evolution
                                  Doc^(0), q^(0), x_(t-1), D_search^(K))
              M >= 2:
                  for m = 1..M:
                      q^(m)   ~ Self-Reflect(q^(m-1), Doc^(m-1), c_t, D_search^(K))
                      Doc^(m) = Search(q^(m))
                  Doc* = Doc^(M)
              commit (q, Doc*) -> D_search
              S = Doc*

x_t = M_theta(prompt(c_t) + S)                   # OpenEvolve's diff proposer, prompt + evidence
s_t = E(x_t)                                     # the task's evaluator, unchanged
Delta_t = s_t - E(x_{t-1})  ->  UpdateDelta(D_search, q_t, Delta_t)
```

Query construction is round zero of the same inner loop that performs reflection: `construct → Search` produces `(q^(0), Doc^(0))`, followed by zero or more `reflect → Search` rounds. The complete chain is persisted, while the committed pair is always `(q^(M), Doc^(M))` — the query whose documents directly reached the mutation LLM. `Δ` measures the completed chain's outer-loop outcome; it is not copied onto every intermediate round because the evaluator provides no per-round counterfactual effect. A reflection step that reproduces its input query is left to search again rather than short-circuited: whether the inner loop converges or circles is one of the things this trace exists to measure.

**One frozen layer LLM plays every reasoning role.** Mutation, gating, query construction/reflection, and result reranking are the same `M_theta` — the run's proposer model — separated only by the role prompt. Tavily remains the clean default for that experimental claim. Selecting `claude_code` deliberately introduces a second model inside `Search(q)`, so results from that backend should be reported as a separate retrieval-provider ablation.

## Components — all six are OpenEvolve's

| Slot | Implementation | Changed? |
|---|---|---|
| Population | `MapElitesIslandsPopulation` | no |
| SelectionPolicy | `OpenEvolveSelectionPolicy` | no |
| PromptBuilder | `OpenEvolvePromptBuilder` | no |
| Proposer | `OpenEvolveProposer` | no |
| Evaluator | task-supplied | no — the layer never touches scoring |
| Memory | `NullMemory` | no |

## Where the layer attaches instead

`self.world_knowledge` on the scaffold — built in `__init__` only when `world_knowledge.enabled`, `None` otherwise — and three call sites:

| Hook | When | What |
|---|---|---|
| `augment_prompt(prompt, selection)` | after selection, before the proposal | runs the inner loop and inserts `S` between the current program and `# Task`. Once per iteration, so inner retries reuse the evidence rather than re-gating. |
| `after_step(child, result)` | after evaluation | `UpdateDelta` — writes `Δ_t` onto the entry that fed the step. |
| `setup` / `_resume` | before anything is spent | preflight the search backend, then seed `D_search` (RAG settings). |
| `_checkpoint_scaffold_state` | on checkpoint | `D_search` is scaffold-owned search state, not component state. |

`augment_prompt` is a generic no-op hook on `GalapagosScaffold`, alongside `before_step` / `after_step` / `periodic` — it is the seam for *any* layer, and it names nothing about this one.

The layer itself lives in `galapagos/world_knowledge/`, not in this directory, because its whole claim is that it is an *addition* to an existing method rather than a new one: mounting it on another scaffold is the same one object and the same three calls.

## Running it

```bash
export TAVILY_API_KEY=tvly-...            # the search backend
pip install -e ".[all,web-search]"        # tavily-python

galapagos run --scaffold openevolve_world_knowledge --task circle_packing \
  --proposer.model_name openai/gpt-5.5 --proposer.api_base openrouter \
  --general.max_iterations 100
```

To use an existing Claude Code login instead of Tavily:

```bash
claude auth status
galapagos run --scaffold openevolve_world_knowledge --task circle_packing \
  --world_knowledge.retrieval_backend claude_code \
  --world_knowledge.claude_code.model haiku
```

Claude Code runs in safe mode with only `WebSearch` available by default. Set
`--world_knowledge.claude_code.allow_web_fetch true` only when fetching source pages is worth the
extra turns and cost, and raise `claude_code.max_turns` to at least `3` with it.

A run configured to retrieve **fails at startup** when its selected backend is unavailable or unauthenticated rather than quietly returning no evidence for a hundred iterations — a closed-world run reported as an open-world one is the failure mode this method most needs to avoid.

The layer runs in the scaffold loop, which lives on the host, so provider credentials never enter the container the candidate program runs in. Nothing retrieved is executed; it is prompt text.

### The ablation ladder

Every row of the experiment table is this one scaffold under a different config, so nothing else varies between them:

| Row | Config |
|---|---|
| OpenEvolve | `--scaffold openevolve` (or `--world_knowledge.enabled false`) |
| \+ LLM tool calling | `--scaffold openevolve --proposer.tool_call_on true` |
| \+ World Knowledge, LCLM result reranking | `--world_knowledge.query_evolution_steps 1` |
| \+ … LCLM reranking without `D_search` | `--world_knowledge.query_evolution_steps 1 --world_knowledge.use_search_database false` |
| \+ … select by LLM relevance/novelty | `--world_knowledge.query_evolution_steps 1 --world_knowledge.select_policy relevance_novelty` |
| \+ World Knowledge, with query evolution | `--world_knowledge.query_evolution_steps 3` (the default) |
| \+ … with RAG grounding | `--world_knowledge.grounding offline --world_knowledge.grounding_file seeds.jsonl` |

Two controls for the gate itself: `--world_knowledge.gate always` removes it, and `--world_knowledge.gate heuristic --world_knowledge.retrieve_probability 0.3` replaces it with a random rate — which is what separates "retrieval helps" from "retrieving *when the model chose to* helps".

In LCLM reranking mode (`query_evolution_steps=1`), query construction has two explicit experimental paths. With `use_search_database=false`, `retrieval_gating_no_search_database.txt` limits the LLM gate to `no-op|retrieve`, and `query_construction_no_search_database.txt` is used on every retrieving iteration. The layer never selects, persists, stages, or credits `D_search`; retrieved documents are still reranked and passed directly to that iteration's mutation prompt. With `use_search_database=true` (the default), the ordinary database-aware gate and query templates receive the selected query/document/Δ history and each retrieval is persisted for later reuse. The retrieve trace records query construction as `no_search_database|with_search_database` and includes `search_database_enabled`.

## What the run records

When enabled, `D_search` is mirrored live to `<run_dir>/world_knowledge/search_database.jsonl`, one JSON line per entry. Each `SearchRecord` contains the full `query_evolution` chain: every constructed query, its reranked web results, relevance/novelty assessments, and the reflection analysis fed into the next constructor. It also contains the final query/result pair used as direct evidence and structured `impacts` tying each completed chain use to the selected target score, the evaluator's continuous child score `S`, `Δ = S - target_score`, and `score_change` (`improved`, `decreased`, `unchanged`, or `not_evaluated`). No database file is created by stateless retrieval. The final aggregates are means over the reranked documents that can later be reused (`documents_per_entry`); `SearchResult` keeps its existing checkpoint-compatible schema, with unavailable provider fields set to `None`. The layer also emits ETIF events on the `query` evolution track (`memory`/`gate`, `/retrieve`, `/look_up`, `/credit`). A retrieve event carries every full round, the committed document order, aggregate scores, reflection analyses, and model-call identities.

## Configuration

All under the `world_knowledge` section (`config.yaml` ships the defaults):

| Key | Default | Meaning |
|---|---|---|
| `enabled` | `true` | Off → this scaffold behaves exactly as `openevolve`. |
| `gate` | `llm` | `llm` (the method) / `always` (no-gate ablation) / `heuristic` (random-rate control). |
| `retrieve_probability` | `0.1` | `heuristic` gate only. |
| `query_evolution_steps` | `3` | Must be at least `1`. `1` = one search plus LCLM reranking with no query evolution; `M >= 2` = `M` complete construct→search→rerank→reflect rounds. |
| `use_search_database` | `true` | False makes retrieval stateless: no `D_search` selection, prompt context, persistence, reuse, or Δ credit. |
| `select_policy` / `select_num` | `delta` / `6` | `pi_select` over `D_search`, and `K`. Policies: `delta`, `relevance_novelty`, `recency`, `full`. |
| `documents_per_entry` | `3` | Documents rendered per shown entry. |
| `max_document_chars` | `2000` | Per-document cap in the evidence block. |
| `rerank_max_document_chars` | `8000` | Per-candidate cap in the long-context reranking prompt. |
| `rerank_relevance_weight` / `rerank_novelty_weight` | `0.5` / `0.5` | Equal normalized weights for actionable relevance and marginal novelty. |
| `evidence_block_prompt` | `retrieved_evidence_block_wo_image` | Per-document template inserted into the mutation prompt; use `retrieved_evidence_block_w_image` to expose returned image metadata. |
| `history_programs` | `5` | Recent candidates summarized for the gate's stagnation call. |
| `grounding` / `grounding_file` | `none` / — | `offline` (a jsonl seed) or `online` (one pre-loop search). |
| `retrieval_backend` | `tavily` | `tavily` or `claude_code`; injected/offline retrieval objects still take precedence in code. |
| `tavily.*` | see `config.yaml` | The `Search(q)` request — `max_results` is the retrieval top-K. |
| `claude_code.*` | see `config.yaml` | Restricted CLI WebSearch settings, including model, timeout, result count, and optional WebFetch. |

## Cost

With `query_evolution_steps=1`, a retrieving iteration spends a gate call, one query-construction call, one search, and normally one reranking call; reranking is skipped when fewer than two documents were returned. With `M >= 2`, it normally spends `1 + 3M` layer-model calls (one gate, then construct + rerank + reflect in each round) and `M` searches. A reranking call is skipped in any round returning fewer than two documents. The last reflection is persisted for later `D_search` context. With `retrieval_backend=claude_code`, each search is also a separate Claude CLI model/tool call with its own latency and billing.

`select_policy=relevance_novelty` uses the same normalized weights as the reranker and computes `utility = relevance_weight × relevance + novelty_weight × novelty`. Scores are snapshots from retrieval time; selecting from `D_search` does not trigger another LLM call. Records without a complete assessment fill any remaining slots in newest-first order.

## Limitations

- **Provider outputs differ.** Tavily exposes a provider relevance score and can return cleaned page content; Claude Code WebSearch returns model-selected source passages and therefore stores `score=None` unless Galapagos's own reranker assesses it. Optional WebFetch adds latency and cost.
- **`Δ` is attributed per entry, not per document.** One iteration injects one entry's documents and produces one score; splitting the credit across documents would invent an attribution nothing measured.
- **Reward hacking is not prevented here.** A task whose solution is published on the web can be looked up rather than discovered. That is a property of the open-world setting itself, and detecting it is one of the project's own research questions — the full query and document trace is recorded precisely so it can be audited.