Skip to content

ShinkaEvolve worked port

This example follows a realistic contribution scenario:

A community contributor studies ShinkaEvolve, maps its distinguishing mechanisms onto the Galapagos schema, validates the portable artifact, and prepares a scaffold pull request.

The implementation is an educational, Galapagos-native port. It is not authored or endorsed by Sakana AI. The complete runnable source lives in examples/shinkaevolve.

Start with the upstream contract

The port cites the official ShinkaEvolve repository and paper, and pins the studied source commit in card.yaml.

The three central mechanisms mapped by this example are:

  • parent sampling that balances exploration and exploitation;
  • novelty rejection before spending evaluations on near-duplicates; and
  • bandit selection across a pool of mutation LLMs.

Reading the upstream contract first prevents a common porting error: giving a familiar name to an implementation that preserves only the surface API.

Declare the fidelity boundary

The example deliberately teaches the component mapping without embedding the full upstream runtime:

Upstream behavior Educational Galapagos mapping
SQLite program database and archive Checkpointable in-memory island population
Asynchronous proposal/evaluation workers Deterministic sequential Galapagos loop
Power-law and exploratory parent sampling Compact power-law/explore policy
Cost-aware UCB over mutation LLMs Small cost-aware model-arm bandit
Embedding and optional LLM novelty judges Token-Jaccard near-duplicate resampling
LLM-generated meta summary Deterministic notes from successful improvements
Diff, rewrite, and crossover modes sampled within a run Both Galapagos diff_based_edit and full_rewrite, selected per run, plus inspiration context

These differences are recorded in the README and card instead of being hidden behind a claim of full fidelity.

Map the six components

Galapagos slot Example file Responsibility
Population population.py Island archive and migration
SelectionPolicy selection_policy.py Parent sampling and UCB model-arm choice
PromptBuilder prompt_builder.py Task prompt plus accumulated recommendations
Proposer proposer.py Routed diff/full-rewrite call and novelty-gated result
Evaluator supplied by the selected task Recomputes the task-owned score
Memory memory.py Rolling meta-recommendation scratchpad

scaffold.py wires the five scaffold-owned objects together. model_pool.py exposes normal Galapagos models as bandit arms, while config.yaml contains only typed GalapagosConfig fields.

examples/shinkaevolve/
├── README.md
├── __init__.py
├── card.yaml
├── config.yaml
├── memory.py
├── model_pool.py
├── population.py
├── prompt_builder.py
├── proposer.py
├── run_example.py
├── scaffold.py
└── selection_policy.py

Validate without calling a model

From a contributor checkout:

python scripts/validate_scaffold_submission.py examples/shinkaevolve

The expected summary is:

1 scaffold submission(s): 1 passed, 0 failed

This exercises the actual galapagos submit --dry-run bundle, reloads the card, checks all six declarations, imports the controller, and verifies the five scaffold-owned runtime components. It makes no live or paid model call.

After configuring the API key for your model host, run the default diff_based_edit approach:

python -m examples.shinkaevolve.run_example \
  --task function_minimization \
  --model openai/gpt-4o-mini \
  --host openrouter \
  --iterations 3

Or request a complete-program rewrite:

python -m examples.shinkaevolve.run_example \
  --task function_minimization \
  --model openai/gpt-4o-mini \
  --host openrouter \
  --mutation-approach full_rewrite \
  --iterations 3

Both modes use the same parent selection, UCB model routing, novelty gate, and memory. This teaching port fixes the selected mutation approach for one run; upstream ShinkaEvolve can sample diff, full, and cross from configured probabilities per proposal. That within-run sampling is an explicit fidelity difference, not an omitted capability claim.

Repeat --model to expose multiple UCB arms:

python -m examples.shinkaevolve.run_example \
  --model openai/gpt-4o-mini \
  --model anthropic/claude-3.5-haiku \
  --iterations 5

The tutorial defaults to local task evaluation for a quick run. Select --eval-mode docker when checking the reproducible task container path.

Promote the example into a submission

Create a contribution branch and copy the complete artifact:

git switch -c contrib/shinkaevolve upstream/main
cp -r examples/shinkaevolve src/galapagos/scaffolds/shinkaevolve

Before submitting:

  1. Change the card controller to galapagos.scaffolds.shinkaevolve.scaffold.ShinkaEvolveScaffold.
  2. Replace organization: community-example with the contributor's identity.
  3. Re-check the upstream commit, license, scope, and fidelity statement.
  4. Remove tutorial-only assets that do not belong in the submitted bundle.
  5. Validate the promoted directory:
python scripts/validate_scaffold_submission.py \
  src/galapagos/scaffolds/shinkaevolve

Use the scaffold submission PR template. The focused PR runs submit-scaffold; it should not claim upstream approval.

Request upstream review separately

An upstream author or maintainer can add executable fidelity assertions in a separate PR:

cp -r porting_reviews/templates/scaffold \
  porting_reviews/scaffolds/shinkaevolve

That PR runs validate-scaffold. It answers whether the mapping preserves the upstream method, while the original submit-scaffold check answers whether the Galapagos artifact is portable and runnable.

For exact copy-and-paste instructions and every source file, read the repository example.