Review a port as an upstream author

A Galapagos task or scaffold can be mechanically valid and still misrepresent the work it adapts. Card parsing cannot tell whether a benchmark changed its metric, an invalid case became valid, or a scaffold lost the mechanism that made the published method distinct. Galapagos therefore supports an executable upstream-author review through a pull request.

This review complements the normal submission gate:

Gate	Question	Author
Task/scaffold submission	Does the exact Hub upload bundle load and run through Galapagos?	Port contributor
Upstream port review	Does that port faithfully preserve the upstream contract?	Upstream author/maintainer

Task port review

Copy the task template:

mkdir -p porting_reviews/tasks
cp -r porting_reviews/templates/task porting_reviews/tasks/<task-name>

Fill in review.yaml:

version: 1
kind: task
artifact: my_task
verdict: approved
reviewer:
  name: Upstream Author
  github: upstream-github-login
  relationship: first_author
upstream:
  paper: https://arxiv.org/abs/...
  repository: https://github.com/upstream/project
notes: "Checked the candidate interface, fixtures, validity rules, and score."

Then replace the example assertions in test_port.py. A useful task review normally checks:

• the public problem statement and exact candidate entry point;
• at least one known-valid and known-invalid candidate;
• a score or metric with an upstream-comparable expected value;
• fixture/split/protocol versions that materially affect results;
• any intentional difference from upstream.

The test receives a temporary directory containing only the Hub bundle:

import os
from pathlib import Path

from galapagos.tasks import load_task


def test_upstream_contract():
    root = Path(os.environ["GALAPAGOS_ARTIFACT_ROOT"])
    task = load_task(path=str(root), download_external=False)
    assert task.card.references["source"] == "the expected upstream source"
    # Add upstream-specific candidate/evaluator assertions here.

Remove the PORT_REVIEW_TEMPLATE marker and run:

python scripts/validate_task_port_review.py porting_reviews/tasks/<task-name>

Scaffold port review

Copy the scaffold template:

mkdir -p porting_reviews/scaffolds
cp -r porting_reviews/templates/scaffold porting_reviews/scaffolds/<scaffold-name>

The manifest has the same shape with kind: scaffold. Replace test_port.py with assertions about the method's six-component mapping:

• population and archive semantics;
• parent/inspiration selection policy;
• prompt construction;
• proposal/mutation behavior;
• the fact that evaluation remains task-owned;
• memory or adaptation across iterations;
• the mechanism and defaults that distinguish the upstream method.

Tests may use a fake model and a tiny fake task, but must not call a live or paid model API.

python scripts/validate_scaffold_port_review.py porting_reviews/scaffolds/<scaffold-name>

Open the review PR

Use the matching GitHub PR template:

• .github/PULL_REQUEST_TEMPLATE/task_port_review.md
• .github/PULL_REQUEST_TEMPLATE/scaffold_port_review.md

When review files change, GitHub Actions compares reviewer.github with the PR author's login. A maintainer separately verifies that the account belongs to the upstream author using the paper, canonical repository, or another public source. CI cannot prove that social relationship on its own.

The test is executed against a newly materialized galapagos submit bundle. After the review merges, changes to the reviewed task/scaffold automatically rerun it. A later change to the review test itself must again come from the GitHub account recorded in review.yaml.