Write a recipe

A recipe is the method a deployment runs: which recorded traffic is eligible, how it becomes a batch, what signal that batch carries, and whether the candidate it produces replaces the served version.

Quickstart defines the term.

Accepting records, replaying them after a restart, holding a batch until it is acknowledged, committing algorithm state, running the backend, and publishing the next version are Reef's job, not the method's.

This page covers a weight recipe. For a harness-evolution method with propose, evaluate, and a selection policy against a fixed model, see Evolve your harness.

Rendering diagram…

The shaded steps are the method's. A processor judges each resolved unit. A unit consists of one record plus the reports that reference it. TRAIN batches it, WAIT holds it until its remaining references land, NEVER drops it. After the backend runs, the method's evaluator measures the candidate and its selector decides whether it is published.

Before you write one #

If an existing recipe's processor, preparer, loss family, gate, and surface already match your method, change its config instead. Re-read Choosing a recipe.

Build one #

A weight recipe is four pieces plus the class that binds them.

step preparera plain function turning a typed batch into a StepSignal: the loss family, the per-sample advantages, and the next algorithm state. No torch, Ray, or Slime import.

processordecides which reports are eligible and shapes the accepted ones into one typed batch

report typethe ReportBase subclass Reef validates at ingress, so a malformed report is HTTP 400 rather than a training-time surprise

candidate evaluationmeasures the checkpoint the backend exported and decides select or reject. Every recipe carries one; the default selects whatever the backend produced

recipe classa frozen dataclass whose training_spec() names the processor, the preparer (by dotted path), and the loss family

Python API is the contract for each. recipes/sao/ is the smallest cookbook implementation and the one to read alongside this page. Its four files total fewer than 200 lines: recipe.py, processor.py, preparer.py, and the slime/ loss family.

Configure it #

Keep your module in a package installed in the environment used by both the Reef service and the training driver, and verify the import they will perform:

python -c "from my_pkg.my_method import MyMethodRecipe"

Copy a weight-training config as described in Evolve your model and select the class by dotted path:

reef:
  recipe: "my_pkg.my_method:MyMethodRecipe"
  batch_size: 4

This fragment shows only the new keys; keep the model, storage, runtime, and services settings from the config you copied. Set training.global_batch_size to the same value, and add the driver flags your loss family requires (the mapping). The driver reads the same reef.recipe value from the deployment config and gets the loss family from the class's training_spec(). Do not repeat either value in the driver environment. Reef has no global recipe-implementation registry.

reef serve -c path/to/my-method.yaml

Gate a candidate #

A runtime finishes training by exporting a candidate. The recipe's candidate_evaluation decides what happens to it. A weight recipe declares its evaluator in the deployment config; the top-level evaluation section serves weight recipes only, and a harness recipe builds its evaluator in code instead:

evaluation:
  module: my_pkg.evaluation:build_evaluator
  config:
    benchmark: gsm8k
    threshold: 0.8

Reef calls the factory once per scenario with that opaque config and the scenario's training runtime. The trainer runs the plugin between the backend step and publication, calling evaluate before decide. A rejection leaves the previous version serving. Python API documents the plugin contract: evaluate, decide, the fail-closed rule, and idempotency by candidate.candidate_id. The section fields are in Configuration.

Where the feedback comes from #

Reef never invents feedback. Use whatever already judges your agent; for the numeric score field, a consistent scale where higher is better. If you have no number, Choosing a recipe lists the methods that need none.