Configure Reef serving and training

Local inference needs no YAML file:

uv run reef serve --inference.model-path Qwen/Qwen2.5-1.5B-Instruct

This starts SGLang on an automatically selected loopback port, waits for its health endpoint, then starts Reef on 127.0.0.1:8900. Use --inference.tensor-parallel-size 2 for two visible GPUs; the default is one. --inference.backend sglang makes the default backend explicit; --inference.backend vllm starts vllm.entrypoints.openai.api_server instead. The service interpreter (REEF_PYTHON, otherwise the launcher's interpreter) must have the selected engine and GPU-enabled PyTorch installed. The engine validates its GPU environment, model compatibility and available device memory during startup. Its output is available in .reef/run/sglang.log or .reef/run/vllm.log. The managed path currently supports a single GPU node. Use the SGLang installation guide or the vLLM installation guide to prepare the inference environment; the base Reef installation stays CPU-only.

Local model paths and Hugging Face IDs use the existing model resolver. Reef resolves a downloaded snapshot once and preserves the original model identifier as SGLang's served model name. Startup has a one-hour readiness deadline for SGLang and 30 seconds for Reef. On failure or interruption, Reef cleans up both processes. Logs live under .reef/run/. Local --inference.model-path cannot be combined with upstream URL/model selection; --model remains provider shorthand. Native engine options use inference.options as described below.

An external-provider deployment also needs no YAML file:

reef serve --inference.upstream-url http://localhost:8000 --inference.upstream-model my-model

Reef starts its core record-only recipe, listens on 127.0.0.1:8900, and stores state under .reef/ in the launch directory. It records inference and feedback without training weights. Use --reef.host or --reef.port to change the bind address. Logs live under .reef/run/. Reef checks its own HTTP readiness, runs in the foreground, and cleans up its process on Ctrl-C; it does not launch or stop the upstream provider. Readiness does not verify provider credentials or model availability.

REEF_UPSTREAM_URL, REEF_UPSTREAM_MODEL, REEF_UPSTREAM_API_KEY and REEF_TOKEN supply optional environment fallbacks for this mode. Explicit CLI settings win. --model ollama/my-model fills the Ollama endpoint and model; --model openai/my-model uses REEF_UPSTREAM_API_KEY. A model ID with any other prefix still needs an upstream URL.

Configuration file layout #

The canonical CLI paths match the sections of a configuration file. For example, the local inference command above can also use:

reef:
  host: 127.0.0.1
  port: 8900
inference:
  backend: sglang
  model-path: Qwen/Qwen2.5-1.5B-Instruct
  tensor-parallel-size: 1
  options:
    mem-fraction-static: 0.8
recipe:
  implementation: recipe

Run it with reef serve -c reef.yaml --inference.options.mem-fraction-static 0.7. An explicit CLI value overrides the corresponding YAML value. Omitted public fields use their declared defaults. Hyphens and underscores are accepted in section names and declared YAML fields; supplying both spellings of one field is an error. Unknown sections and undeclared public fields are rejected.

The public layout groups fields by their owner:

• reef: HTTP host, port, authentication, run directory and readiness deadline.

• inference: model path, backend, TP size, provider connection and native options.

• recipe.implementation: the selected recipe class or preset.

• recipe.config: fields declared by that recipe.

• recipe.runtime: the runtime type and its declared settings.

• training: backend selection, startup and worker-call timeouts, Ray connection and native training options.

• storage: artifact backend and repository, work/cache directories and record retention.

• execution / executors: role placement and named executor profiles.

• evaluation / observability: their existing component-owned settings.

• pipeline: the stage classes that replace a weight-training recipe's own, with their config.

• generator: the generator service that writes and checks Harbor tasks and runs their rollouts.

• harness: how harness clients reach the deployment's services.

A configuration file contains no process definitions. service and services are rejected; HTTP settings belong in reef. The selected Recipe and inference or training backend determine the processes, dependencies and connections. training.config holds workload variables such as checkpoint directories; native training flags belong in training.options and engine flags in inference.options.

A weight-training stack trains and serves on separate processes and GPUs. The training backend runs a training component inside the HTTP service. Reef submits each job's ready data to that component as a bounded job, and the component saves checkpoints. Reef evaluates each saved checkpoint and publishes the selected ones as releases. The serving engines load only published releases, from files. With the default Slime backend and an engine (inference.backend: sglang or vllm), reef serve starts these services:

• inference: a local inference component (python -m reef.inference.remote) on a free loopback port. It starts the serving engines and loads each published release into them. An inference.remote section replaces it with a component on another host.

• distill-teacher: a frozen SGLang teacher engine, only when a teacher section is set.

• reef: the HTTP service. It hosts the Slime training component, which starts a Slime driver on Ray for each job. The driver starts Slime's workers on the GPUs of Slime's serving session when one runs (see Task jobs on Slime), and otherwise reserves the job's GPUs and releases them when the job ends.

The inference component and the HTTP service join the same Ray cluster (RAY_ADDRESS or training.ray-address), so the engines and the training workers reserve separate GPUs. With no Ray address, Reef starts a local cluster and stops it on exit; an existing cluster is left running. Model topology, optimizer settings and checkpoint paths still need the complete options for the selected recipe. Training components, serving and executors describes the job and serving contracts.

The same path supports CLI-only training with --recipe.implementation package.module:WeightRecipe, --inference.model-path and the corresponding --training.options.* flags. training.backend defaults to slime. The optional tinker backend trains LoRA adapters remotely without local GPUs. Tinker's sampler serves them by default. Reef's SGLang engines serve them instead when the inference section configures engines: the control plane downloads each release's adapter from its sampler path; see Train with Tinker. The spindle_full backend trains all the parameters of one Hugging Face revision on a Spindle deployment, and exports each release's weights to a Hugging Face repo; see Train all the parameters on Spindle. training.backend also accepts an installed reef.training_backends entry-point name or an importable package.module:Deployment class. The selected definition validates its settings. It names the services that the HTTP service depends on. It also describes the training component that the HTTP service builds. Serving is not the backend's: the inference section selects it (see Serving of a weight-training deployment). Other backends do not inherit Slime's Ray or native-argument requirements.

An in-process integration can use InProcessTrainingDeployment: it starts only Reef HTTP, whose registered runtime factory builds the training component inside that process. training.options is parsed by that runtime factory, with CLI leaf overrides taking precedence over YAML. Runtime type, model and shared timeout settings cannot be overridden inside the options map. Such integrations reject Ray and training executor settings.

training.ready-timeout (default 3600 seconds) bounds the startup of the services that the training backend adds, such as the inference component and the teacher engine. reef.ready-timeout (default 30 seconds) bounds HTTP startup. When the HTTP service loads the model into training workers or starts serving engines before it answers, as with Slime and whenever Reef's engines serve, Reef raises reef.ready-timeout to at least training.ready-timeout.

training.timeout-s bounds each call to the training workers, in seconds. With Slime, it bounds each worker call of a job, such as training and checkpoint saves; when omitted, the bound is four hours. With Tinker, it bounds each remote call. For example, set training: {timeout-s: 43200} in YAML or pass --training.timeout-s 43200 to allow calls of up to 12 hours. The value must be a positive finite number. A timeout stops waiting, not the work on the workers. The job then fails; Reef does not retry it, and the next job starts from the last saved checkpoint.

Serving of a weight-training deployment #

The inference section alone decides how a weight-training deployment serves its releases. The training backend builds only its training component, and that component declares the format of its releases: full weights, or a PEFT adapter with its rank and target modules.

The engine settings are inference.backend, inference.num-gpus, inference.tensor-parallel-size, inference.options, inference.remote, inference.handler-factory, inference.handler-config, inference.max-loaded-adapters and inference.prefix-sharing. Another serving mode refuses them. The engines start with the release format: an adapter format enables SGLang's adapter table with the adapter's rank and target modules. Releases without weight files, such as Tinker's, are served by engines only when the training backend names how to fetch their weights (TrainingDeployment.engine_weights); Tinker's does. Otherwise the HTTP service refuses that pairing when it starts.

The control plane checks the inference component every 30 seconds. When the component has restarted, or an engine has failed, the control plane pauses generation, starts the lost engines again, loads each scenario's served release into them, and resumes.

Engine-serving inference uses the same inference fields as standalone serving. inference.num-gpus is the total inference GPU budget; inference.tensor-parallel-size is the GPU count per engine (default 1). The total defaults to the per-engine count and must be a positive multiple of it. For example, 4 GPUs with tensor parallel size 2 creates two engines. Standalone serving currently supports one engine, so its total must equal its tensor parallel size. An external provider does not accept local GPU requests.

inference:
  model-path: Qwen/Qwen2.5-1.5B-Instruct
  num-gpus: 1
  tensor-parallel-size: 1
  options:
    mem-fraction-static: 0.6
    router-port: 30000  # SGLang engines of a training stack only
training:
  backend: slime
  options:
    actor-num-nodes: 1
    actor-num-gpus-per-node: 1
    global-batch-size: 32
    # Add the recipe's optimizer, model and checkpoint options here.

CLI overrides use the same parser, for example --inference.num-gpus 4 --inference.tensor-parallel-size 2 or --inference.options.mem-fraction-static 0.7. Native engine options use SGLang's names without a sglang- prefix. Reef builds the engines' input from the inference section and the release format; Slime starts no serving engine. router-port binds the first router's HTTP port, and other supported router flags keep their native router-* names. router-ip is refused, because the serving engines use no external router. The standalone engine launcher does not include a router.

For the SGLang engines of a training stack, each engine probes free ports on its host at startup. A probe reserves nothing, so another process, such as another stack on the same host network, can bind a probed port first. If the engine then fails to start and another process holds one of its ports, Reef stops the engine and starts it again on new ports. Reef does the same when another process listens on the engine's HTTP port and answers the readiness check. Reef makes at most three startup attempts per engine. Other startup failures, and any failure of an encoder engine, stop the launch.

When Reef starts a router, it probes a free metrics port on the host, from port 4000 up. It also probes a free HTTP port, unless the router uses router-port. Only the first model's router uses router-port. The router serves both ports on the host's address. If the router then fails to start and another process holds one of these ports, Reef stops the router and starts it again on new ports. Reef does the same when another process listens on the router's HTTP port and answers the readiness check. Reef makes at most three attempts. A configured router-port does not change, so a conflict on it stops the launch. Another stack's metrics port can take a router-port near 4000, so use a port outside that range.

Encoder engines keep the weights they started with. Published weights load only into the other engines of the model. When a model with an encoder group trains, keep its vision encoder frozen.

Slime's own engine and placement options in training.options (rollout-num-gpus, colocate, sglang-*, router-* and the rest) reach each job's Slime driver unread. They configure the rollout engines of Slime's serving session, which the first task job starts (see Task jobs on Slime), never the serving engines; before that session runs, a driver of any other job starts no rollout engines. Reef's serving never shares GPUs with training, so training.colocate is refused. Native options cannot override managed model, placement or parallelism settings. Pipeline/data parallel and prefill/decode-disaggregated inference topologies are not supported by this managed path yet. Reef binds training.options.hf-checkpoint to inference.model-path; an explicit value must agree. Slime also needs an explicit training.options.global-batch-size, the samples in one optimizer step: Slime cuts each job's data into optimizer batches of that size.

The SGLang integration lives in reef.inference.sglang. Its native engine launch and control do not depend on Slime. The serving runtime turns the inference section and the release format into plain engine configuration data (reef.inference.remote.runtime.serving_engine_config), and the inference component starts the engines from that data. See Independent SGLang backend for the boundary.

Training GPU capacity is set in training.options.actor-num-* (and, for rollout engines on GPUs of their own, rollout-num-gpus); the physical node size is training.options.num-gpus-per-node. Until the first task job, each job's driver reserves those GPUs while it runs, so between jobs they are free, and a job waits while another reservation holds them. From the first task job on, Slime's serving session holds them until the component closes. The inference component reserves inference.num-gpus GPUs on the same Ray cluster, and the inference control process takes one Ray CPU and no GPU. When a training worker fails, the running job fails, and the next job's driver starts again from the latest checkpoint. A configuration error that only Slime's workers detect fails the first job. Engine failures are handled by the inference component (see Engine supervision).

inference.backend: vllm selects the vLLM engine integration. The inference component launches one vLLM server per engine on its GPUs with VLLM_SERVER_DEV_MODE=1, Reef's connector and --logprobs-mode processed_logprobs. It loads each published release through vLLM's control routes: /pause?mode=keep (a retracting pause adds /reset_prefix_cache?reset_running_requests=true), /update_weight_version, /collective_rpc reload_weights and the /v1/load_lora_adapter routes. Engines are single-node; more than one engine needs router_url.

Each engine takes one port, probed upward from engine_port_base (default 15000) on its host. Reef can lose that port to another process. This happens when the engine fails to start and another process holds the port. It also happens when another process listens on the port when the engine becomes ready. In both cases, Reef stops the engine and starts it again on a new port. Each engine gets at most three startup attempts.

Engine options use vLLM's engine-argument names. Reef sets model, host, port and tensor_parallel_size itself and refuses enable_sleep_mode. It defaults generation_config to vllm, so the model's own generation defaults cannot add truncation or a temperature the trainer never sees. It rejects kv_offloading_size (list OffloadingConnector in kv_transfer_config instead) and enables prefix caching only under a retracting pause.

vLLM engines serve full-weight releases only. The HTTP service refuses adapter releases, such as Slime's with training.options.megatron-lora-rank or Tinker's, when it starts. Reef starts no vLLM router, so one engine serves the stack: inference.num-gpus must equal inference.tensor-parallel-size. Engine options come from inference.options in vLLM's own names.

Reef's serving engines run on an inference component, local or remote. The HTTP service uploads each published release to the component as files:

• Full-weight training publishes the checkpoint's Hugging Face export. The component reloads it in place of the served weights. Each load moves the whole model, for example about 16 GB for an 8B bf16 model.

• LoRA training (training.options.megatron-lora-rank) publishes the scenario's PEFT adapter. The component loads it as the scenario's adapter beside the base model. Only inference.backend: sglang serves LoRA adapters.

The HTTP service reads each release from the checkpoint directory. With multi-node training, put the checkpoint directory on storage that the HTTP service's host can read.

Reef starts only its own inference, training and HTTP services. PRM and user-simulation services are independently deployed by OpenClawRL; Reef does not discover, launch, schedule, probe or stop them. The recipe consumes recipe.config.prm-url and recipe.config.prm-tokenizer-path, with the same CLI-over-YAML precedence as other recipe fields. Its client owns request timeouts and error handling. The example's Docker Compose owns auxiliary model commands, health checks and GPU allocation, with separate devices from Reef/Slime.

Managed engine launches use one generic builder. A backend definition supplies its command template, public parameter bindings, reserved aliases and HTTP health path. Adding an engine with this launch contract does not require a backend-specific deploy module or a second process lifecycle implementation. Reef supplies the SGLang and vLLM definitions; --inference.backend selects one.

The launcher translates public paths to the existing internal service and recipe contracts before starting children. Config references such as ${reef.port} and ${inference.model-path} use the same field mapping. The HTTP service binds to loopback by default. The reef section contains only declared HTTP settings; model and recipe fields belong to their owning sections.

Train without an inference engine #

inference.backend: none runs no inference engine. Use it for supervised fine-tuning on data that does not come from the served model, such as uploaded trajectory items (POST /reef/namespaces/{owner}/data) or imported records (POST /reef/records). Only the Slime backend supports it. The trainer takes every GPU that training.options.actor-* names:

inference:
  model-path: /models/Qwen3-8B   # the weights to train; nothing serves them
  backend: none
training:
  backend: slime
  options:
    actor-num-nodes: "1"
    actor-num-gpus-per-node: "8"
    global-batch-size: "32"
    # Add the recipe's optimizer, model and checkpoint options here.

Training in this mode works as follows:

• Each job trains its data on Slime's workers, which stay on their GPUs between jobs.

• Each job saves one checkpoint: the Megatron checkpoint and its HF export. This mode refuses an evaluation section, so Reef publishes every checkpoint as a release. See Set the checkpoint cadence.

• A restart continues from the last saved checkpoint, as with an engine.

The deployment serves no inference. /v1/chat/completions, /v1/responses and /v1/messages answer 404 for every scenario, because no scenario serves a model. To use a trained version, serve its published checkpoint from a separate deployment.

reef serve refuses the settings that configure an engine or call the model: inference.num-gpus (other than 0), inference.tensor-parallel-size, inference.options, inference.url, inference.remote, inference.handler-factory, inference.handler-config, training.colocate, a teacher engine, an evaluation section, a generator section and a composite recipe.

Native backend options #

Public fields and backend-specific flags share the same CLI-over-YAML merge. For managed SGLang, use:

uv run reef serve --inference.model-path Qwen/Qwen2.5-1.5B-Instruct \
  --inference.options.mem-fraction-static 0.8 \
  --inference.options.trust-remote-code true

These become native --mem-fraction-static=0.8 and --trust-remote-code arguments to python -m sglang.launch_server. SGLang owns their types, defaults and validation. Reef does not duplicate the engine argument schema. For argv-based engines such as Slime, true emits a switch and false or null omits it. In-process training passes values to its runtime parser; false stays false and null follows the declared field type. To disable an engine feature enabled by default, use that engine's native disabling flag. Lists supply multiple argument values; objects are passed as JSON. Use native flag names without their leading -- inside options.

A field override preserves its YAML siblings; an explicit whole object, such as --inference.options '{}', replaces the entire object. Model, served-model name, TP size, bind address, authentication and unsupported multi-node launch controls cannot be overridden through native options in managed serving. Use the public fields for those settings.

For Slime, a training stack can contain:

training:
  options:
    lr: 0.000001
    use-critic: true

Override an individual native flag with reef serve -c training.yaml --training.options.lr 0.000002. The normalized options reach each job's Slime driver through the effective config. The training component parses them once at startup, which refuses a malformed option before the first job. The driver takes out Reef's own options, such as --reef-checkpoint-policy, and the loss family's options, then passes the rest to Slime's native parser.

inference.handler-config has a different owner: it configures Reef's selected inference.handler-factory adapter, for example its tool parser. The factory path must name an InferenceHandler subclass with a from_config class method; handler functions are not supported. It does not configure the managed SGLang process. Executor options and recipe-owned option objects likewise stay with their selected components.

Two token-native handlers render the prompt with the model's chat template and call the engine's native generate route. reef.inference.sglang.chat.SGLangInferenceHandler calls SGLang /generate and relays its incremental stream. reef.inference.vllm.chat.VLLMInferenceHandler calls vLLM /inference/v1/generate; it serves buffered responses and answers a streaming request with the completed turn as one burst of protocol frames. The answer is the provider response without training tensors. The handler returns token log probabilities only to a client that sends the OpenAI logprobs: true. Training renders each recorded call itself (see Chat template arguments).

Both handlers accept tool_call_parser (the engine's parser name), sampling_defaults, force_reasoning, chat_template_kwargs and rollout_continuation. Any other option, such as capture_topk, is unknown and fails the handler's start. Per-request sampling extras pass through sglang_sampling_params or vllm_sampling_params.

A scenario can set temperature, top_p and top_k over sampling_defaults while it serves, with its serving config. A scenario's value is a default as well: it applies when a request does not set that field. Both chat protocols carry these three fields, so a value applies alike to OpenAI chat and Anthropic messages requests.

chat_template_kwargs renders a request that omits them, for example {enable_thinking: false} for a hybrid-thinking model served without reasoning; a request's own chat_template_kwargs win key by key.

rollout_continuation: true serves a multi-turn rollout from its exact sampled tokens. The client names the rollout in the OpenAI user field and sends each later request as the previous messages, the sampled assistant turn and the new user or tool messages. The handler renders only the new messages and appends them to the previous call's tokens, so a template that rewrites earlier turns, as Qwen3's drops their reasoning, cannot change the prompt that the engine continues. A request without user renders as usual. A request that names a rollout but does not extend it is refused.

A vLLM engine serving a training stack also needs Reef's connector, which stamps every sampled token with the weight version that produced it:

--kv-transfer-config '{"kv_connector": "ReefConnector", "kv_connector_module_path": "reef.inference.vllm.connector", "kv_role": "kv_both"}'

The connector moves no KV and composes with another connector under vLLM's MultiConnector. To combine it with vLLM's native CPU offloading, list OffloadingConnector and ReefConnector as MultiConnector children in --kv-transfer-config; the --kv-offloading-size flag replaces the configured connector, so it cannot be combined with this one. Without the connector the engine reports no runtime load ID: a scenario that trains rejects the response, and a scenario that does not train serves it under its release id.

rollout_log_probs are the sampled tokens' log probabilities that a training component's own task rollouts record; serving records none. They must mean the same thing as the trainer's. Sampling runs through:

raw logits -> penalties, logit_bias -> / temperature -> [A] -> top-k, top-p, min-p -> [B] -> sample

SGLang and Slime's trainer both read at [A] (full vocabulary, trainer with rollout_temperature, no penalties), so they agree as long as a recipe uses no penalties or logit_bias. vLLM --logprobs-mode processed_logprobs reads at [B], so it matches only with top-k, top-p and min-p off. Reef's vLLM client rejects a request whose effective top_p is below 1, top_k above 0 or min_p above 0. Verify with Slime's train_rollout_logprob_abs_diff on identical weights before training.

Both handlers split each reply into content and reasoning_content with the rules of SGLang's Qwen3 reasoning parser: <think> starts reasoning, and the first </think> or <tool_call> ends it. A reply that ends inside reasoning has an empty content.

Set inference.handler-config.force_reasoning to true if the chat template pre-opens <think>, or false if it does not. When omitted, the handler checks whether each request's rendered prompt ends with <think>. The check follows the request's chat_template_kwargs: a Qwen3.5 template pre-opens <think> by default but closes it for enable_thinking: false, so such a request keeps its reply in content. An explicit value bypasses this check.

Chat template arguments #

Serving records the provider exchange only. The training component renders each recorded call with the model's chat template, and must render it as the serving engine did. Give the training component the same chat template arguments that the engine renders with: the handler's chat_template_kwargs, or the arguments that the clients send.

• Slime renders with the tokenizer of --hf-checkpoint and training.options.apply-chat-template-kwargs, a JSON object.

• Tinker renders with the base model's tokenizer and training.options.chat_template_kwargs, a mapping.

inference:
  handler-config:
    chat_template_kwargs: {enable_thinking: false}
training:
  backend: slime
  options:
    apply-chat-template-kwargs: '{"enable_thinking": false}'

A sample that cannot render, or that is longer than a limit of its stage, does not train. The job metric unrenderable_samples counts these samples. A chat template that drops the reasoning of earlier turns, as Qwen3's does, cannot render a multi-turn sample whose earlier trained turns carry reasoning.

Explicit file selection #

Relative config paths resolve from the directory where you run the command. With no -c, Reef uses CLI inference inputs; it does not discover REEF_CONFIG or ./reef.yaml. File deployments must use reef serve -c reef.yaml or reef serve -c "$REEF_CONFIG" instead of relying on the previous implicit discovery. REEF_CONFIG remains the internal way the launcher passes effective settings to its HTTP child. A selected missing or invalid file is an error, even when provider flags are present. Reef reports the selected config source and does not search the Reef installation for your config. From outside a checkout, pass an absolute path to a cookbook config.

Before any model download or process start, the launcher logs the selected source (file or command line) and then every resolved setting with where its value came from:

[reef] config: /srv/reef/stack.yaml
[reef] resolved settings:
[reef]   recipe.implementation = recipes.sao.recipe:SAORecipe  (file)
[reef]   reef.host = 127.0.0.1  (default)
[reef]   reef.port = 9000  (command line)
[reef]   reef.tokens = ["****"]  (file)
[reef]   inference.model-path = /models/demo  (file)
[reef]   inference.backend = sglang  (automatic)
[reef]   training.backend = slime  (automatic)

Sources are file, command line, environment (the REEF_* variables a configuration-free start reads, or a field's declared fallback variable), automatic for a choice Reef made because the field was omitted, and default. Settings left at their defaults are not listed except the recipe and the HTTP bind. Tokens, API keys, passwords and database URLs are masked by key name at any depth, including inside native options objects.

reef serve ... --print-config prints the same report on standard output, defaults included, and exits with status 0 (or 2 for an invalid config) without downloading a model, allocating GPUs or starting a process. It takes the same -c, --model and override flags as a real start, so it shows exactly what that start would use. A Hugging Face model path is shown as written; the snapshot is resolved only at startup.

reef serve -c stack.yaml --reef.port 9000 --print-config

Relative state paths use the launch directory. Values interpolate from the environment with ${VAR} and from the config itself with ${dotted.path}. Use full public paths for command-line overrides: --inference.model-path /models/demo or --training.config.checkpoint_dir /tmp/ckpt. Logs go to .reef/run/; set reef.run-dir to move them.

Configuration-free startup accepts public settings and the selected weight recipe's declared fields, plus native inference or training options for the selected launch mode. Unknown public flags are rejected. Declared recipe runtimes are constructed by the HTTP child. Method-specific processes are prepared by the selected recipe's Python deployment hook. The effective settings are handed to the child using a private temporary config, removed when the launcher exits. No user YAML file is created.

Public service settings use the same argument parser for YAML and CLI values. Their types, defaults, and help are declared on ServiceConfig. Explicit CLI values override YAML values; omitted values use the setting's default. Run reef serve --help to see these options. Use --inference.upstream-model as the canonical spelling. Compatibility aliases include --upstream-model, --upstream_model and --reef.upstream_model. The last explicit CLI spelling of a setting wins. --recipe.implementation overrides the deployment's recipe setting; a bare --recipe <name> is refused because built-in profiles were removed.

String settings retain their text: --inference.upstream-model 00123 remains 00123. Numeric and boolean settings are parsed according to their declared type; invalid values fail before model downloads or process startup. Booleans accept an explicit value or a bare flag; a negative flag can disable a YAML setting:

reef serve -c stack.yaml --reef.port 9000 --no-reef.allow-implicit-scenario-creation

List and object options take one quoted JSON/YAML value. Empty lists and objects are preserved, and an explicit container replaces the YAML value:

reef serve -c stack.yaml --reef.tokens '[]' \
  --inference.handler-config '{"tool_call_parser": "qwen25"}'

The parsed public values are also supplied to service commands and the HTTP child's config. An empty string, as an unset ${VAR} reference expands to, is treated as an omitted service value and keeps its default. An explicit null is a value: optional fields resolve to null and any other field rejects it (see below). reef.token and reef.tokens remain distinct inputs whose credentials are combined. The selected Recipe, runtime adapter and executor settings use the same field parser.

Component configuration #

Standalone recipe files loaded by embedding scripts use the same layout. load_recipe_config and recipe_config_from_file_layout translate it into the existing recipe construction contract. Recipes may declare opaque sections in config_sections; for example, HarnessEvolutionRecipe owns recipe.config.evolution. Those sections use object/leaf CLI overrides, then their recipe validates the payload. Executor placement remains shared with the deployment and reaches dotted recipes as well as named presets.

After selecting recipe.implementation, Reef loads that class's declarations without constructing the recipe. A dotted weight-training recipe exposes its fields as --recipe.config.<field>, for example --recipe.config.training-mode, with the aliases --training-mode / --training_mode / --reef.training_mode. Other dotted recipes use the same --recipe.config.* namespace; their --reef.data.* spellings remain accepted aliases. reef serve -c stack.yaml --help includes the selected component's flags; basic reef serve --help does not load a recipe. The selected package must be importable in the launcher and child environments.

Declared component fields follow CLI > YAML > declared environment fallback > dataclass default. False, zero, empty strings and empty containers remain explicit values. Strings are not guessed as YAML scalars. The last CLI alias wins, and boolean fields support --no-.... A declaration that conflicts with a public option is rejected. The normalized values retain their types when handed to the HTTP child.

null is distinct from omission for every declared field: an optional field (str | None and similar) resolves to null, and writing null for any other field is an error naming the field, rather than a silent fall back to its default. Omit the field to use the default. The literal string null remains text for string fields. A file that repeats a YAML key, reef.port twice for example, is rejected with both line numbers instead of the last occurrence silently winning, and two spellings of one field (model-path and model_path) are a duplicate field error. Recipe floats retain their historical non-finite support; a recipe can declare allow_nonfinite=False to require finite values. Service fields and backend resource/timeouts require finite values. Errors identify the field and expected type without echoing its value.

Executor fields are declared by ExecutorSettings and WorkerResources:

reef serve -c stack.yaml \
  --execution.evolution.workers 4 \
  --execution.evolution.resources.cpus-per-worker 0.5

A nested override of a named executor profile makes a local copy for that role; it does not modify the shared profile. Backend options objects remain owned by the selected backend. Slime's native model/optimizer flags continue through Slime's own parser; Reef does not duplicate that schema.

A non-weight dotted recipe can supply recipe.runtime to select a registered or dotted runtime factory. Its declared fields use names such as --recipe.runtime.timeout-s. The inference proxy declares its connection settings. Custom RuntimeFactory implementations can opt in with config_type(); legacy callable factories keep receiving their existing mapping. Unknown fields in a declared component schema fail validation. Recipe-specific sections and opaque adapter option objects continue to be validated by their owning component.

If a service exits before readiness or exceeds its ready_timeout, Reef stops the stack and reports the service, the failure reason, and the local log directory. An exited service's message includes its exit code. CLI startup failures exit with status 1; invalid configuration exits with status 2. Child output is retained in the logs and forwarded to the terminal. Interrupting startup with SIGINT or SIGTERM also stops the services already launched, including when they are still loading a model.

The basic and SAO example run.sh launchers wait for the HTTP health endpoint and stop waiting if Reef exits. Startup deadlines are configured through reef.ready-timeout; the scripts do not add a second deadline. Each HTTP probe has a five-second timeout. Startup errors are recorded in work/reef.log, and exiting the script stops the Reef process it started.

Use ${VAR:?} for a required environment variable, for example upstream_model: ${REEF_UPSTREAM_MODEL:?}. If it is unset, empty, or only whitespace, Reef reports the missing variable names and their config fields before downloading models or starting processes. Command-line overrides are applied before this check, so --inference.upstream-model <model-id> can supply the value instead. Plain ${VAR} keeps resolving to an empty string when unset; use it for optional values such as an API key for a provider without authentication.

REEF_PYTHON defaults to the interpreter that launched reef serve and can be overridden in the environment. Use it when a service must share Reef's Python environment. A literal python keeps its normal meaning and is resolved from that service's PATH; Reef never rewrites command names.

Start from a cookbook stack #

The source checkout's runnable stacks live under the recipes/ cookbook and tutorials/: the learn-nothing ones use the core recipe implementation in recipes/basic/, each weight-training method owns its examples, and harness evolution ships as a tutorial.

Each weight-training example ships its stack as serve.yaml. recipes/sao/examples/imo_answerbench/serve.yaml is the smallest, two GPUs for one actor and one rollout engine; recipes/tttd/examples/tttd/serve.yaml adds LoRA training, and recipes/openclawrl/examples/openclawrl/serve.yaml adds a PRM engine and a student model.

Service and storage settings #

The HTTP service reads these fields from the recipe, reef, inference, harness and storage sections.

Storage paths default under .reef/, which the basic and sao stacks keep; the openclawrl stack overrides them to /var/lib/reef. Point them somewhere persistent.

Training completion updates consumption progress without retiring stored bodies. Storage manages cleanup independently: the HTTP service runs a sweep at startup and every 60 seconds. When the total record-body size exceeds the configured budget, storage evicts the oldest records by stored creation time, breaking ties by append sequence. Both consumed and unconsumed data are eligible, with one exception: storage keeps the model calls that a stored training sample holds. A sample that a recipe made from model calls stores their IDs and its own fields, such as its reward and training tokens, and renders the calls when it is read. Records do not expire by age.

The budget measures the stored UTF-8 JSON of payloads, references and artifact references across active and archived scenarios. A scenario's store keeps each model call as ATIF and links it to the earlier call with the same tools whose messages are the longest prefix of its own. It keeps only the messages that the call adds, and the tools once per trajectory. Thus the history and the tools that an agent resends in every call count once. Evicting a record frees its own JSON, and its messages once no remaining call continues from them. SQLite shares the budget across the record directory; PostgreSQL shares it across the deployment schema. Each eviction logs a warning with the scenario, record count, sequence range and the body bytes that it freed. Per-scenario loss totals survive restart. The pipeline status exposes record_data_incomplete, evicted_record_count and evicted_body_bytes after a loss; metrics also expose records/evicted_count and records/data_incomplete. These describe missing stored data, not proof that an already committed model missed those examples. Pending reports whose inputs are no longer stored are skipped with a warning; remaining records continue through the usual processor.

Retry hashes and commit metadata survive eviction, so retrying an upload does not restore evicted records or train them again. A consumer needing multiple passes must tolerate missing records once capacity eviction occurs.

This is a body budget, not a hard filesystem quota or an emergency disk-full handler. Concurrent writes can exceed it between sweeps. Indexes, retry hashes, commit histories and WAL require additional disk headroom. SQLite reuses freed pages without automatically shrinking its file. Cleanup failures are logged and retried on the next sweep. Standalone stores need an explicit maintenance call; see Python API.

Existing stores are upgraded transactionally. Older retirement markers and receipts become consumption records without deleting original bodies. The old columns, indexes and receipt table are removed; PostgreSQL advances its schema version to 3. Training commits store only consumption progress. Already deleted bodies cannot be recovered. Do not share stores between old and new writers, or downgrade without restoring a compatible backup.

Recipe settings such as training_mode, and any others the recipe declares with config_field, go under recipe.config. A field the selected recipe does not declare is an error. With a named preset, the recipe reads its configuration from the preset file.

Recipe configuration #

A configuration file puts declared recipe fields under recipe.config and the runtime under recipe.runtime. Harness settings live in recipe.config.evolution.

recipe.config.training_mode is shared by all recipes and defaults to auto. It has two values:

• auto: a job starts when the buffer holds job_samples samples: the untrained samples that the next job takes. POST /reef/train is refused.

• manual: a job starts only for a POST /reef/train instruction. The job trains the manual buffer: the samples that a client chose with the buffer routes (Choose the samples that train). New samples do not join it by themselves, in any recipe. A weight recipe waits until the manual buffer holds a sample, then trains the whole buffer; the instruction text goes into the commit's training_request metrics. Harness evolution runs the instruction at once: the method gets the instruction and the traces of the manual buffer, possibly none.

Records arrive and the sample store fills in both modes, so a switch from manual to auto can train on data already collected. A switch from auto to manual puts the samples that the next automatic job would take into the manual buffer. A recipe's Sample2Job declares supported_training_modes; an unsupported mode raises NotImplementedError. Harness evolution supports both modes, and every method takes a request step. Setting manual on an inference-only recipe does not create a training backend.

recipe:
  config:
    training_mode: manual

The reef serve option --recipe.config.training-mode manual sets the same field at deployment. The mode controls training initiation, independently of the activation rule. Use POST /reef/scenarios/{scenario}/update to select another mode at runtime. This changes subsequent job data; reserved job data completes under its original mode. Reef stores the selected mode in the scenario's settings, so reloads and restarts apply it again, and it takes precedence over the configured mode. A stored mode that the recipe does not support keeps the configured mode and logs a warning. In every mode, the samples of ready record sets wait in the scenario's sample store until a job trains them.

recipe.config.job_parameters is an object that a weight recipe sends with every job, beside method. The training component reads it: for example scheduling (Step scheduling) or save_interval. The component refuses a parameter that it does not take, and the job is not submitted. The recipe refuses a method key, because the recipe's objective is the method. A recipe that overrides parameters_by_kind decides where the setting goes.

recipe:
  config:
    job_parameters:
      scheduling:
        packing_length: 8192
      save_interval: 50

A recipe is selected three ways:

• The core record-only recipe: recipe.implementation: recipe

• A dotted class: recipe.implementation: "my_pkg.my_method:MyMethodRecipe"

• A named preset: recipe.implementation: my-preset, resolved to my-preset.yaml under REEF_RECIPE_CONFIG_DIR

There is no recipe-implementation registry. A bare name other than recipe is always a preset name; it never imports a learning method implicitly.

A dotted class does not have to be pip-installed. reef serve looks for its top-level package beside the config, walking up from the config file's directory to the nearest ancestor that holds <package>/__init__.py, and appends that directory to PYTHONPATH for every service it starts. This is how recipes.sao.recipe:SAORecipe resolves from a source checkout: the recipes/ cookbook sits next to the example's serve.yaml, so the launcher does not export PYTHONPATH itself. Entries already in PYTHONPATH keep their precedence.

REEF_RECIPE_CONFIG_DIR is the directory preset YAML is read from, and it has no default: a bare recipe name resolves to a preset only when it is set. Reef bundles no presets.

A preset uses the configuration file layout and is read as-is. ${VAR} interpolates in a deployment config, never in a preset. A preset carries its recipe.implementation and recipe.config, plus an optional recipe.runtime when the recipe builds its own runtime instead of using the deployment's upstream proxy. inference.upstream-model or inference.model-path names the preset's model. When using the deployment's runtime, a preset may omit the model to inherit that runtime's model (inference.upstream-model for an upstream proxy). An explicit model takes precedence. A preset with its own recipe.runtime must still name its own model. Harness-evolution presets also carry recipe.config.evolution:

recipe:
  implementation: reef.recipe.harness:HarnessEvolutionRecipe
  config:
    evolution:
      adapter: pi
      method: methods.mine:MyMethod
      evaluation:
        tasks: ["..."]
        scorer: methods.mine:MyScorer
inference:
  upstream-model: qwen3-8b

The preset's recipe.implementation is recipe or a dotted recipe class. Weight-training recipes are selected directly by dotted class in the deployment config, so the service can build their training component. Presets suit recipes whose runtime can be built from the preset or the deployment's upstream proxy. There, recipe.config holds the training mode and the recipe's own sections, and pipeline holds the stage config.

A composite recipe serves and evolves several release components in one scenario, one recipe per recipe component. Its components object maps each recipe component name to a section with the shape of the top-level recipe section. In that section, implementation names the recipe class, and config holds the recipe's fields and sections. A recipe component also keeps its own pipeline stage config under config.

A recipe component has no runtime: every recipe component uses the deployment's model, inference runtime and training component. A composite whose recipe component trains weights is deployed the way that recipe is. training.backend selects the backend. Its training component and inference runtime serve every recipe component. A top-level pipeline or evaluation section goes to the recipe component that trains weights.

A scenario's root release keeps one directory per recipe component. Reef writes each recipe component's default start there: its seed, and the model snapshot under the directory of the recipe component that trains weights. A recipe component whose recipe seeds nothing starts empty. Each recipe component's trainer runs as its own worker and commits into the same release chain. Every step therefore checkpoints, and the recipe components share one training_mode. Reef admits a report when the contract of any recipe component accepts it. Each trainer keeps the reports its own contract parses:

recipe:
  implementation: reef.recipe.composite:CompositeRecipe
  config:
    components:
      harness:
        implementation: reef.recipe.harness:HarnessEvolutionRecipe
        config:
          evolution:
            adapter: pi
            method: methods.mine:MyMethod
            evaluation: {tasks: ["..."], scorer: methods.mine:MyScorer}
      config:
        implementation: my_pkg.config:ConfigRecipe
inference:
  upstream-model: qwen3-8b

The same composite with a recipe component that trains weights is a deployment file. Its dotted class selects the composite, and training.backend selects the backend. The stage config of the recipe component that trains weights goes under its config.pipeline:

recipe:
  implementation: reef.recipe.composite:CompositeRecipe
  config:
    components:
      weights:
        implementation: recipes.sao.recipe:SAORecipe
        config:
          pipeline: {sample2job: {config: {job_samples: 8, buffer_size: 32}}}
      harness:
        implementation: reef.recipe.harness:HarnessEvolutionRecipe
        config:
          evolution: {adapter: pi, method: methods.mine:MyMethod, evaluation: {tasks: ["..."], scorer: methods.mine:MyScorer}}
inference:
  model-path: Qwen/Qwen3-8B
training:
  backend: slime

Slime's --reef-executor-backend option selects the executor of its training workers and defaults to auto (Slime's Ray launcher). See Slime worker executors for its contract.

Stack execution backends #

Reef chooses native process placement in backend implementation code; execution.services is rejected. execution.training selects the executor of Slime's training workers (default auto, resolving to ray). The serving engines' executor belongs to the inference component. execution.rollout selects it, for every training backend. All selectors accept a backend name/import path or a profile under executors. Inline objects/profiles accept backend (default auto), options, workers, and resources containing cpus_per_worker and gpus_per_worker. Counts must be positive integers; resource quantities must be finite nonnegative numbers. Numeric environment interpolation is accepted.

execution.generator selects the executor of the generator service a generator section adds (default auto, one local process); it accepts the service executor settings below, with one worker.

execution.evolution selects harness evaluation workers (default auto). Unlike Slime, ordinary harness evolution calls external model endpoints and does not need local GPUs: one worker selects uni, multiple workers select mp. Local GPU worker requirements follow the same topology rule after checking visible CUDA capacity; insufficient GPUs fail with a prompt to explicitly select ray. Multiple workers inside a Ray placement group or declared cluster/actor options select ray. Local allocations require whole GPUs; fractional reservations require explicit Ray. The former worker-level local executor is removed (the separate rollout-isolation option is unchanged). Omitted resources retain component defaults (normally one CPU and no GPUs). CPU-only local executors do not reserve cores or enforce CPU quotas.

Service executors accept the same resources, mapped to per-service launch requests, but workers must be one: service replicas are not implemented. The training and rollout roles reject these generic workers/resources fields; their model-parallel topology and placement groups still come from Slime's existing training configuration. Resource declarations are never silently treated as model-parallel resizing.

For services, auto selects uni unless resource/worker options or an existing Ray placement group call for Ray. Explicit local CUDA visibility selects uni and cannot be combined with cluster resource options. Explicit service selectors and Slime CLI flags override the corresponding role defaults. Backend startup failures never silently fall back to another backend.

For Ray services, resources supplies actor launch options such as num_gpus and num_cpus.

Harness evolution keys #

These keys go under recipe.config.evolution. Evolve your harness describes what each one changes, and Write a harness method describes the method contract. A key may use underscores or hyphens, and startup refuses an unknown key.

The default stages train on every valid scored report as a trace, including successful outcomes. A step starts when pipeline.sample2job.config.job_samples traces are held, 1 by default. To train on recorded traffic alone, every job_samples requests with unscored samples, set pipeline.record2sample.module: reef.record2sample.traces:InferenceSamples. Reef refuses batch_size and batch_policy directly under recipe.config.

To evolve offline, on the train split of the scenario's imported tasks instead of on traffic, set pipeline.sample2job.module: reef.sample2job.traces:TrainTaskJobData. Each uploaded train task then joins the buffer and trains once, in upload order. A step holds either traces or train tasks; a step of train tasks rolls out up to pipeline.sample2job.config.buffer_size of them (default job_samples) on the head tree as the method's traces. Startup refuses it without an adapter that plays task directories (an npm install) and evolution.evaluation.scorer.

evolution.evaluation decides how Reef evaluates a candidate and when it publishes one: the tasks, the scorer, and the publication policy. It takes one task source: tasks, task_manifest with tasks_root, or imported_tasks. The task source and scorer are required unless policy is none.

evolution:
  evaluation:
    tasks: ["[fib] Compute fib(90) and reply with the number alone."]
    scorer: harness.scoring:AnswerScorer
    policy: compare
    statistic: wins

The evaluation context is what a run's score depends on besides the tree and the task: the served model and the named models of evolution.models; the adapter, its version and evolution.binary; the scorer setting; task_timeout_s and forbid_residue; and the run executor, evolution.executor and evolution.sandbox. The history reuses a scored run only in the context it ran in, and never reuses a failed run.

The beta Reefine recipe runs its agent on pi with evolution.agent, adds evolution.agent.trial_timeout_s, and reads evolution.multimodal. Its README documents them.

The served model's binding is merged in when a run renders the tree; it never enters a published tree. The seed is the baseline the first candidate is measured against. The step records hold the method's raw model traffic and the clients' traces, and the history holds every evaluation run's session log, so treat evolution.state_dir like the commit history.

The inference section #

Read by every serving mode. inference.timeout-s limits one inference request; an engine that does not answer within it gets a 504.

An inference.remote section runs the engines that inference.backend names on a remote inference component instead of this host. The component's --engine must name the same backend. Serve from a remote GPU host shows the setup. Any training backend can serve on a remote component. Without the section, a training stack's engines run on a component that reef serve starts on this host. The release format sets what the engines serve: one scenario's full weights, or one adapter per scenario. The section tells Reef how to reach the component:

The training section #

Read by the weight-training stack. See Evolve your model for how to size it.

Slime fills architecture flags such as layer counts and hidden sizes from inference.model-path. Do not put them in the config.

The activation section #

Training makes artifacts: every checkpoint that a weight training job saves, and every tree that a harness trainer keeps. A harness trainer keeps the trees that its evolution.evaluation publishes (see Harness evolution keys). Training never decides whether an artifact serves. The activation rule that this section names decides that. The section applies to every recipe, a composite included. Without the section, the rule is always. A scenario can select one of the built-in rules for itself with POST /reef/scenarios/{scenario}/update (Scenario updates). The section's rule applies to each scenario that selects none.

An artifact that the rule does not activate stays inactive. A person can activate it with POST /reef/scenarios/{scenario}/activate. Activation does not change where training continues. Activate an artifact describes activation, and Go back to an older artifact describes how to make an older artifact serve for good.

activation:
  module: always
  config:
    review_kinds: [extension]

The built-in rules:

• always activates every new artifact, except a harness tree that changes a kind in review_kinds (a list, default empty). Each name is a kind of the evolve section of the adapter's adapter.yaml, such as rules, config, skill, command, extension or native's native_* kinds; the harness recipe refuses a name that the adapter does not evolve. A tree that changes a kind that is always reviewed, such as native's native_loop, waits for a person whether or not the list names it.

• never activates no artifact, for fully manual activation. It takes no config.

• evaluated activates an artifact once an evaluation of it finishes with a mean_score of at least min_mean_score (required). Set the scenario's artifact_evaluation (Automatic evaluation) so Reef evaluates each new artifact. If a scenario has no artifact_evaluation under this rule, Reef logs a warning when it creates or loads the scenario. Such a scenario activates no new artifact.

A rule of your own subclasses ActivationRule:

• activates(made) decides when training publishes an artifact. made names the component and the kinds of the harness tree that the artifact changes.

• activates_evaluated(artifact_id, evaluation) decides when an evaluation of the artifact finishes.

Reef checks the class and builds the rule when it loads the deployment config, so a bad rule fails at startup. Reef rejects functions, abstract classes and structural lookalikes.

The pipeline section #

A recipe's pipeline has two stages (see Training pipeline). The recipe names a default class for each stage. This section gives each stage its config, and it can name another class for a stage. Every stage reads only its own config; the recipe's recipe.config holds no stage settings.

recipe:
  implementation: recipes.tttd.recipe:TTTDRecipe
pipeline:
  record2sample:
    config:
      groups_per_step: 8
      rollouts_per_group: 64

A stage without a key is the recipe's class with no config. Any other key under pipeline fails the startup. Each recipe's page lists the keys of its stages. Keys are written as the stage reads them; a hyphen spelling is also accepted.

Every built-in record-to-sample stage that holds units reads max_staleness, Reef's staleness filter. Reef drops held training data whose producing artifact is more than this many served artifacts behind the artifact that the scenario serves. The limit is a record-to-sample setting, not a recipe setting: recipe.config.max_staleness is an error.

pipeline:
  record2sample:
    config:
      max_staleness: 2

NoSamples and SFTSamples refuse the key: the first holds no unit, and the second never drops an SFT target as stale. A training framework's own off-policy limit goes in that backend's training.options. No built-in backend has one now. See Staleness.

Reef submits one training job for each job's data. job_samples is the fewest samples that one job trains. A weight recipe requires it: a deployment without it fails the recipe build with a RecipeConfigError that names pipeline.sample2job.config.job_samples. The harness evolution stages default it to 1.

The training component cuts each job's data into optimizer batches by its own batch size: Slime's --global-batch-size (training.config.global_batch_size in the examples) or Tinker's batch_size. Usually, set job_samples to that batch size, so that one job is one optimizer step:

pipeline:
  sample2job:
    config:
      job_samples: ${training.config.global_batch_size}

The two values can differ. Job data takes up to buffer_size samples and the rest of each group that those samples touch, so it can hold more than job_samples: a group of rollouts trains in one job, and a job with more samples than the batch size trains more than one optimizer step. A job with fewer samples than the batch size trains one partial step. The remainder field of a job's optional scheduling parameter decides what happens to a shorter last step; by default, the component trains it as a smaller step. See Job size.

Every built-in sample-to-job stage that trains on records reads job_samples and buffer_size. A recipe's page lists the other keys of its stage.

Reef checks each stage when it loads the recipe config, before a scenario starts: it imports a named class, rejects functions and abstract classes, and parses the stage's config. An unknown key or a value out of range fails the startup. A stage key under recipe.config, such as groups_per_step, is an unknown recipe key and also fails the startup.

For each scenario, Reef builds the stages in pipeline order with the class method from_config. The sample-to-job stage receives the record-to-sample stage, and it reads a value that the record-to-sample stage owns from it. For example, the TTTD step grid is set once, in pipeline.record2sample.config. A stage class without from_config cannot be named here.

Generated tasks #

The built-in generated-task recipe, reef.recipe.generated_tasks:GeneratedTaskRecipe, binds the task_group objective on the importance_sampling loss family: Tinker's built-in loss, or on Slime truncated importance sampling at --tis-clip (Task jobs on Slime). Its stages are GeneratedTaskSamples and GeneratedTaskJobData. Without pipeline.record2sample.config.rounds no round runs, and the recipe trains only the train tasks that earlier generations saved in the scenario. Each job is a task job: the training component rolls the job's tasks out on its own sampler and trains on their rollouts, as recipe.config.job_parameters.rollout says. The recipe requires that parameter.

recipe:
  implementation: reef.recipe.generated_tasks:GeneratedTaskRecipe
  config:
    job_parameters:
      rollout:
        rollouts_per_task: 4
        environment: docker
pipeline:
  record2sample:
    config:
      jobs_per_round: 2
      generator_url: ${endpoints.generator}
      rounds: 20
      description: shell tasks with hidden state under /var and /etc
      generator:
        module: recipes.beta.spade:SpadeTaskGenerator
        config:
          generator_url: ${endpoints.generator}
          skills: [inspection, repair]
          state_dir: ${REEF_STATE_DIR}/generations
  sample2job:
    config:
      buffer_size: 2

GeneratedTaskSamples paces the rounds, holds their source sessions, runs the generations, and sends their results to the generator. It trains on no report: the recipe trains on the rollouts that the training component runs, which are not records.

With rounds enabled, each valid task and its split are saved in the scenario's task store. A generation saves tasks and runs no rollouts; the saved train tasks run later, and their results are kept in the store. The task files go in the task file store; see Choose the task file store. Related source sessions keep the same split across generations. A task that joins existing train and eval source groups is rejected. Eval tasks are saved without rollouts or Designer rewards. The generator's pass-rate band does not filter the store.

Set eval_fraction and split_seed here rather than in a generator's configuration. The defaults keep every task in train. A published task keeps its split. A change of these values applies to the tasks generated after it.

The saved train tasks go to the scenario's sample store, as samples generated-tasks/<task name>, and train from the buffer like every other sample. GeneratedTaskJobData makes one task job of the buffer's tasks, whatever Tinker's batch size (training.options.batch-size) is:

The Tinker component runs a task job's rollouts on Tinker's sampler at the job's start weights, and the Slime component on its own rollout engines (see Task jobs on Slime). The rollout job parameter configures them:

Task jobs on Slime #

The Slime component runs task jobs on Slime's own rollout engines: the SGLang engines of Slime's serving session (Slime's ServingCluster), which a task job's driver drives through Slime's RolloutManager as Slime's own training loop does. Reef's serving never calls them. The first task job starts the session; it holds the engines, their router and the training GPUs until the component closes, so the next jobs attach to the started engines, and every job trains on the session's GPUs. A failed or cancelled job leaves the session running for the next job. The save root's reef-serving.json records the session, which the component disposes when it closes. training.options.rollout-num-gpus gives the engines GPUs of their own; training.options.colocate puts them on the training GPUs (all of them, unless rollout-num-gpus names fewer), and Slime moves the GPU memory between the engines and the workers. Slime's sglang-*, router-* and rollout-num-gpus-per-engine options configure the engines.

training:
  backend: slime
  options:
    colocate: true
    rollout-num-gpus-per-engine: 1
    sglang-mem-fraction-static: 0.5

A task job loads the workers' weights into the engines, runs the rollouts the rollout parameter describes, frees the colocated engines' memory, and trains the samples of each task's group with the job's objective. The engines answer OpenAI text chat: system, user and assistant text messages, no tools. A task job whose options give the engines no GPUs fails. A LoRA actor cannot have rollout engines, since Reef's adapters do not reach Slime's weight transfer.

The job's objective must have a loss family that Slime trains; the component checks it when the job is submitted, before any rollout. The generated-task objective, task_group, trains on the importance_sampling family: the loss of a response token is -A * sg[min(p / q, c)] * log p, with q the engines' log-probability of the sampled token, c Slime's --tis-clip (default 2) and Slime's per-sample mean as the reduction. Tinker's loss of that name takes the untruncated ratio and sums the tokens. Reef computes the advantages, so set disable-compute-advantages-and-returns.

With score-centering: true in training.options, the engines also record the top score-centering-top-k log-probabilities of each response token, and the job trains with score centering (Loss families). The recorded distribution must be the one the trainer reads, so the job needs rollout-temperature 1, rollout-top-p 1 and rollout-top-k -1, and the engines answer a request that sets another temperature, top-p or top-k with 400.

training:
  backend: slime
  options:
    colocate: true
    disable-compute-advantages-and-returns: true
    tis-clip: 2.0
    score-centering: true
    score-centering-top-k: 128

Dataset jobs on Slime #

A dataset job trains a prompt dataset the way Slime trains on its own (RFC #1041). Slime rolls the prompts out on its serving session's engines, scores them with its reward model, computes the advantages and trains its loss; Reef records each saved rollout as a candidate. The deployment selects the built-in recipe reef.recipe.dataset:DatasetRecipe with data.training_mode: manual. Its loss family, slime, leaves the training signal to Slime and refuses every other job. Each training instruction (POST /reef/train) whose parameters hold dataset is one job:

{"text": "GRPO on the prompts", "session": "s", "artifact_id": "artifact-1",
 "parameters": {"dataset": {"path": "/data/prompts.jsonl", "epochs": 1}, "save_interval": 10}}

dataset.path is an absolute path, on the Ray cluster, to a JSONL file in Slime's prompt-data format. dataset.epochs (default 1) is the passes over it. The job trains len(dataset) // rollout-batch-size rollouts per epoch, and save_interval counts rollouts between checkpoints (default: one checkpoint at the end). A checkpoint's step is the job's rollouts up to it, and its reward is the rollout's mean raw reward. A resumed job continues after its last checkpoint and skips the prompts of the rollouts it saved. The recipe's job_parameters can set a default dataset.

Slime's own options in training.options configure the rest:

training:
  backend: slime
  options:
    colocate: true
    input-key: messages
    label-key: label
    apply-chat-template: true
    rollout-batch-size: 32
    n-samples-per-prompt: 8
    global-batch-size: 256
    rollout-max-response-len: 1024
    rollout-temperature: 1.0
    advantage-estimator: grpo
    custom-rm-path: my_package.reward.reward
    loss-type: policy_loss

A rollout function (rollout-function-path, custom-generate-function-path), a reward model (custom-rm-path or rm-type) and a hook on the rollout data after the advantages (rollout-data-postprocess-path) are Slime's plugins and run in Slime's processes. Reef sets data-source-path itself: its data source serves each job's dataset to the serving session's rollout manager, which outlives the jobs, so the option is refused. disable-compute-advantages-and-returns is refused too.

Reef calls the generator class's from_config(config, *, scenario, environ) once for each scenario. A generator that calls a remote service reads its credentials through environment references in config. The generator interface and the round are in Training pipeline.

A source session is a report, without a training arm, whose inference records all carry source_tag. The generator service accepts at most 512 KiB of records and files in one request, so keep source_records_per_round small when the sessions are long.

The generator section #

Absent by default. When present, reef serve starts the generator service (python -m reef.service.deploy.generator_process) before the HTTP service, which depends on it, and publishes its address as ${endpoints.generator} for the generated-task stages to consume. The generator writes and checks Harbor tasks for the generated-task recipe, and runs the plays of a task generator such as SPADE, under the same interpreter as the Reef service (REEF_PYTHON, otherwise the launcher's). Its host needs the harbor command line and the Harbor environment that generator.environment selects; the Reef service itself needs neither. The default, docker, needs a Docker daemon on the host. e2b runs each task in an E2B cloud sandbox, so a host without Docker, such as a cluster node with only Apptainer, uses it. It needs E2B_API_KEY exported where reef serve starts and reef-infra[e2b] in the generator's interpreter; the readiness check looks there, so generator.harbor must come from the same Python environment. With e2b, a task runs only its environment/Dockerfile (no compose services), an agent inside the sandbox cannot reach a generator.agent-host on a private host, and E2B builds a template for each new task image. execution.generator selects the generator's executor, so a deployment can place it on another host. The generator runs its proposals, checks and rollouts at the same time. generator.concurrency limits only the rollouts in flight, so a proposal or a check never waits behind rollouts.

generator:
  tasks-root: ${REEF_SPADE_STATE_DIR}/tasks
execution:
  generator: auto
pipeline:
  record2sample:
    config:
      generator_url: ${endpoints.generator}

The generator reads the resolved deployment through REEF_CONFIG like the other children: the Reef address from reef.host and reef.port, the token from reef.token, and the served model from inference.model-path or inference.upstream-model. Without reef serve, python -m reef.service.deploy.generator_process -c serve.yaml runs it from the same file.

The task-import section #

Absent by default. When present, each scenario imports the Harbor tasks of a split manifest when it opens, as its imported tasks. See Import Harbor tasks.

Experiment tracking #

Tracking is optional, off by default, and belongs to a Reef scenario rather than to one training backend. The same provider-neutral logger is shared by the recipe, the processor, backend results, and the commit lifecycle. Install reef[wandb] when the training extra does not already provide it.

observability:
  wandb:
    enabled: true
    project: reef
    entity: your-team             # optional
    group_prefix: prod-us-east    # optional scenario-group namespace
    name_prefix: baseline         # optional run-name prefix
    tags: [openclawrl, qwen]
    mode: online                  # online, offline, or disabled
    directory: /var/lib/reef/wandb
    upload_checkpoints: false

Export WANDB_API_KEY before starting, or log in once with the credential store on the cluster.

online sends data to the project. offline makes no network calls and writes syncable data below directory for a later wandb sync. disabled makes no calls even when enabled is true.

Each scenario maps to a group named after the scenario or <group_prefix>/<scenario>. Within it, Reef opens one run when the scenario binds and another after each train-from. The deterministic run id includes those identities, so restarting resumes the same run with resume=allow. A train-from finishes the current run, marks its summary with the new start (reef/train_from_release_id) and the active release (reef/active_release_id), and resets train/step to zero. An activation does not end a run. The globally monotonic reef/step stays attached for joining a run back to the commit history. A scenario with several trainers shares the run: each step's metrics carry its component name as a prefix (harness/train/loss) on the same train/step axis with reef/component on the row, its optimizer step rows land under <component>/step/* on their own counter, and the run config lists each component's backend under reef.components and backend.<component> with reef.backend null. The names train, step, reef and operations are these prefixes, so a composite recipe refuses them as component names.

Recipe and processor code logs through the same object without importing W&B:

experiment_logger.log({"temperature": 0.6}, namespace="recipe")
self.experiment_logger.log({"accepted": 12}, namespace="processor")

Those become recipe/* and processor/*, each namespace on its own <namespace>/event axis. Only finite numeric values are sent.

Operational metrics #

With W&B enabled, the dispatcher samples each loaded scenario, including inference-only scenarios, every 10 seconds and once during graceful shutdown. Samples use the existing scenario run under operations/*, with Unix time in operations/time_seconds as their horizontal axis. They do not advance train/step and do not wait for a successful commit or a /reef/status request. Offline mode records the same samples locally; disabled tracking starts no sampling thread.

The following names are relative to operations/:

All operation families expose started_total, active (in-flight count), elapsed_seconds (age of the oldest active call, zero when idle), completed_total, failed_total, duration_seconds_total, and last_duration_seconds after a call finishes. Execution calls are serial, so their active count is zero or one. Request and admission calls can overlap. Durations are in seconds; cumulative durations sum individual calls, so concurrent work may accumulate faster than wall time. Divide the change in total duration by the change in completed plus failed count to obtain a mean completed-call latency for an interval. A completed execution call is not necessarily a commit: a poll of a running job that saved no new checkpoint commits nothing. These measurements reset when their trainer is rebuilt, including recovery; training/started_at_seconds identifies that reset. Request and ingestion measurements reset when the scenario is rebuilt, including recovery, with operations/started_at_seconds identifying that reset. They are not persisted training history. Malformed HTTP payloads/headers and failures before scenario resolution are outside these scenario measurements. No arbitrary scenario names, rejection messages, or record IDs become metric keys. Ingestion counters start at the dispatcher's typed-record boundary; wire-payload normalization failures are outside that boundary.

Additional finite numeric processor status fields appear under operations/processor/. Sampling does not advance processor readiness or consume its queue. If ingestion or commit holds the trainer lock, that sample omits queue gauges while still reporting execution measurements. Operational samples contain no request bodies, record IDs, or error messages. They are low-frequency service and training diagnostics; underlying inference engines retain their own high-frequency monitoring. Uploading these values does not configure alert notifications.

Commit correlation #

Durable commit metrics carry experiment/provider, experiment/project, experiment/group, and experiment/run_id. Use them to open the run from a Reef version, and use the run's reef/training_job_id to go the other way.

The job listing, GET /reef/scenarios/{scenario}/jobs, gives the run of each training job as experiment. Control writes it into the job's work item when its tracker records the job, so every service that lists jobs reads it. Its url is the URL that the open W&B run gives. An offline run gives none: the URL is then https://wandb.ai/<entity>/<project>/runs/<run_id>, where the run is once it is synced, with the configured entity. Without a configured entity, the URL of an offline run is null. Checkpoint paths are metadata only unless upload_checkpoints: true.

Import, initialization, logging, summary, and upload failures are reported in the service log and never fail a training step or its commit.

Record tracing #

Record tracing exports every accepted record and every committed training step as OpenTelemetry spans, so a tracing backend shows what an agent did in a scenario and which version it trained. It is optional, off by default, and independent of experiment tracking. Reef speaks OTLP over HTTP and names no vendor: point it at Langfuse, Arize Phoenix, Jaeger, Grafana Tempo or an OpenTelemetry Collector. Install reef-infra[opentelemetry].

observability:
  tracing:
    enabled: true
    endpoint: https://cloud.langfuse.com/api/public/otel/v1/traces  # full OTLP/HTTP traces URL
    authorization: ${LANGFUSE_AUTH}  # the backend credential, sent as the Authorization header
    service_name: reef              # optional resource service.name

endpoint is the complete traces URL. authorization is the credential the backend expects in its Authorization header: Basic <base64 public:secret> for Langfuse, Bearer <token> for most others. It is handled like inference.upstream_api_key: write it as an environment reference in the YAML, or omit it and export REEF_TRACING_AUTHORIZATION; the startup report masks it and it never appears in a span or a log line. A backend that expects its credential under another header name, such as x-api-key, takes it in the headers mapping; the startup report masks every header value, and Authorization itself is rejected there so the credential has one place. When no endpoint, credential or header is configured the exporter reads the standard OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS environment variables instead.

Each accepted inference record becomes the root span of its own trace, named chat <model>. Its trace and span ids derive from the scenario and record id, so a restart or a second Reef host produces the same ids. The span carries:

• session.id and reef.scenario: the scenario.

• reef.agent_record_id and reef.request_type: the receipt and record kind.

• gen_ai.request.model, gen_ai.response.model, gen_ai.response.id, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens and gen_ai.response.finish_reasons from the provider request and response, following the OpenTelemetry GenAI semantic conventions.

• reef.artifact_id: the artifact that answered the request. reef.release_id and reef.content_id remain legacy storage diagnostics when the record carries them; clients must not use them as artifact identities.

• reef.tags: the x-reef-tag-* request tags.

A feedback record becomes a feedback span inside the trace of the first inference it references, with reef.score and reef.references; further references are span links. A training instruction becomes a training request span. A committed step adds a training step N span with the commit's step, release, job id, consumed record counts, its component as reef.component when the scenario runs several trainers, and its scalar metrics as reef.metrics.*, plus one trained in step N child span below every record the step consumed, linked back to the commit span. Records carry one timestamp, so their spans have zero duration and start at the record's creation time.

For streamed provider responses, token usage is read from the captured SSE events, including Chat Completions usage chunks, Responses terminal events and Anthropic message usage. Cumulative counts are not summed across chunks. If the provider sends no usage, token counts remain absent; for Chat Completions, request stream_options: {include_usage: true} when supported. The stored response and the forwarded stream remain unchanged.

Spans carry the exchange itself: the request messages and the reply as JSON in gen_ai.input.messages and gen_ai.output.messages, feedback text in reef.feedback and instruction text in reef.instruction. The backend therefore sees the scenario's traffic; point tracing only at one trusted with it. Export failures are reported in the service log and never fail record acceptance or a commit; the exporter batches spans in a background thread and flushes them during graceful shutdown.