Introducing RL runs: train robot policies in simulation from the API

Most robot learning still happens in someone's terminal. A reward gets tweaked, a job gets launched, and a day later someone watches a video to decide whether it worked. RL Studio, the trainer inside the Kite app, already runs that loop from a prompt. Today the same trainer is an API.

/v1/rl_runs trains a policy for a robot in simulation with reinforcement learning, from a declarative task spec. You say what to learn (walk on command, follow a reference motion, balance on an object), for which robot, and with what budget. Kite checks the spec for free, trains it on a GPU, and hands back a bundle: the policy as ONNX, the MuJoCo scene it trained in with a script that runs it, clips, and a report whose verdict comes from measured checks rather than a reward curve.

The walkthrough below is one real run: a fine-tune of our published Open Duck Mini v2 walk, 500 iterations on an L4.

One run, end to end

The spec

A run is one JSON body. This one asks for a steady forward walk, starting from the published policy:

{
  "robot": "open_duck_mini_v2",
  "title": "Open Duck: steady walk (fine-tuned)",
  "description": "Walk forward at a steady pace with a level body and alternating feet.",
  "task": {
    "objective": "velocity",
    "commands": { "vx": [0.2, 0.4], "vy": [-0.05, 0.05], "yaw": [-0.2, 0.2] }
  },
  "from": { "policy": "4d574ce4c547" },
  "budget": { "iterations": 500, "max_tokens": 400 },
  "hardware_tier": "gcp_gpu_l4",
  "outputs": {
    "formats": ["onnx", "pt", "torchscript"],
    "videos": [{ "command": { "vx": 0.3 }, "seconds": 8 }]
  }
}

Validate first

Before any GPU time, POST /v1/rl_runs/validate compiles the spec and rolls three canned policies through it on a CPU: zero action, random actions, and a published walk for the robot. It returns what each reward term pays each of them, and warns when a reward pays for the wrong thing. The classic case is a robot that learns standing still is nearly as good as walking. Validate creates nothing and costs nothing.

Here a real walk out-earns standing still by 1.2 per step: 1.0 of it from following the command and 0.4 from lifting its feet. Flailing falls over about every two seconds. That gap is what training has to find. If standing still earned most of the tracking reward, validate would say so and return the one-line change to the spec that fixes it.

Train

curl https://api.kiteml.com/v1/rl_runs \
  -H "Authorization: Bearer $KITE_API_KEY" \
  -H "Kite-Version: 2026-09-27" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d @walk.json

The run compiled the scene and trained 500 iterations in under seven minutes: 24.6 million environment steps across 2,048 parallel worlds. While it trains, GET /v1/rl_runs/{id} reports the reward per step and the same broken down by term, so you can see what the policy is learning instead of guessing from one number:

The verdict

When training ends, Kite compares the last checkpoints, keeps the best, and measures it: 24 worlds for 12 seconds each, with training noise and pushes on. The verdict comes from those checks. Here are both policies, measured the same way:

Tracking and posture improved. In exchange, two of the 24 worlds fell where the parent's never did, and the step rhythm loosened a little. Both stay well inside their limits, and this is exactly the kind of trade the report exists to make visible. The verdict is pass.

A vision model also reads the clip against your description. It can flag a passing run for review but never pass a failing one. On this run it agreed, with a caveat worth having:

The frames show the robot translating steadily toward the camera with a level torso and consistent upright posture, with feet swinging past one another rather than shuffling in place, so the forward-walk command is met. Gait quality is mediocre though.

Its suggestions are reward parameters to scale in the next fork.

What comes back

The bundle is one zip with everything needed to run the policy without Kite:

rlr_01M3MQ39HVAEBQ3F64K6JDF9XR/
  policy/policy.onnx     57 observations in, 14 joint targets out, 50 Hz
  policy/policy.json     every observation block, joint order, PD gains, rates
  policy/policy.pt       the training checkpoint, for fine-tuning
  mujoco/scene.xml       the MuJoCo model the policy trained in
  mujoco/play.py         runs the policy, with no Kite code
  media/                 clips and a contact sheet
  report.json            the verdict, its checks, the judge's read
  metrics.csv            the training curve, by term
  spec.json              everything needed to train it again

pip install mujoco onnxruntime numpy
cd rlr_01M3MQ39HVAEBQ3F64K6JDF9XR/mujoco
python play.py --headless --seconds 10
# {"seconds": 10.0, "distance_m": 2.939, "falls": 0, "command": [0.3, 0.0, 0.0]}

That's 2.94 m in 10 seconds against a 0.3 m/s command: 98% of the commanded speed, in plain MuJoCo, from the ONNX file. policy.json is the contract you'd port to a real robot: nine observation blocks, the action rule, the gains, and the control rate. The spec also asked for a clip at 0.3 m/s:

Built for iteration

RL rarely works the first time, so the API is shaped around short loops:

• Probe first. "budget": {"preset": "probe"} trains 300 iterations. That's enough to see whether the reward teaches the right thing, in minutes.
• Fork what almost works. "from": {"run": "rlr_…"} starts from a run's weights and training schedule, with your changes. The judge's suggestions make a good first edit.
• Run seeds side by side. "seeds": 3 starts three runs of one spec, and the SDK's best() keeps the winner.
• Cap the spend. budget.max_tokens stops a run at a token budget and keeps what it learned.

A probe is a first signal, not a finished policy. Here is a 300-iteration probe of the same walk from scratch, next to the fork:

The same loop without HTTP, from Python or the CLI:

import json
from kite_sdk import Kite

kite = Kite()
spec = json.load(open("walk.json"))
runs = kite.rl_runs.create(**spec)                  # one run per seed
best = kite.rl_runs.best(r.wait() for r in runs)    # pass beats needs_review, then reward
best.download("./policies")                         # checked against its sha256

kite rl train walk.json --wait
kite rl fork rlr_01M3MQ39HVAEBQ3F64K6JDF9XR --set air_time=4.2

Agents get the same flow through MCP: kite_rl_train, then kite_rl_status, whose next field is the exact command that downloads the bundle.

Get started

The trainer behind RL runs covers three robots and three kinds of task: walking on command, balancing on an object in the scene, and following a reference motion. Here are some policies trained with it in RL Studio, the app built on the same trainer:

Each one is a single spec: a robot, an objective, and a few settings. The balancing runs add an object to task.scene.items and point task.item at it. The spin and the laps are velocity runs with different command ranges. The shove-proof run tunes how often the simulator pushes the robot.

Training on an L4 costs 1,000 tokens per hour, billed by the minute, and only for runs that produce a policy. The example above cost 117 tokens. Every run is pinned to a dated engine, so a spec keeps training the same way until you change the date.

To get a key, open the Kite app, click your name in the sidebar, and choose Developers → API keys with the rl_runs scopes. The guide walks through this same run with every request and response. The API reference has every field.

Read the RL runs guide→