Tasks and sessions

One inference returns a block of actions. To carry out a continuous task, your program repeatedly observes the robot, requests actions and reports what was executed. A session links these turns to record task progress and lets you look up each request and its execution feedback.

The local console manages this loop for you. To write your own control program, complete Your first API call, then add sessions using the flow below.

Create session → Observe → Infer actions → Execute and report → Observe again → … → Close session

All paths below are relative to the Base URL in your connection configuration and use Authorization: Bearer <API_KEY>. Keep the same service address and model ID throughout a session; select MODEL_ID from that service's /models list. See Cloud API for connection settings and request formats.

1. Create a sessionCopy Anchor Link

Send POST /sessions with the model for this task:

{"model":"MODEL_ID"}

Example response:

{
  "id":"session_1",
  "model":"MODEL_ID",
  "status":"open",
  "next_turn":1,
  "last_request_id":null,
  "history":[],
  "messages":[]
}

Save id and next_turn. Use them to associate subsequent requests with this session and turn.

2. Request actions for the current turnCopy Anchor Link

Send the current image, state and instruction to POST /chat/completions. Include the session ID and turn in metadata:

{
  "model":"MODEL_ID",
  "metadata":{"session_id":"session_1","turn":"1"},
  "messages":[{"role":"user","content":[
    {"type":"text","text":"{\"instruction\":\"Pick up the block\",\"state\":[-0.13,-0.46,0.49,0,1,0,0,1]}"},
    {"type":"image_url","image_url":{"url":"data:image/png;base64,<PNG_BYTES>"}}
  ]}]
}

turn is a string in the request. Replace <PNG_BYTES> with a real camera image and the sample state with the robot's current observation. See Observations and actions for their meaning.

The response's execute_actions tool call contains the action block. Save the complete choices[0].message, the X-Request-ID response header and metadata.next_turn. The next turn needs the original assistant message and tool call ID, not just the parsed action array.

3. Report execution and continueCopy Anchor Link

Your control program validates and executes the actions. After each block, create a tool receipt that accurately describes what it executed:

{
  "role":"tool",
  "tool_call_id":"call_0123456789abcdef0123456789abcdef",
  "content":"{\"executed_steps\":1,\"status\":\"completed\"}"
}

This example describes a block containing one step, fully dispatched, with that step's execution period finished. Use the actual tool call ID from the model response and the actual number of dispatched steps in your receipt.

A receipt records what the control program did. It does not establish physical tracking or completion of the task goal.

To continue inference, set metadata.turn to the previous response's next_turn and send three messages in order:

1. The original assistant message from the previous turn.
2. The tool execution receipt for that call.
3. A new user observation containing the current image, state and task instruction.

Wait for the current request to finish before starting another turn. Read the robot's state again; do not calculate a new state from the previously predicted actions.

4. Look up sessions and turnsCopy Anchor Link

Keep the session ID, turn and request ID to inspect the task context, returned actions and request processing separately.

For example, look up turn 1:

curl "$PLATFORM_BASE_URL/sessions/$SESSION_ID/turns/1" \
  -H "Authorization: Bearer $PLATFORM_API_KEY"

Set PLATFORM_BASE_URL and PLATFORM_API_KEY from your connection configuration, and SESSION_ID from the session creation response. A succeeded result means inference succeeded, not that the robot completed its task. Retrieving historical actions should not automatically execute them again.

5. End the task and close the sessionCopy Anchor Link

When ending a task, stop local action dispatch first. Then send the reason and any unreported messages to POST /sessions/{session_id}/close.

If the final receipt has already been submitted:

{"reason":"stopped","messages":[]}

If an action block is still unreported, messages must contain that turn's original assistant message followed by its tool receipt. For example, stopping before dispatching any steps produces a receipt of {"executed_steps":0,"status":"interrupted"}.

The response is the session object with status: closed. You can then use GET /sessions/{session_id} to inspect the closed status and final execution record. Stopping the robot must not depend on the close request succeeding.

Local tasks, runs and cloud sessionsCopy Anchor Link

RoboAPI uses these concepts for different records:

After stopping, restarting the same instruction without choosing a new task continues the current local task by default, but creates a new run and cloud session. To save a separate task, use the console's new-task control or set new_task: true in the local HTTP API's start request. MCP's robot_start_policy does not expose this parameter.

Sending an instruction from the browser during a run stops it first and creates a new task, run and cloud session, even if the text is unchanged. An HTTP/MCP active-instruction update takes effect at the next inference: a changed instruction starts a new local task record while the current run and cloud session continue.

RoboAPI local runtime requirementsCopy Anchor Link

When connecting a model service to the RoboAPI 0.4.5 local runtime, it must also meet the protocol below. The runtime validates responses and ends the run with invalid_response if required fields are missing or mismatched.

Setup and the console retrieve available models with GET /models and use the selected entry's actual ID. In the console, Connect, select a model, then Save. Fetching the list does not save credentials or create a session. See installation for connection setup.