Robot Brain · Programs, interaction, and learning

Better interaction.
Better experience.
More capable robots.

Two systems that connect model reasoning to physical action—and turn the outcome into the next improvement.

01 / NAVIGATIONASENACode as Policy → workspace RSI → recorded experience↗02 / MANIPULATIONRecovaReal → sim → real → continuous data collection↗03 / INTERACTIVE DEMOSee the loopA multi-turn program reacting to real observations↗

Explore each system: ASENA ↗Recova ↗

01 / ASENACode as Policy + RSI

ASENA: Self-Evolving Agents
for Embodied Navigation

ASENA connects a coding model to embodied tools, execution feedback and a persistent workspace.

Presentation slides ↗

ASENA / SYSTEM DESIGN

The agent composes; the robot stack executes

ASENA connects a coding agent and persistent workspace to embodied tools. The agent can delegate navigation to a learned policy or construct motion proposals; execution authority stays with the platform.

G1 robot geometry — Navigation-policy results came from simulation; G1 demonstrated supervised execution. Adapted from Anjie Cheng’s slide 4.

ASENA / WORKSPACE RSICode and skills improve · model weights stay fixed

Parallel workers improve a shared workspace.

Sixteen simulation workers start from separate copies of a shared workspace, develop reusable skills and retain execution feedback. Between passes, their improvements are merged into the shared workspace for the next round. All model weights stay fixed.

Fixed coding agentCoding model serves independent worker contextsEach worker has its own task context.

Task allocation · private workspace per worker

01Worker

Private code + skills

↕

Simulator + policy instance

02Worker

Private code + skills

↕

Simulator + policy instance

…16Worker

Private code + skills

↕

Simulator + policy instance

Gather traces, local edits and task outcomes between passes

Fixed improver modelMerge, test and prune code and skills

Distribute the updated notes and skills to the next pass.

↶Wₖ₊₁ = Improve(Wₖ, traces, feedback)

Implemented in simulation: workers execute independently, inspect peer work and exchange tips through a shared bulletin. Between passes, consolidation merges, tests and prunes code and notes. This diagram shows logical instances, not a one-GPU-per-worker allocation.

Workspace-evolution protocol · paper §4.4 ↗

ASENA / EXECUTION AUTHORITY

Safety is enforced outside the coding agent

The deployment bounds the agent and its subprocesses. Motion passes through checks outside the agent, operator approval, and a controller that maintains balance.

Restricted process
+ every child process

Within the boundaryCoding agent+ scripts / subprocesses

Scoped resources

Python · NumPy · MuJoCo
Scratch files + reusable skills
Robot device API
Model API proxy

Unix identityProtected files + credentials

Network allowlistDevice + model proxy only

Inherited seccompBlocks direct kernel/device paths

No direct path

Raw CAN /
packet sockets
Operator
credentials
Platform writes /
privilege escape

Checked execution path

Motion proposalWaypoints / joint trajectory
External validationBounds · joint limits
Robot self-collision checks
Operator approvalApprove or return a reason
SONIC → G1Balance + locomotion
Independent stop

Launch preflight verified the boundary.Reflection disabled motion.

MuJoCo gesture checks used the robot’s self model, not full environment physics. Supervision remained necessary. Slide 6.

How the workspace relates to agent trainingOptional comparison: workspace updates today; a proposed model-training extension

The existing workers execute programs through the simulation interface. The comparison below changes the update target from retained code and skills to the coding agent’s weights.

Fixed coding agentCoding model serves independent worker contextsEach worker has its own task context.

Task allocation · private workspace per worker

01Worker

Private code + skills

↕

Simulator + policy instance

02Worker

Private code + skills

↕

Simulator + policy instance

…16Worker

Private code + skills

↕

Simulator + policy instance

Gather traces, local edits and task outcomes between passes

Fixed improver modelMerge, test and prune code and skills

Distribute the updated notes and skills to the next pass.

↶Wₖ₊₁ = Improve(Wₖ, traces, feedback)

ASENA / POLICY & AGENTEvaluate the tool, then the combination

The navigation policy improves route geometry.

Reported monocular · 4B models

ASENA-VLN is the optional navigation tool the coding agent can call. Evaluated alone, it reaches closer to the goal on both benchmarks. Success improves on R2R; RxR shows a tradeoff between trajectory fidelity and successful completion.

R2R

Metric	Qwen-RobotNav 4B	ASENA-VLN 4B
Success ↑	66.9%	68.7%
Navigation error ↓	4.22 m	3.52 m
Path efficiency · SPL ↑	60.5	64.2

+1.8 points in success−0.70 m navigation error

RxR

Metric	Qwen-RobotNav 4B	ASENA-VLN 4B
Success ↑	71.3%	70.2%
Navigation error ↓	4.15 m	3.90 m
Path efficiency · SPL ↑	61.5	59.7
Trajectory fidelity · nDTW ↑	68.6	73.1

−1.1 points in success+4.5 points in nDTW

Reported policy-only results: ASENA, Table 1 ↗ and Qwen-RobotNav v3, Table 1 ↗. Both use monocular observations; training recipes, data mixtures and evaluation filtering differ.

What changes when the coding agent calls the policy?

Agent + tool · fresh workspaces

Delegating navigation reduces tool calls for both coding agents. Success improves for Sonnet, while Astra trades some success for fewer interactions.

Claude Sonnet 5

Adding ASENA-VLN
Metric	Agent only	With policy
Success	50%	61%
Tool calls / episode	103.83	38.91

+11 points in success62.5% fewer tool calls

GPT-6 Astra

Adding ASENA-VLN
Metric	Agent only	With policy
Success	93%	86%
Tool calls / episode	40.80	19.98

−7 points in success51.0% fewer tool calls

RxR Agentic Split, 100 tasks per configuration, before workspace evolution. Sonnet improves from 50% to 61% success; Astra drops from 93% to 86%. Tool calls are agent–tool interactions, not physical steps or elapsed time. Paper §4.3 and Table 3 ↗

ASENA / WORKSPACE EVOLUTIONPrograms and experience accumulate · weights stay fixed

The workspace improved across repeated tasks.

Recorded experiment · 10 passes

Workers retained code, notes and tested skills between passes. Navigation success increased while the coding model, improver and policy weights stayed fixed.

Success rate (%)16 workers · 100 tasks per benchmark

Pass over recurring task set

R2R + policy72 → 98%
R2R agent only60 → 97%
RxR + policy65 → 89%
RxR agent only48 → 80%

Protocol: 16 Claude Sonnet 5 workers at high effort; each pass repeats the same scenes, instructions and starting states. Traces are retained and replay is allowed, so these gains can include route and scene memory. The first pass belongs to this evolution campaign; it is separate from the fresh-workspace agent comparison above.

Source: ASENA, Fig. 4 and §4.4 ↗; recorded navigation series reproduced from An-Chieh Cheng’s slide 12. Download plotted values (CSV).

Inspect all ten passes

Success rate (%) · identical values to the plotted curves
Pass	R2R + policy	R2R agent only	RxR + policy	RxR agent only
1	72	60	65	48
2	79	70	70	57
3	86	78	68	61
4	91	89	70	68
5	90	93	70	70
6	93	94	78	74
7	95	93	80	77
8	98	95	83	79
9	98	96	86	80
10	98	97	89	80

ASENA / RECORDED EXPERIENCEReal G1 recordings

Replay what the robot saw and did.

ASENA records camera observations, measured robot motion and agent events in MCAP. The replay aligns the camera, joint states and body pose on the same clock, so an execution can be inspected after the run.

One recorded 10-second window: the camera, joint trace and body replay share the same clock. The rendered G1 follows measured robot states. The camera was recorded at approximately 2 Hz; repeated frames preserve its original timing.

Inspect the saved events and measured motionAgent requests · operator feedback · a two-second motion example

Keep the decisions and feedback, too.

Additional event channels

Agent proposal · recorded example

What did the agent request?

upper_body(motion="yes_circle")

A motion request from the snack mission, linked to its task context and execution record.

Operator & outcome

What was permitted? What worked?

Approval fields: decision + reason
Recorded task verdict: completed

Approval authorizes execution. The task verdict is a separate outcome signal.

Speech input · field structure

What did the person say?

via: "voice"
user_content: "…"

Accepted ASR text enters the agent event stream. This recorder does not store raw microphone audio.

The proposal and task verdict are recorded examples from a separate snack mission. Approval and speech fields show the interface structure; these cards are not synchronized events from the clip above.

A measured motion example

What happened over two seconds

Take the preceding camera observation and robot state, then measure the motion over the next two seconds.

Recorded interval: 582.5–584.5 s

Measured local motion

Forward: 0.466 m
Lateral: 0.191 m
Yaw: 0.369 rad

These values come from recorded odometry and describe the motion that actually followed the observation.

Recorded signals and provenance

The camera and measured-body replay come from deck-20260908-175129.mcap, bag time 579–589 s. The two-second motion example is a subset of that same recording. The separate snack-mission examples come from September 9; they are not labels for the walking clip.

The recorded channels include camera observations, robot state, control telemetry and agent events. Accepted voice text is logged in the event stream. The parallel simulation rollout pipeline is separate from this physical-robot recorder.

Paper §3.2 and Fig. 2 ↗ · Recorded-data walkthrough ↗ · Open the replay ↗

Possible reuse: curated real-robot recordings could later support SFT; an RL extension would use the simulation environment.

Next: Recova. Recova uses task and recovery experience to train manipulation policies that return to deployment. Explore the Recova tutorial ↗

02 / RECOVAKeep collection running

Recova: Agent-guided failure recovery
for autonomous robotic manipulation

Turn a failure that stops data collection into a skill that keeps the next rollout going.

The goal

Robots that keep collecting useful experience, with progressively less human intervention.

Real → sim → real → learning.

Select a stage to follow the loop

→ → →

← New failures and real demonstrations guide the next recovery skill and policy update

01 / Physical grounding

A twin tied to the real workstation.

Reconstruct the scene so task attempts and failure recovery can be explored before returning to hardware.

Output: a reconstructed environment for task and recovery exploration.

Recorded digital-twin demo with the real camera inset.

Why simulation matters

The agent can diagnose, write a correction, test it, and try again. Successful programs become reusable recovery procedures and generate training data.

Why real feedback matters

Contact and visual mismatch still need physical validation. New failure cases and human corrections expand what the system can handle next.

RECOVA / THE DATA ENGINERecovery closes the collection loop

Recover the scene.
Keep the experience.

RUNTask policy

→

MONITORFailure?

→

RESTORERecovery policy

→

VERIFYResume the task

When recovery is missing or unsuccessfulHuman demonstration → save the correction → expand recovery coverage

Task data

Successful task rollouts
+ human task demonstrations

↓

DAgger update → task policy

Recovery data

Verified recoveries
+ human recovery demonstrations

↓

Separate training → recovery policy

Twin trajectories initialize separate policies. Real DAgger updates happen between collection rounds; policy weights remain fixed within a round.

Parallel real-world collection1 operator
4 workstations

Monitor rollouts, coordinate recovery, request help when needed, and save experience to the appropriate data pool.

Mean real-robot task success

Initial task policy23.8%

After DAgger77.5%

+ recovery87.5%

4 tasks · 20 trials per task and configuration

↻

Autonomy improves by expanding recovery coverage. In one four-round tile-drawing study, human takeovers fell from 7 of 8 episodes to 0 of 7. Fully unattended, open-ended data collection remains the goal.

Does the robot invent a new skill online?

In the reported real-robot experiments, autonomous recoveries use the learned recovery policy. The coding agent develops and tests corrective programs in the digital twin; a new real failure can trigger a recovery instruction and a first human demonstration, followed by twin development, training and registration of the new skill.

The demonstrated loop is targeted failure recovery and separate policy improvement. A general agent that autonomously identifies every missing data regime, invents new collection tasks and trains indefinitely is a natural next direction, not an established result here.

03 / WATCH THE LOOPobservation → decision → code → action → feedback

The next action depends
on what just happened.

Follow a real recording one turn at a time. Pause on a decision, inspect the program, then see the feedback that changes the next turn.

Check whether the machine has my snack. Report back silently.

Recorded G1 demonstration

Desk → vending machine → person00:00

Code as PolicyTURN 01 / 07

OBSERVATION

DECISION

PROGRAMReadable pseudocode

FEEDBACK → NEXT TURN

Paused · advance at your own pace

State carried forward

Replay sources and timing

Open the complete demo clip ↗

The adjacent programs and decision summaries explain the control flow; they are not verbatim private reasoning or raw execution logs. Each turn plays its selected footage and holds briefly for reading. Step controls let you inspect it without speeding up the recording.

ASENA

Execute programs, inspect outcomes and retain useful skills.

Recova

Build the recovery loop that keeps embodied experience coming.

Better interaction.
Better experience.
More capable robots.

ASENA: Self-Evolving Agents
for Embodied Navigation

The agent composes; the robot stack executes

Observations

Coding agent

Learned VLN policy

Programmed motion

Validate
+ approve

SONIC

Parallel workers improve a shared workspace.

Safety is enforced outside the coding agent

Restricted process
+ every child process

The navigation policy improves route geometry.

R2R

RxR

What changes when the coding agent calls the policy?

Claude Sonnet 5

GPT-6 Astra

The workspace improved across repeated tasks.

Replay what the robot saw and did.

Keep the decisions and feedback, too.

What did the agent request?

What was permitted? What worked?

What did the person say?

What happened over two seconds

Recova: Agent-guided failure recovery
for autonomous robotic manipulation

Real → sim → real → learning.

A twin tied to the real workstation.

Recover the scene.
Keep the experience.

Successful task rollouts
+ human task demonstrations

Verified recoveries
+ human recovery demonstrations

The next action depends
on what just happened.

Pass	R2R + policy	R2R agent only	RxR + policy	RxR agent only
1	72	60	65	48
2	79	70	70	57
3	86	78	68	61
4	91	89	70	68
5	90	93	70	70
6	93	94	78	74
7	95	93	80	77
8	98	95	83	79
9	98	96	86	80
10	98	97	89	80

Pass	R2R + policy	R2R agent only	RxR + policy	RxR agent only
1	72	60	65	48
2	79	70	70	57
3	86	78	68	61
4	91	89	70	68
5	90	93	70	70
6	93	94	78	74
7	95	93	80	77
8	98	95	83	79
9	98	96	86	80
10	98	97	89	80

Better interaction.Better experience.More capable robots.

ASENA: Self-Evolving Agents for Embodied Navigation

The agent composes; the robot stack executes

Observations

Coding agent

π Learned VLN policy

Programmed motion

Validate + approve

SONIC

Parallel workers improve a shared workspace.

Safety is enforced outside the coding agent

Restricted process+ every child process

The navigation policy improves route geometry.

R2R

RxR

What changes when the coding agent calls the policy?

Claude Sonnet 5

GPT-6 Astra

The workspace improved across repeated tasks.

Replay what the robot saw and did.

Keep the decisions and feedback, too.

What did the agent request?

What was permitted? What worked?

What did the person say?

What happened over two seconds

Recova: Agent-guided failure recovery for autonomous robotic manipulation

Real → sim → real → learning.

A twin tied to the real workstation.

Recover the scene.Keep the experience.

Successful task rollouts+ human task demonstrations

Verified recoveries+ human recovery demonstrations

The next action dependson what just happened.

Better interaction.
Better experience.
More capable robots.

ASENA: Self-Evolving Agents
for Embodied Navigation

Learned VLN policy

Validate
+ approve

Restricted process
+ every child process

Recova: Agent-guided failure recovery
for autonomous robotic manipulation

Recover the scene.
Keep the experience.

Successful task rollouts
+ human task demonstrations

Verified recoveries
+ human recovery demonstrations

The next action depends
on what just happened.

Pass	R2R + policy	R2R agent only	RxR + policy	RxR agent only
1	72	60	65	48
2	79	70	70	57
3	86	78	68	61
4	91	89	70	68
5	90	93	70	70
6	93	94	78	74
7	95	93	80	77
8	98	95	83	79
9	98	96	86	80
10	98	97	89	80