Zero2Skill: Bootstrapping Robot Skills through Autonomous Data Collection, Training, and Deployment

Abstract

In long-horizon collection, the same failure modes recur across episodes — and a correction scoped to the episode at hand must be re-issued at every recurrence, so the cost of oversight grows with session duration rather than with the number of distinct problems. Zero2Skill is a human-robot symbiotic agentic system that formulates the collection loop as a verification-gated decision process: the robot collects, verifies, and resets autonomously and pauses to notify a remote operator only when a phase fails verification repeatedly, a boundary set by an explicit retry budget. The operator describes the problem in natural language; an LLM parser translates each utterance into a structured adjustment stored in a persistent Corrective Memory consulted on all subsequent rounds, so an addressed failure mode rarely needs a second correction. On a real-robot desktop-clearing testbed, Zero2Skill matches the episode collection success rate of full teleoperation with 16% of the human working time; language corrections repair both verifier criteria and execution strategies — improving verifier agreement with human labels in all four evaluated settings and raising single-attempt collection success from 12.5% to 47.5% (20.0% to 50.0% in a separate arm-selection comparison) — and policies fine-tuned on its data match those trained on teleoperation data.

Method

One instruction becomes a plan — collection and reset routines plus their verification prompts. Collection then runs as a verification-gated collect–verify–reset loop: each phase must earn an explicit yes verdict from a VLM verifier, up to a retry budget of 3. When a phase exhausts its retries, the system halts in the Alert state, asks the operator in language, and persists the answer to Corrective Memory.

Within a session the system changes its behavior through text alone; between sessions the policy changes its weights through data alone. Built on OpenClaw — VLM verifier: Seed1.8; LLM parser: DeepSeek-V3.2.

Try the Loop

Play the operator. When the verifier fails repeatedly, the system halts in the Alert state and asks you. Pick a correction — a persistent rule never alerts again; a one-off nudge does.

Autonomous Collection

50 / 50 episodes on the desktop-clearing testbed with 0 minutes of teleoperation and 4.8 minutes of human working time — the operator types a few sentences, from anywhere.

Language-Guided Correction

One channel repairs both perception and action — a success criterion that is too strict, an object to skip, a grasp that needs a condition.

Single-attempt collection success across eight object categories under cumulative language corrections (five attempts per object per setting). The depth offset improves every category (+22.5 pts); the SAM3 re-prompt (+12.5 pts) concentrates on the four categories with segmentation errors.

A separate controlled comparison isolating the arm-selection rule (+30.0 pts, improving seven of eight categories): the default assigns the closest arm, but the camera views often reveal that the farther arm has the clearer grasping approach.

Corrections accumulating during live collection. A stored correction is guaranteed to be reused, not guaranteed to help — three of four groups end above the pre-intervention level; the hardest scene recovers only partially.

Data Quality via Downstream Policy

Policies fine-tuned from π_0.5 on 50 demonstrations each, evaluated over 20 blind physical trials — no agent, no verifier, no memory, no human.

Zero2Skill data is good enough, not merely cheap — the same 80% policy success as full teleoperation on this 20-trial evaluation, at a fraction of the human cost.

Comparable illumination and higher image quality, lower RMS jerk, but a remaining smoothness (continuity) gap — the diagnostics indicate comparably usable data, not overall superiority.

BibTeX

@article{wang2026zero2skill,
  title   = {Zero2Skill: Bootstrapping Robot Skills through
             Autonomous Data Collection, Training, and Deployment},
  author  = {Wang, Boyuan and Zhang, Zhenyuan and Yang, Zhiqin and
             Gu, Peijun and Wang, Shuya and Wang, Xiaofeng and
             Ze, Xianghui and Chang, Yifan and Zhao, Guosheng and
             Shao, Jiangnan and Huang, Guan and Liu, Hengyu and
             Zhang, Yonggang and Xue, Wei and Guan, Chunyuan and
             Pu, Chenglin and Guo, Yike and Wang, Xingang and
             Zhu, Zheng},
  journal = {arXiv preprint arXiv:2607.14047},
  year    = {2026}
}