--- title: "Robot Demonstration and Episode Data Collection | YPAI" url: https://ypai.ai/physical-ai-data-collection/ description: "Capture robot and human demonstrations as training-ready episodes for robot learning: egocentric video, teleoperation, synchronised sensors and episode QA." source: "src/copy/routes (route /physical-ai-data-collection/)" --- # Train robots on episodes, not isolated frames. > Capture robot and human demonstrations as training-ready episodes for robot learning: egocentric video, teleoperation, synchronised sensors and episode QA. YPAI turns human and robot demonstrations into synchronised, training-ready episodes with failures, interventions and recoveries intact. ## A box around an object can train perception. It cannot teach the task. Physical AI systems need more than isolated labels. They need to know: - when each task stage began and ended YPAI structures these relationships as complete episodes rather than disconnected frames. The same holds for industrial vision: a visual anomaly becomes useful training data only when it is connected to the product, station, process stage, tolerance, viewpoint, inspection method and disposition. The project is built around the decision the system must make. - how to grasp or move it - whether a surface contains a defect - whether the system should continue, stop, retry or escalate ## Start with the data bottleneck. YPAI runs one defined stage or the complete data operation. ### Collect new demonstrations - You need new human or robot interaction data. - Structured demonstrations with synchronised observations, actions, states, outcomes and rights records. ### Structure existing robot logs - You have data but cannot use it consistently across training, evaluation or review. - Versioned episodes with normalised schemas, validated stream alignment and reproducible exports. ### Annotate and evaluate - You need structured task meaning, outcomes or industrial-vision ground truth. - Annotated episodes or visual datasets with ontology, reviewer records, QA results and acceptance status. ### Run the ongoing data operation - You need operational infrastructure around the dataset. - A controlled data and evaluation workflow connected to your model-development environment. ## The edge is part of the task. For a robotic mower, a route learned on clean grass is not enough. Wet ground, stones, cable and the change from lawn to border all alter the next move. The useful record keeps the context that changes the route, not only the seconds when everything works. ## Every episode explains what happened. The exact schema is defined for the customer's embodiment, policy stack and training objective. A robot-learning episode can include the following layers. ### Task identity ### Instruction and objective ### Observations - tactile or force data where available ### Robot and environment state ### Actions ### Task structure ### Outcome ### Failure and recovery ### Provenance and quality The episode schema is frozen before production scale. The delivery is built around the robot task, the capture setup and the ## Capture the task where it actually happens. The capture architecture depends on the model, embodiment, task and deployment environment. ## Label the interaction, not only the object. Physical AI annotation must preserve the relationship between time, state, action and outcome. - Object state / empty to filled The taxonomy is built against the customer's model objective and operating environment. ## Industrial vision data should represent the decision on the line. A defect label without process context may not be enough to train or evaluate the system that must act on it. The final dataset records the label and the conditions under which the inspection decision was made. ## Real data grounds the system. Synthetic data expands the coverage. Real episodes, simulation output and synthetic scenarios pass through one documented quality and acceptance process. Synthetic data is a coverage tool, not a substitute for real-world acceptance. ## Quality follows the complete episode. The quality process begins before collection or annotation and ends only when the dataset version is accepted. Where accuracy or pass rate is the appropriate metric, YPAI typically proposes a 95% to 99% target range after calibration, with the exact metric, threshold and review method defined in the SOW. episode manifest · sensor & stream schema · calibration package · ontology & annotation guide · quality report · provenance & rights record · dataset version record · checksums Delivered in or mapped to RLDS, LeRobotDataset, MCAP, Parquet, JSONL, COCO, YOLO, KITTI or a customer-native schema. Format support is checked against the actual data, loader, toolchain and delivery requirement. Different systems need different data operations. Robot foundation model · manipulation and humanoid · industrial inspection · sensor and device · existing-data recovery. ## Send the robot, task and data requirement. The system, the task, the data source, the sensors, the scale, the annotation and the delivery format. YPAI returns feasibility, method, schema, QA and acceptance, delivery format and the open decisions before launch. Scope a Physical AI data project ## Related ### Physical AI data - Choose the data path: recover logs, collect, structure, annotate, fill a gap or evaluate the task. ### Data collection - Custom video, image, sensor and multimodal collection. ### Video annotation - Temporal events, tracking and action labels. ### LiDAR and 3D ### Sensor fusion ### Image annotation ### Managed annotation ### Pilots - Validate the task, output and acceptance plan before scale.