Train robots on episodes, not isolated frames.
YPAI turns human and robot demonstrations into synchronized, training-ready episodes with failures, interventions and recoveries intact.
A box around an object can train perception. It cannot teach the task.
Physical AI systems need more than isolated labels. They need to know:
A labelled frame
- object class
- object location
A complete episode
- what the system observed
- what instruction or objective applied
- which action followed
- what changed in the environment
- whether contact occurred
- when each task stage began and ended
- whether the attempt succeeded
- why it failed
- whether a human intervened
- how the system recovered
YPAI structures these relationships as complete episodes rather than disconnected frames.
The same holds for industrial vision: a visual anomaly becomes useful training data only when it is connected to the product, station, process stage, tolerance, viewpoint, inspection method and disposition.
The project is built around the decision the system must make.
That decision may be:
- which object to manipulate
- how to grasp or move it
- whether an assembly is correct
- whether a surface contains a defect
- whether the system should continue, stop, retry or escalate
The ontology, capture protocol, annotation and acceptance plan follow that decision.
Start with the data bottleneck.
YPAI can run one defined stage or the complete data operation.
-
Entry point: Collect new demonstrations
Start here when: You need new human or robot interaction data.
Typical output Structured demonstrations with synchronized observations, actions, states, outcomes and rights records.
-
Entry point: Structure existing robot logs
Start here when: You have data but cannot use it consistently across training, evaluation or review.
Typical output Versioned episodes with normalized schemas, validated stream alignment and reproducible exports.
-
Entry point: Annotate and evaluate
Start here when: You need structured task meaning, outcomes or industrial-vision ground truth.
Typical output Annotated episodes or visual datasets with ontology, reviewer records, QA results and acceptance status.
-
Entry point: Build the data engine
Start here when: You need operational infrastructure around the dataset.
Typical output A controlled data and evaluation workflow connected to your model-development environment.
OUTDOOR AUTONOMY
The edge is part of the task.
A route learned on clean grass is not enough. Wet ground, stones, cable and the change from lawn to border all alter the next move.
Approach → boundary changes → route recovered
The useful record keeps the context that changes the route, not only the seconds when everything works.
Every episode should explain what happened.
The exact schema is defined for the customer's embodiment, policy stack and training objective.
A robot-learning episode can include the following layers.
Representative episode schema
- Task identity
-
- episode ID
- task family
- task variant
- environment
- scene
- robot or embodiment
- operator or collection method
- protocol version
- start and end conditions
- Instruction and objective
-
- natural-language instruction
- structured task goal
- target object
- destination
- constraints
- prohibited actions
- success definition
- failure definition
- Observations
-
- fixed-camera video
- egocentric video
- wrist-camera video
- multi-camera streams
- RGB
- depth
- point cloud
- audio
- tactile or force data where available
- environmental and device metadata
- Robot and environment state
-
- joint state
- end-effector pose
- gripper state
- base pose
- object state
- tool state
- scene state
- sensor state
- safety state
- controller state
- Actions
-
- translation
- rotation
- gripper commands
- joint commands
- navigation actions
- tool use
- control mode
- intervention
- stop or retry action
- Task structure
-
- task phase
- subtask
- action segment
- prerequisite
- transition
- contact event
- state change
- completion event
- Outcome
-
- successful
- partially successful
- failed
- interrupted
- unsafe
- invalid
- human intervention required
- retry initiated
- recovered
- abandoned
- Failure and recovery
-
- failure point
- failure category
- preceding state
- attempted action
- environmental cause
- perception error
- planning error
- control error
- operator intervention
- recovery action
- final disposition
- Provenance and quality
-
- source
- rights status
- collection protocol
- calibration record
- sensor synchronization status
- annotation version
- reviewer status
- issue history
- acceptance status
- dataset version
The episode schema is frozen before production scale.
The delivery is built around the robot task, capture setup and evaluation, not a generic image-labelling export.
Capture the task
where it actually
happens.
The capture architecture depends on the model, embodiment, task and deployment environment.
Label the interaction,
not only the object.
Physical AI annotation must preserve the relationship between time, state, action and outcome.
- Approach
- Contact
- Grasp
- Transfer
- Release
The taxonomy is built against the customer's model objective and operating environment.
Industrial vision data should represent the decision on the line.
A defect label without process context may not be enough to train or evaluate the system that must act on it.
The final dataset records not only the label, but also the conditions under which the inspection decision was made.
Real data grounds the system.
Synthetic data expands the coverage.
Real episodes, simulation output and synthetic scenarios pass through one documented quality and acceptance process.
Real baseline Synthetic coverage - Baseline
- Coverage gaps
- Generate
- Validate plausibility
- Evaluate against real
- Close the loop
Synthetic data is a coverage tool, not a substitute for real-world acceptance.
Quality follows the complete episode.
The quality process begins before collection or annotation and ends only when the dataset version is accepted.
Where accuracy or pass rate is the appropriate metric, YPAI typically proposes a 95% to 99% target range after calibration, with the exact metric, threshold and review method defined in the SOW.
Ships with the accepted version / episode manifest · sensor & stream schema · calibration package · ontology & annotation guide · quality report · provenance & rights record · dataset version record · checksums
Delivered in or mapped to RLDS, LeRobotDataset, MCAP, Parquet, JSONL, COCO, YOLO, KITTI or a customer-native schema. Format support is confirmed against the actual data, loader, toolchain and delivery requirement.
Different systems need different data operations. Robot foundation model · manipulation and humanoid · industrial inspection · sensor and device · existing-data recovery.
Send the robot, task and data requirement.
The system, the task, the data source, the sensors, the scale, the annotation and the delivery format. YPAI returns feasibility, method, schema, QA and acceptance, delivery format and the open decisions before launch.
Related
- Physical AI dataThe whole operation: recovery, structuring, gap-fill, evaluation and delivery.
- Data collectionCustom video, image, sensor and multimodal collection.
- Video annotationTemporal events, tracking and action labels.
- LiDAR and 3DPoint-cloud annotation and spatial data.
- Sensor fusionSynchronized multimodal annotation.
- Image annotationDetection, segmentation and keypoints.
- Managed annotationOntology, production, review and acceptance.
- PilotsValidate the task, output and acceptance plan before scale.