--- title: "Robotics perception and task data, made in Europe | YPAI" url: https://ypai.ai/solutions/robotics/ description: "Stereo, LiDAR and pose collection in consented gardens, fields, warehouses and homes, task demonstrations from workplaces, and a record per frame." source: "src/copy/routes (route /solutions/robotics/)" --- # Every frame knows where it stood. Every space said yes. > Stereo, LiDAR and pose collection in consented gardens, fields, warehouses and homes, task demonstrations from workplaces, and a record per frame. Stereo, LiDAR and pose collection across gardens, fields, warehouses and homes, task demonstrations from consented workplaces, and annotation with measured agreement, with consent on every record and the calibration your perception team maps to its own rig. ## A robot that sees is easy. Three things make its data usable Where · how precisely · lawfully Whether it was collected where the robot will run. Whether the frame knows where it stood. Whether the footage can lawfully exist. ### Was it collected where the robot will run? DE · aisle · artificial · clutter · people one cell of the coverage grid · counted, never assumed - A model trained on lawns in one climate meets moss and gravel in another. One trained in one warehouse meets a different floor, racking and light in the next. The programme is specified as a grid of country, ground, light, condition and presence of people, and coverage is counted per cell against that grid, with the denominator reported. ### Does the frame know where it stood? Pose fixed · stereo pair · calibration cal-214 a position and a depth a reviewer can check - Outdoors an RTK position, indoors a pose anchored to the space, and the rig's calibration in every delivery, so the depth in the image and the position on the ground can be checked against each other and mapped to your camera baseline. A float or lost pose stays in the record and never counts as ground truth. ### Can the footage lawfully exist? Consent c-3180 · space owner · Article 6 per record · third parties de-identified before annotation - A garden camera sees neighbours, a warehouse camera sees staff, a home camera sees a family. Every record carries the consent of the space's owner and the people in it, third parties are de-identified before an annotator sees the frame, and the material stays in the EEA. ## Start from the ground your robot has to cross Four engagements · one robot at a time Collection, task demonstrations, annotation and held-out ground truth, each bought on its own and configured for one robot, one rig and one coverage grid. Four things are settled before the first walk. - The coverage grid, cell by cell - The rig, its mount and its calibration - What the record per frame contains ### Collection where the robot will run - Stereo, LiDAR and pose sequences from the gardens, fields, aisles and homes your robot will actually work in. - Identity-verified contributors walk a stereo camera rig, with RTK GPS outdoors and an anchored pose indoors, through consented gardens, orchard rows, pavements, warehouses and homes in several European countries. The light, conditions and presence the grid names, hazards staged where the robot will meet them, and back to the same place when the grid asks for it again. - **Rig**: Stereo pair · LiDAR where the robot has it · RTK or anchored pose · calibration file - **Where**: confirm. Consented gardens, fields, warehouses and homes · several countries - **When**: Light, conditions and presence from the grid Every frame with its pose and its consent ### Task demonstrations - First-person video of the tasks an arm or a humanoid has to learn, recorded in consented workplaces. - Egocentric video of hand-object interaction, tool use, grasp and place, assembly and task sequences, recorded by identity-verified contributors in consented workplaces with synchronised device and sensor data, and annotated with action labels, temporal segments and keypoints under an action taxonomy written for the task. - **Format**: First-person video · synchronised sensors - **Tasks**: Grasp and place · tool use · assembly · sequences - **Labels**: flag. Action labels · temporal segments · keypoints Demonstrations with their timing, in the training set ### Perception annotation - Masks, keypoints, 3D boxes and fused sensors two annotators agree on, measured on the delivered frames. - Semantic and instance segmentation, keypoints for pick-and-place and for people and pets, 3D cuboids and per-point segmentation on LiDAR, and sensor-fusion alignment across camera, LiDAR and IMU, drawn under a written task protocol. Two annotators per frame, agreement measured as IoU per class, and a frame below the threshold goes to adjudication instead of being averaged. - **Classes**: Floor · boundary · obstacle · person · part - **Agreement**: confirm. IoU per class · two annotators - **Below threshold**: Adjudicated, both masks kept Labels with their agreement, in the delivery ### Held-out ground truth for evaluation - Sequences your model has never seen, with the pose and depth to score it against. - A held-out set built to the same grid and kept out of training: stereo pairs, point clouds, trajectories and calibration, so VSLAM, depth, segmentation and a vision-language-action policy can be scored against recorded ground truth rather than against another model, and a simulation-trained model against ground that exists. The same set reruns on every release. - **Material**: Stereo pairs · point clouds · trajectories · masks - **Kept out**: confirm. Never in training - **On release**: Rerun on every release, with regressions reported A held-out set to score every release against ## Six kinds of robot, each with its own ground Six kinds of robot · outdoors, indoors, on the line A mower, a warehouse robot and an assembly arm do not fail in the same places. The grid, the rig and the hazards are written for the ground your robot has to cross. Collection and annotation are configured per engagement. Agricultural programmes add a crop taxonomy and a seasonal protocol; task-demonstration programmes add an action taxonomy written for the task. - The edge, the hose and the dusk - Grass against gravel and beds, staged hoses and toys on the lawn, leaves and frost on the same coordinates, and dusk captures for the hours a mower runs unwatched. - Rows, canopy and a crop taxonomy - Orchard and vineyard rows, canopy occlusion and mud, with a crop-segmentation taxonomy and a seasonal protocol written per programme. - Pavements, dropped kerbs and driveways where the camera meets people and plates, collected under consent and de-identified before annotation. - Aisles, pallets and people at work - Racking, pallets left in the aisle, spills, dock doors and staff crossing the path, under artificial light and dust, with an anchored pose on every frame. - Floors, thresholds and a family's things - Cables, toys, rugs and thresholds in consented homes, pets and children staged on purpose, low light and clutter, and third parties de-identified before annotation. - Grasps a line can act on - Pick-and-place keypoints, defect masks, task demonstrations for assembly, and navigation segmentation for the robots that run indoors, under the same two-annotator agreement. ## Specify the grid. Walk a cell. Read the record Try it yourself: the coverage grid Five axes · one walk · the record per frame You are the perception lead. Pick the countries, grounds, light, conditions and presence your robot will meet, walk one cell of that grid, and read the record every frame arrives with. The grid is a toy, so the mechanics are visible. A toy grid and a toy rig, so the mechanics are visible. Nothing on this page is a real delivery or a real result. ### Collection demo · Robot programme Coverage counts a cell only when it holds the frames the grid asks for, from fixed-pose frames, RTK outdoors and an anchored pose indoors. Float and lost poses stay in the record and never count. ### The grid is the denominator. - Coverage is counted cell by cell against the grid you specified, never as a percentage of frames collected. ### A lost pose is not ground truth. - Frames with a float or lost pose stay in the record as calibration evidence and are excluded from coverage and from every ground-truth set. ### Every frame carries its consent. - The consent record, the contributor, the rig and its calibration travel with the frame, so any sample can be traced back to the space and the day. ## Two masks on one frame. The agreement is measured. Try it yourself: two annotators on one frame Two annotators · IoU per class · one threshold You are annotator one. Three frames from the images on this page, a lawn edge, a warehouse aisle and a kitchen floor, each have floor, boundary and obstacle to draw. Annotator two has drawn them blind. The overlap per class is computed here, one class below the threshold sends the frame to adjudication, and a model can be run on the frame in your browser to see what a first pass looks like next to a person's mask. Three frames from this page and toy masks, so the mechanics are visible. The model runs in your browser. Nothing on this page is a real delivery or a real result. ### Task protocol RP-2 SlimSAM, an open segmentation model, runs in this tab on this frame, at the marked point. Loading the model into this tab {label} · model score {iou} · box IoU against annotator two {biou} · {ms} ms in this tab The model's first pass is shown on the same frame as the annotators' masks. Neither is a delivered result. ### The second annotator is blind. - Annotator two never sees annotator one's masks, so the agreement measures two judgements, not one copied. ### One class fails the frame. - Floor, boundary and obstacle share edges. A wrong boundary moves the others, so a single class below threshold sends the whole frame to adjudication. ### The threshold is written down. - The protocol names the IoU floor for the task, and every verdict is logged against it with both masks kept. ## The record your perception team and your review will read One row per frame · the fields a reviewer checks Every frame with its cell, its pose fix, the hazard in view, the consent it belongs to, the contributor, the rig and the calibration. Written by the walk you ran above. Example record, built from what you ran above Read a row left to right: the walk and frame id, the cell it was collected in, then the pose fix, the hazard in view and the consent, contributor, rig and calibration it travels with. Counted says whether the frame counts toward coverage or stays in the record only. No frames yet. Walk a cell in the demo above. ## One country, one ground, one rig configuration One country · one ground · one rig A pilot is one country, one ground and one rig configuration from the programme above. The grid, the rig and the record's fields are fixed before the first walk, that country and ground are collected and labelled, and the pilot ends with a delivery you can map to your own hardware and a decision. You judge it against your model. - The IoU floor below which a frame is adjudicated - What the record per frame must contain - Countries, grounds, light, conditions and presence, written as cells with a frame count each. - Stereo baseline, LiDAR where the robot has it, mount and calibration matched to your rig, with the file in the delivery. - Consented gardens, aisles and homes walked cell by cell, hazards staged, third parties de-identified before annotation. - Floor, boundary and obstacle masks with IoU per class, adjudication kept in the record. - Continue to the full grid, change the grid, or stop, with the record as the evidence. The pilot ends with a delivery you can map to your rig. ## The controls a review of robot footage will ask for GDPR Article 6 · de-identification · EEA A Norwegian company under GDPR. Every record carries the consent of the space's owner and the people in it, third parties are de-identified before annotation, and the material is stored and processed in the EEA by default. - **Jurisdiction**: Norway · GDPR-native - **Lawful basis**: Consent per record · Article 6 - **Third parties**: Faces and plates de-identified before annotation - **Contributors**: Identity-verified · EEA-only sourcing on request - **Storage**: EEA by default - **Calibration**: Rig intrinsics and extrinsics in the delivery - **Contracts**: Standard DPA terms · SCCs available - **Erasure**: 30-day end-of-contract SLA - [Robotics and industrial vision annotation](https://ypai.ai/annotation/robotics-industrial-vision-annotation-services/) - LiDAR and 3D point cloud annotation - [Sensor fusion annotation](https://ypai.ai/annotation/sensor-fusion-annotation-services/) - [Data collection](https://ypai.ai/data-collection/) ## Tell us the robot and the ground A feasibility read · a bounded first grid We scope against the ground your robot has to cross, agree the grid, the rig and the consent model, and prove the method on one country and one ground. Scope · Grid · Rig · First walks · Delivery A bounded first grid carries its own acceptance criteria and its own record. You judge it against your model. Briefs are treated as confidential. We are used to programmes that cannot be named and footage that cannot leave the EEA. Tell us the robot, where it runs, and your rig and sensors. We reply with a feasibility read. We come back with a feasibility read on the robot, the grid and the rig. The brief is treated as confidential.