AI Data & Evaluation · For AI companies and model developers
People make and check your training data. Qualified, calibrated and reviewed for each engagement.
Speech and multimodal collection, annotation, demonstration and preference data, evaluation and red teaming. Made and checked by people, in 150+ languages, with lineage on every unit.
What we deliver
Buy the data, the judgement, or both, from one team.
Ten things a lab buys from YPAI. Five make data, five judge it. The record that trains the next checkpoint is the record that proves the last one: same people, same consent, same lineage.
-
D01 · Data made
Speech and audio collection
Read and spontaneous speech in 150+ languages: studio, field, remote and device-based, transcripts verified by native reviewers.
Speech data Lands as Speech corpus
-
D02 · Data made
Image, video and sensor collection
Collected to your specification, the rare case covered on purpose, consent recorded per contributor.
Data collection Lands as Annotated image set
-
D03 · Data made
Annotation and validation
Polygons, boxes, attributes, segmentation, transcription; second annotator on sampled items, agreement tracked.
Annotation Lands as Annotated image set
-
D04 · Data made
Demonstration and SFT data
Instruction and demonstration data written by people qualified in the domain: engineers, mathematicians, lawyers, clinicians.
AI Data & Evaluation Lands as Demonstration set
-
D05 · Data made
Dataset licensing
Sourcing and rights-cleared licensing where building from scratch is the wrong spend.
Datasets Lands as Licensed corpus
-
D06 · Data judged
Preference data
Pairs judged against the last accepted checkpoint by calibrated graders, with a written rationale on every choice.
Evaluation Lands as Preference and red-team set
-
D07 · Data judged
Model and multimodal evaluation
Human grading of what automated metrics read worst: speech, translation, generated image and video, artefact and authenticity review.
Evaluation Lands as Evaluation record
-
D08 · Data judged
Red teaming
Taxonomies, probe libraries and domain-specific attacks in the languages your users write; every break filed as a regression case.
Evaluation Lands as Preference and red-team set
-
D09 · Data judged
Agent trajectories
Multi-step task records with tool-use traces, judged at the step where the run failed, not only at the final answer.
Evaluation Lands as Trajectory record
-
D10 · Data judged
Production feedback loops
Human review of live outputs after ship, returned as graded training and evaluation data for the next run.
Evaluation Lands as Feedback record
How the data is built
Your data is built in layers, by people.
Contributors come on in waves, language by language, until the network covers what your spec asks for. Then the layers go down: capture first, annotation, demonstrations, preference on top.
No layer is synthetic. Each one is made by people who were calibrated before production.
- 210,000+identity-verified contributors
- 150+languages, native reviewers
- 50+countries
Network reach, not a headcount on your project: each engagement mobilises the slice it needs, by language, country, dialect, demographic, domain and professional credential.
Clients include
One unit, layer by layer
Every delivered unit traces back to the people who made it.
A person made it under a consent they can revoke. A qualified contributor labelled it. A calibrated reviewer checked it and a native reviewer sampled it. When it is a pair, a qualified human judged it and wrote down why.
- L4 · Preference Pairs judged against the last accepted checkpoint, rationale written
- L3 · Demonstration Instruction and response written by people qualified in the domain
- L2 · Annotation Labelled to your schema, second annotator on sampled items
- L1 · Capture Recorded by identity-verified people, consent per unit, revocable
L4 · one preference judgement
A · last checkpoint
No. Python may reorder items that compare equal, so keep an index if the original order matters.
B · candidate
Yes. Python's sort is stable: items that compare equal keep their original order, so you can sort by one key and then another.
The test
Then we prove it.
Every engagement ends the same way: the candidate's map laid over the last accepted one. Units your data improved, units that regressed and were filed as regression cases the next checkpoint has to hold. Unit by unit, judged by calibrated humans.
Example evaluation · 341 units
- Improved
- 37
- Regressed, filed
- 9
- Unchanged
- 239
- Not judged
- 56
Every regression is a case the next checkpoint has to hold. Preference data, model and multimodal evaluation, red teaming, agent trajectories, production feedback. Evaluation
Made and judged by the same people, under the same consent.
What lands in your pipeline
What you receive: four kinds of delivery, each with its acceptance record.
How each one was made, and how it was judged. The same team does both.
-
Speech corpus
Read and spontaneous clips, verified transcripts, consent per contributor
- Made
- Studio, field, remote and device-based collection; transcription verified by native reviewers.
- Judged
- Sampled by an independent native reviewer; accepted against agreed floors; rights cleared for training.
Lands from D01 · Speech and audio collection
-
Annotated image set
Polygons, boxes, attributes to your schema
- Made
- Multimodal annotation and dataset validation, the rare case covered on purpose.
- Judged
- Second annotator on sampled items; agreement tracked; artefact and authenticity review for generated media.
Lands from D02 · Image, video and sensor collection D03 · Annotation and validation
-
Demonstration set
Instruction and accepted response, domain-matched
- Made
- Written by qualified contributors: code by engineers, proofs by mathematicians, contracts by lawyers.
- Judged
- Checked by a second qualified contributor; native review in every language shipped.
Lands from D04 · Demonstration and SFT data
-
Preference and red-team set
Pairs with rationale, findings filed as regression cases
- Made
- Pairs presented against the last accepted checkpoint; probes from taxonomies and probe libraries.
- Judged
- Chosen with a written rationale by calibrated graders; every break rechecked on the next checkpoint.
Lands from D06 · Preference data D08 · Red teaming
Run like an operation
Data you can audit, delivered in batches you can accept.
Five gates before a batch is yours. The fourth is a person, in every language shipped. Rework until it passes; then it is yours.
4 delivered · 1 in production Acceptance against agreed criteria · manifests versioned · rework until it passes Example batches: nothing in this table is a real delivery or a real result.
How the data is governed
Consent, lineage and jurisdiction, per unit.
Every contributor is identity-verified. No contributor is shown on this page.
- Contributors
- Identity-verified, under consent and legal-basis handling
- Consent
- Recorded per unit, revocable
- Lineage
- On every unit, from contributor to manifest
- Jurisdiction
- Norway, under GDPR
- Processing
- EEA where the engagement requires it; residency and subprocessors per project
- Controls
- Isolation, client-specific environments, audit records, retention and deletion
Detail: the ethical framework.
What model teams ask first
Can you deliver in our schema and format?
Yes. Scope defines the schema, the acceptance criteria and the delivery manifest before production. Batches are versioned and reworked until they pass; then they are yours.
How are graders calibrated?
Contributors are tested into the task by language, domain and credential, then graded against shared rubrics before production. Agreement is tracked throughout and an independent native reviewer samples every language shipped.
Where is the data processed?
YPAI operates from Lysaker, Norway, under GDPR. EEA processing where the engagement requires it; residency, subprocessors and transfer controls are defined per project.
Do you red-team, and what do we get back?
Yes. Taxonomies, probe libraries and domain-specific attacks in the languages your users write. Every break is filed as a regression case with a recheck against the next checkpoint.
One decision
Bring the modality, the languages, the volume, and what you need to prove.
YPAI scopes the part of the network your data needs, the rubric, and the first batch.
- Scoping call
- Pilot batch
- Production