Schema design
Co-designed with your team. Versioned. Tied to the eval metric you will run against.
Deliverable: Versioned schema spec
TEXT DATA / SEVEN SERVICES
Seven text-data services. 150+ languages. Linguist-staffed.
From flat NER through preference data and red-team evaluation, the full text-data stack for LLM and NLP teams. Native-speaker linguists, EEA processing by default, kappa reports per delivery.
- Annotation through RLHF
- 150+ languages
- EEA processing
NATIVE SPEAKERS
Bulgarian Czech 한국어 Português Suomi العربية Magyar Bahasa Català Norsk Polski తెలుగు Türkçe Eesti Kiswahili हिन्दी Galego Svenska Tiếng Việt Føroyskt Latviešu Cymraeg Ελληνικά Yorùbá Македонски Euskara ไทย Български Беларуская Lëtzebuergesch isiXhosa 日本語
PROCUREMENT READINESS
Compliance posture for text training and evaluation data.
Article 10 enforcement begins 2 December 2027 for standalone Annex III systems, 2 August 2028 for AI embedded in regulated products. YPAI ships every text engagement with the artefacts a regulated buyer needs in their file.
Compliance posture
Annotation guideline
Gold set
Per-schema kappa report
Error analysis
Records of processing (Article 30)
Signed DPA + sub-processor list
EU AI Act Article 10
Data and data governance. Annotation guideline, gold set, kappa report, and error analysis included in every delivery.
GDPR Articles 7, 28, 30
Per-contributor consent records (Article 7). Processor agreement (Article 28). Records of processing (Article 30). 30-day erasure SLA.
EEA processing by default
Norwegian company structure, EEA processing by default, EEA-only sourcing on request. Norwegian jurisdiction; transfer controls defined per project.
Request the delivery documentation →
We map the delivery documentation to your modality mix, regulatory class, and deployment environment.
WHAT WE LABEL
Seven services across the text-data lifecycle.
The services an LLM or NLP team procures, in one provider relationship.
| Service | Sub-tasks | Schema / format | Eval metric |
|---|---|---|---|
| Semantic (NER + linking + relations) | Flat NERnested NERentity linkingrelation extractioncoreference | CoNLL-U, BIO/IOB, BRAT, spaCy JSON, BioC | Span F1, Cohen's kappa |
| Pragmatic (sentiment / intent / emotion / stance) | PolarityABSAPlutchik or Ekman or VADintentstanceclaim-evidence | JSON-L with per-annotator IDs | Krippendorff alpha, macro F1, micro F1 |
| Classification (topic, multi-label, hierarchical, toxicity) | Flatmulti-labelhierarchical (ICD-10)topicroutingtoxicityPII | JSON-L, taxonomies in SKOS | Hierarchical F1, calibration ECE, FPR-at-low-FNR |
| Linguistic (POS / dependency / coref / morphology / WSD) | UD POS and dependencyconstituencycoreferencemorphologylemmatisationWSD | CoNLL-U (UD), Penn Treebank | LAS, UAS, CoNLL-F1 |
| Generation (summarisation) | Extractiveabstractivequery-focusedlong-documentmeeting with attribution | JSON-L with source-span links | BERTScore, FActScore, SummaC, human rubric |
| Preference + safety (RLHF) | Pairwise or n-way preferencecritique writingred-teamfactualityDPO/IPO/KTO | HH-RLHF JSON-L, custom with metadata | Preference agreement, win-rate, MT-Bench-style |
| Document understanding | OCRreading orderlayoutstructured extractiontable-QAsignatures | PAGE-XML, ALTO, hOCR, FUNSD | CER, WER, field F1, ANLS |
ANNOTATION TAXONOMY
Where text annotation fits in the LLM stack.
Pre-training corpora, SFT alignment data, RLHF preference pairs, eval and red-team. Every layer has a labelling problem.
TEXT DATA
Pre-training
filtering quality classifiers PII strip
SFT and instruction
prompt-response pairs tool-call traces agentic trajectories chain-of-thought
Preference
pairwise n-way critique DPO / IPO / KTO
Safety
red-team refusal factuality dual-use CBRN and cyber
Domain annotation
NER classification schema labelling
Evaluation
summarisation faithfulness hallucination labels per-language regression cross-lingual transfer
HOW WE LABEL
Every project clears the same seven gates.
Calibration before production. Documented kappa thresholds. Human QA applied according to the agreed acceptance and sampling plan. The artefacts you need for your Article 10 file.
Annotation guidelines
Edge cases enumerated, examples per class, glossary of domain terms. Updated as adjudication surfaces new cases.
Deliverable: Annotation guideline document
Calibration round
Pilot batch on a shared subset. Disagreements adjudicated before scale. Guidelines refined on findings.
Deliverable: Calibration kappa report
IAA gate
Production starts only when calibration clears Landis-Koch substantial agreement (kappa 0.6+) on the schema.
Deliverable: Signed record that the gate was cleared
Production labelling
Trained linguists, native speakers for multilingual. Annotator selection criteria documented per project.
Deliverable: Labelled batches
Human QA + adjudication
Review coverage follows the agreed acceptance and sampling plan. Gold-set items are injected at a documented rate. Disagreements above threshold escalate to a lead annotator with an adjudication trail.
Deliverable: QA review log + adjudication record
Delivery with kappa report
Per-schema metrics, error analysis, audit trail, and Article 30 records ship with the dataset.
Deliverable: Final delivery pack
Seven gates. One trail of evidence. Every delivery.
DOMAIN DEPTH
Three regulated verticals where generic crowdsource fails.
Medical ontologies, legal benchmarks, financial regulation. Domain knowledge plus YPAI infrastructure. Annotation only; not medical, legal, or investment advice.
Medical NLP
Clinical-note de-identification, ICD-10 / ICD-11 coding, SNOMED-CT and RxNorm linking, MedDRA pharmacovigilance, FHIR R4 export.
ICD-10SNOMEDFHIR R4
Legal NLP
Contract clause annotation (CUAD), LegalBench-style task suites, jurisdiction tagging, eDiscovery TAR, DORA and NIS2 rule extraction.
CUADLegalBenchDORA
Financial NLP
FOMC and ECB sentiment, earnings-call transcripts, ESG taxonomy and greenwashing, KYC and AML, MiFID II suitability, market abuse.
ESGKYCMiFID II
150+ LANGUAGES / NATIVE SPEAKERS
Where the LLM data gap lives.
LLM training data is roughly 45% English. EU AI Act Article 55 expects per-language disclosure across the 24 EU official languages. YPAI staffs native speakers per language.
EU official (24) 24 languages
Bulgarian Croatian Czech Danish Dutch English Estonian Finnish French German Greek Hungarian Irish Italian Latvian Lithuanian Maltese Polish Portuguese Romanian Slovak Slovenian Spanish Swedish
EU minority and Nordic 9 languages
Sami Faroese Welsh Basque Catalan Galician Frisian Luxembourgish Icelandic
Indic and Southeast Asian 15 languages
Bengali Hindi Marathi Tamil Telugu Gujarati Kannada Malayalam Punjabi Thai Vietnamese Tagalog Indonesian Burmese Khmer
African and low-resource 10 languages
Swahili Yoruba Amharic Hausa Wolof Zulu Xhosa Igbo Tigrinya Oromo
58 languages shown / 150+ supported. FLORES-200 baseline; 200 languages total. Native-speaker availability is set per project at the brief.
WHAT WE REPORT
Quality terms for every text engagement.
Accuracy SLAs are written against your gold set.
- HUMAN QA PER AGREED ACCEPTANCE AND SAMPLING PLAN
- 0.6+ KAPPA THRESHOLD
- 30 days ERASURE SLA
- 210,000+ VETTED CONTRIBUTORS
INDUSTRIES WE SERVE
For teams whose data goes through review.
Healthcare, mobility, finance, government and industrial teams get traceable data, documented QA and delivery records their reviewers can read.
Public Sector & Critical Infrastructure
Controlled data workflows for public-sector and critical-infrastructure AI deployments.
Sovereignty Public sector Controls
Healthcare & Life Sciences
Clinical NLP, medical imaging, and consent-aware healthcare datasets.
Consent Clinical data GDPR
Financial Services
Document AI, model governance support, and risk-focused data workflows.
Risk review Documents Governance
Automotive & Mobility
ADAS, in-cabin AI, mobility datasets, and safety-sensitive annotation.
ADAS Voice Perception
Manufacturing & Industrial
Quality vision, industrial data capture, and reporting automation support.
Quality vision Sensor data Operations
Enterprise & AI
LLM training data, RAG evaluation, RLHF, and benchmark datasets.
RLHF Evaluation Benchmarks
Read the industry solutions →
Or describe the operating environment and YPAI will scope the data path.
WHAT YOU RECEIVE
Every delivery includes the records your Article 10 file needs.
The records a regulated buyer expects, included with every text engagement.
01
Annotation guideline and edge cases.
Versioned guideline with edge cases enumerated, examples per class, and glossary of domain terms. Updated as adjudication surfaces new patterns; every version preserved for audit.
02
Gold set with injection rate.
Held-out gold set injected during production at a rate documented for you and not disclosed to annotators; per-annotator accuracy gates. Calibration before queue entry, blind re-runs on failure.
03
Per-schema kappa or alpha report.
Cohen's kappa, Fleiss' kappa, or Krippendorff's alpha computed per schema and per language pair on every delivery. Span tasks ship with per-class F1 as primary metric.
04
Error analysis with top failure modes.
Per-class confusion matrix, top failure modes by frequency, and recommended schema or guideline updates. The signal you need to decide whether to retrain or re-label.
05
Records of processing, DPA, and sub-processor list.
Article 30 records of processing, signed Article 28 DPA, lawful-basis documentation, 30-day erasure SLA, and full sub-processor list with Article 28(2) change notifications. All included with every engagement.
START A PROJECT
Brief us. We reply inside one EU business day.
Short brief now, deeper scoping in the reply.
Looking for work on AI projects? This form is for companies. Apply as a freelancer
QUESTIONS
Frequently asked questions
Annotation, SFT data, RLHF preference pairs, red-team, summarisation, document understanding, and evaluation. Seven services staffed by trained linguists; the service table above lists sub-tasks, schema, and eval metrics.
The contributor pool covers the 24 EU official languages, EU minority and Nordic languages, Indic and Southeast Asian languages, and a growing set of African low-resource languages. Native-speaker availability is set per project at the brief. Total of 150+ supported.
Annotation guideline (versioned), gold set, per-schema Cohen's kappa or Krippendorff alpha report, error analysis, Article 30 records of processing, signed DPA, and sub-processor list. Per-class F1 for span tasks. No accuracy SLAs without your gold set.
Norwegian company, EEA processing by default, EEA-only sourcing on request. 30-day GDPR erasure SLA. Norwegian jurisdiction; transfer controls defined per project.
Screened opt-in. Exposure limits per shift. Mandatory rotation off sensitive content. Counselling access. The specifics, and the duty of care they carry, are documented in the project brief.
GDPR jurisdiction EU AI Act Article 10 EEA operations Consent evidence
Brief us on your text-data project.
Reply inside one EU business day. NDA on request. DPA included.
Annotation through RLHF 150+ languages
Looking for work on AI projects? This form is for companies. Apply as a freelancer
Or connect on LinkedIn →
Your information is never shared. We respond with the next scoping step.
FAQ
Frequently asked questions
What text annotation tasks does YPAI deliver?
YPAI annotates text for named-entity recognition (NER), relation extraction, intent and sentiment classification, semantic-role labelling, coreference resolution, document classification with hierarchical taxonomies, span-level highlighting for legal and clinical extraction, RLHF preference data, and tool-use traces for agentic AI fine-tuning. Each engagement is scoped to the customer model architecture.
Which languages does YPAI cover for text annotation?
YPAI annotates text across 150+ languages and dialects with native-speaker reviewers in tier-1 European, Nordic, and major Asian languages. Lower-resource and morphologically complex languages (Finnish, Hungarian, Arabic dialects, Tagalog) are quoted per-project because the limiting factor is qualified-reviewer recruitment. Multilingual projects share a single delivery lead so spec consistency holds across locales.
Does YPAI annotate clinical or legal text?
Yes. Clinical reviewers operate against ICD-10 and SNOMED CT vocabularies for medical NER; legal reviewers cover contract clause extraction, EU regulation cross-reference, and litigation discovery markup. Domain expertise is contracted for each engagement. Data is processed inside EU residency under GDPR Article 9 controls where special-category personal data is involved.
How does YPAI handle taxonomy ambiguity in NER?
Ambiguous spans are resolved through a calibration cycle (spec author plus 2-3 senior reviewers) that produces edge-case examples appended to the spec. Bulk annotation then runs against the calibrated spec with double-pass review on dispute-prone classes. The calibration record ships with the delivery so model owners understand why borderline cases were resolved a particular way.
Can YPAI build RLHF preference data for LLM fine-tuning?
Yes. RLHF and DPO preference data, multi-turn instruction-tuning corpora, and tool-use traces for agentic fine-tuning are delivered as bespoke engagements. Reviewers are screened for the specific domain (legal, medical, code, customer-support) so preferences reflect domain expertise. Annotator wage transparency is part of the engagement under the ethical framework.
What output formats does YPAI deliver?
Deliveries are shipped in CoNLL, BIO/IOB, JSON-Lines, brat-standoff, HuggingFace-Datasets-compatible Parquet, and customer-specified schemas. Token offsets, character offsets, and source-document identifiers are preserved so the delivery is auditable against the original corpus.
How is YPAI text annotation priced?
Engagements are quoted per-project after a feasibility scope. The pricing inputs include taxonomy depth, IAA target, locale, domain complexity, and any compliance overhead. A scoping conversation at /contact-us/ produces a written proposal.