---
title: "Text Annotation, RLHF and Evaluation Data | YPAI"
url: https://ypai.ai/annotation/text-annotation-services/
description: "Text data for LLM and NLP teams: seven services across 150+ languages, native-speaker linguists, EEA processing and per-delivery kappa reports."
source: "src/copy/routes (route /annotation/text-annotation-services/)"
---

# Seven text-data services. 150+ languages. Linguist-staffed.

> Text data for LLM and NLP teams: seven services across 150+ languages, native-speaker linguists, EEA processing and per-delivery kappa reports.

From flat NER through preference data and red-team evaluation, the full text-data stack for LLM and NLP teams. Native-speaker linguists, EEA processing by default, kappa reports per delivery.

### Bulgarian

### Czech

### 한국어

### Português

### Suomi

### العربية

### Magyar

### Bahasa

### Català

### Norsk

### Polski

### తెలుగు

### Türkçe

### Eesti

### Kiswahili

### हिन्दी

### Galego

### Svenska

### Tiếng Việt

### Føroyskt

### Latviešu

### Cymraeg

### Ελληνικά

### Yorùbá

### Македонски

### Euskara

### ไทย

### Български

### Беларуская

### Lëtzebuergesch

### isiXhosa

### 日本語

## Compliance posture for text training and evaluation data.

Article 10 enforcement begins 2 December 2027 for standalone Annex III systems, 2 August 2028 for AI embedded in regulated products. YPAI ships every text engagement with the artefacts a regulated buyer needs in their file.

### EU AI Act Article 10

- Data and data governance. Annotation guideline, gold set, kappa report, and error analysis included in every delivery.

### GDPR Articles 7, 28, 30

- Per-contributor consent records (Article 7). Processor agreement (Article 28). Records of processing (Article 30). 30-day erasure SLA.

### EEA processing by default

- Norwegian company structure, EEA processing by default, EEA-only sourcing on request. Norwegian jurisdiction; transfer controls defined per project.

## Seven services across the text-data lifecycle.

The services an LLM or NLP team procures, in one provider relationship.

- Semantic (NER + linking + relations)
- Flat NER, nested NER, entity linking, relation extraction, coreference
- Pragmatic (sentiment / intent / emotion / stance)
- Polarity, ABSA, Plutchik or Ekman or VAD, intent, stance, claim-evidence
- Krippendorff alpha, macro F1, micro F1
- Flat, multi-label, hierarchical (ICD-10), topic, routing, toxicity, PII
- Linguistic (POS / dependency / coref / morphology / WSD)
- UD POS and dependency, constituency, coreference, morphology, lemmatisation, WSD
- Extractive, abstractive, query-focused, long-document, meeting with attribution
- Pairwise or n-way preference, critique writing, red-team, factuality, DPO/IPO/KTO
- OCR, reading order, layout, structured extraction, table-QA, signatures

## Where text annotation fits in the LLM stack.

Pre-training corpora, SFT alignment data, RLHF preference pairs, eval and red-team. Every layer has a labelling problem.

- prompt-response pairs, tool-call traces, agentic trajectories, chain-of-thought
- pairwise, n-way, critique, DPO / IPO / KTO
- red-team, refusal, factuality, dual-use CBRN and cyber
- summarisation faithfulness, hallucination labels, per-language regression, cross-lingual transfer

## Every project clears the same seven gates.

Calibration before production. Documented kappa thresholds. Human QA applied according to the agreed acceptance and sampling plan. The artefacts you need for your Article 10 file.

Seven gates. One trail of evidence. Every delivery.

### Schema design

- Co-designed with your team. Versioned. Tied to the eval metric you will run against.

### Annotation guidelines

- Edge cases enumerated, examples per class, glossary of domain terms. Updated as adjudication surfaces new cases.

### Calibration round

- Pilot batch on a shared subset. Disagreements adjudicated before scale. Guidelines refined on findings.

### IAA gate

- Production starts only when calibration clears Landis-Koch substantial agreement (kappa 0.6+) on the schema.
- Signed record that the gate was cleared

### Production labelling

- Trained linguists, native speakers for multilingual. Annotator selection criteria documented per project.

### Human QA + adjudication

- Review coverage follows the agreed acceptance and sampling plan. Gold-set items are injected at a documented rate. Disagreements above threshold escalate to a lead annotator with an adjudication trail.
- QA review log + adjudication record

### Delivery with kappa report

- Per-schema metrics, error analysis, audit trail, and Article 30 records ship with the dataset.

## Three regulated verticals where generic crowdsource fails.

Medical ontologies, legal benchmarks, financial regulation. Domain knowledge plus YPAI infrastructure. Annotation only; not medical, legal, or investment advice.

### Medical NLP

- Clinical-note de-identification, ICD-10 / ICD-11 coding, SNOMED-CT and RxNorm linking, MedDRA pharmacovigilance, FHIR R4 export.

### Legal NLP

- Contract clause annotation (CUAD), LegalBench-style task suites, jurisdiction tagging, eDiscovery TAR, DORA and NIS2 rule extraction.

### Financial NLP

- FOMC and ECB sentiment, earnings-call transcripts, ESG taxonomy and greenwashing, KYC and AML, MiFID II suitability, market abuse.

## Where the LLM data gap lives.

LLM training data is roughly 45% English. EU AI Act Article 55 expects per-language disclosure across the 24 EU official languages. YPAI staffs native speakers per language.

FLORES-200 baseline; 200 languages total. Native-speaker availability is set per project at the brief.

- Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Irish, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish
- Sami, Faroese, Welsh, Basque, Catalan, Galician, Frisian, Luxembourgish, Icelandic
- Bengali, Hindi, Marathi, Tamil, Telugu, Gujarati, Kannada, Malayalam, Punjabi, Thai, Vietnamese, Tagalog, Indonesian, Burmese, Khmer
- Swahili, Yoruba, Amharic, Hausa, Wolof, Zulu, Xhosa, Igbo, Tigrinya, Oromo

## Quality terms for every text engagement.

Accuracy SLAs are written against your gold set.

For teams whose data goes through review.

Healthcare, mobility, finance, government and industrial teams get traceable data, documented QA and delivery records their reviewers can read.

- PER AGREED ACCEPTANCE AND SAMPLING PLAN

## Every delivery includes the records your Article 10 file needs.

The records a regulated buyer expects, included with every text engagement.

### Annotation guideline and edge cases.

- Versioned guideline with edge cases enumerated, examples per class, and glossary of domain terms. Updated as adjudication surfaces new patterns; every version preserved for audit.

### Gold set with injection rate.

- Held-out gold set injected during production at a rate documented for you and not disclosed to annotators; per-annotator accuracy gates. Calibration before queue entry, blind re-runs on failure.

### Per-schema kappa or alpha report.

- Cohen's kappa, Fleiss' kappa, or Krippendorff's alpha computed per schema and per language pair on every delivery. Span tasks ship with per-class F1 as primary metric.

### Error analysis with top failure modes.

- Per-class confusion matrix, top failure modes by frequency, and recommended schema or guideline updates. The signal you need to decide whether to retrain or re-label.

### Records of processing, DPA, and sub-processor list.

- Article 30 records of processing, signed Article 28 DPA, lawful-basis documentation, 30-day erasure SLA, and full sub-processor list with Article 28(2) change notifications. All included with every engagement.

## Brief us. We reply inside one EU business day.

Short brief now, deeper scoping in the reply.

## Frequently asked questions

### What text capabilities do you cover?

Annotation, SFT data, RLHF preference pairs, red-team, summarisation, document understanding, and evaluation. Seven services staffed by trained linguists; the service table above lists sub-tasks, schema, and eval metrics.

### How many languages, and what level of native-speaker depth?

The contributor pool covers the 24 EU official languages, EU minority and Nordic languages, Indic and Southeast Asian languages, and a growing set of African low-resource languages. Native-speaker availability is set per project at the brief. Total of 150+ supported.

### What quality evidence do I receive with each delivery?

Annotation guideline (versioned), gold set, per-schema Cohen's kappa or Krippendorff alpha report, error analysis, Article 30 records of processing, signed DPA, and sub-processor list. Per-class F1 for span tasks. No accuracy SLAs without your gold set.

### Where is data processed?

Norwegian company, EEA processing by default, EEA-only sourcing on request. 30-day GDPR erasure SLA. Norwegian jurisdiction; transfer controls defined per project.

### How do you handle annotator wellbeing on red-team and toxicity work?

Screened opt-in. Exposure limits per shift. Mandatory rotation off sensitive content. Counselling access. The specifics, and the duty of care they carry, are documented in the project brief.

## What text annotation tasks does YPAI deliver?

YPAI annotates text for <a href="/annotation/named-entity-recognition-ner-annotation-services/">named-entity recognition (NER)</a>, relation extraction, intent and sentiment classification, semantic-role labelling, coreference resolution, document classification with hierarchical taxonomies, span-level highlighting for legal and clinical extraction, RLHF preference data, and tool-use traces for agentic AI fine-tuning. Each engagement is scoped to the customer model architecture.

## Which languages does YPAI cover for text annotation?

YPAI annotates text across 150+ languages and dialects with native-speaker reviewers in tier-1 European, Nordic, and major Asian languages. Lower-resource and morphologically complex languages (Finnish, Hungarian, Arabic dialects, Tagalog) are quoted per-project because the limiting factor is qualified-reviewer recruitment. Multilingual projects share a single delivery lead so spec consistency holds across locales.

## Does YPAI annotate clinical or legal text?

Yes. Clinical reviewers operate against ICD-10 and SNOMED CT vocabularies for medical NER; legal reviewers cover contract clause extraction, EU regulation cross-reference, and litigation discovery markup. Domain expertise is contracted for each engagement. Data is processed inside EU residency under GDPR Article 9 controls where special-category personal data is involved.

## How does YPAI handle taxonomy ambiguity in NER?

Ambiguous spans are resolved through a calibration cycle (spec author plus 2-3 senior reviewers) that produces edge-case examples appended to the spec. Bulk annotation then runs against the calibrated spec with double-pass review on dispute-prone classes. The calibration record ships with the delivery so model owners understand why borderline cases were resolved a particular way.

## Can YPAI build RLHF preference data for LLM fine-tuning?

Yes. RLHF and DPO preference data, multi-turn instruction-tuning corpora, and tool-use traces for agentic fine-tuning are delivered as bespoke engagements. Reviewers are screened for the specific domain (legal, medical, code, customer-support) so preferences reflect domain expertise. Annotator wage transparency is part of the engagement under the <a href="/data-solutions/ethical-framework/">ethical framework</a>.

## What output formats does YPAI deliver?

Deliveries are shipped in CoNLL, BIO/IOB, JSON-Lines, brat-standoff, HuggingFace-Datasets-compatible Parquet, and customer-specified schemas. Token offsets, character offsets, and source-document identifiers are preserved so the delivery is auditable against the original corpus.

## How is YPAI text annotation priced?

Engagements are quoted per-project after a feasibility scope. The pricing inputs include taxonomy depth, IAA target, locale, domain complexity, and any compliance overhead. A scoping conversation at <a href="/contact-us/">/contact-us/</a> produces a written proposal.