Speech data · Evaluation program
A scoped evaluation before production. The same controls, on a corpus you can judge.
Evaluation datasets are scoped, produced under the production controls, delivered with their QA artifacts, and reviewed by your team before any production commitment.
Production terms are negotiated separately, after the evaluation is reviewed.
| Criterion | What the corpus shows | Reviewed by |
|---|---|---|
| Format compliance | Encoding, sample rate and channel structure against the specification agreed in scoping | Your engineering team |
| Audio quality | Acoustic characteristics against the agreed thresholds, with the validation results | YPAI QA, then your team |
| Metadata structure | Manifests and provenance fields as specified, one record per recording | Your data team |
| Fit for purpose | Alignment with the project requirements the evaluation was scoped against | Your technical and business review |
| Provenance | Sourced through YPAI's own contributor network and platform, documented | Legal and compliance, on request |
Volume, timeline and usage terms bounded in the evaluation agreement
Purpose validate · decide
Validate quality, format and fit before committing to production scale.
Evaluation datasets allow enterprise teams to validate speech data quality, format, and fit-for-purpose before committing to production-scale engagement.
- What it validates
- Technical format compliance, audio quality characteristics, metadata structure, and alignment with project requirements.
- Its shape
- A bounded corpus with agreed volume, timeline and usage terms. Scope and constraints are defined during evaluation scoping.
- Intended outcome
- Evaluation results inform internal technical and business decisions regarding subsequent engagement.
Evaluation scope and deliverables are defined contractually during scoping.
Scoping languages · domains · limits
Technical and contractual expectations are aligned before the first recording.
Evaluation scoping aligns technical requirements and contractual expectations before work begins.
- Scope definition
- Languages, domains, audio formats and quality criteria are discussed and agreed during scoping.
- Constraints
- Volume limits, timeline boundaries and usage restrictions are established before the evaluation starts.
- Alignment
- Technical teams and legal or procurement stakeholders share the same expectations of the deliverables.
Scoping discussions address your specific requirements. Parameters are set per evaluation.
Controls collection · qa · provenance
The evaluation corpus is produced under the operational controls production uses.
Evaluation datasets are produced using the same operational controls applied to production data where supported.
- Collection controls
- Data is collected through controlled channels with the consent mechanisms and quality validation production uses.
- Quality assurance
- Validation and QA procedures are applied during the evaluation. The methods are documented in the deliverables.
- Provenance
- Evaluation data is sourced through YPAI's own contributor network and platform. Provenance is documented and available for review.
Specific controls and validation procedures are defined during evaluation scoping.
Delivery and transition deliver · review · decide
Delivered with its artifacts, reviewed by your team, then a decision that is yours to make.
Delivery method, access duration and artifact scope are defined during scoping. Production scope, terms and pricing are negotiated separately afterwards.
-
Delivery
Format, encoding and metadata structure follow the technical specification agreed in scoping.
-
Artifacts
QA summaries, metadata manifests and provenance documentation, where supported and agreed.
-
Review
Your internal team reviews the deliverables. Feedback is documented and addressed per agreed procedure.
-
Outcome
Results inform your internal decision on subsequent engagement. Outcomes are documented for your review.
-
Production
Starts on explicit approval from designated stakeholders, under standard enterprise scoping and contracting.
The decision, and what follows each answer
-
Go
Production starts on explicit approval from your designated stakeholders, under standard enterprise scoping and contract terms.
-
Revise the scope
Feedback is documented and addressed per the agreed procedure; the scope is revised in the agreement before anything further is produced.
-
No go
Evaluation access ends as the agreement defines. Terms set no precedent, and reuse beyond the agreed scope needs separate authorization.
Boundaries scope · reuse · access
Five boundaries, stated in the evaluation agreement.
Evaluation programs operate within defined boundaries so scope and expectations match on both sides.
The agreement states
- Evaluation informs the decision; production outcomes are agreed separately.
- Evaluation data is used within the agreed scope; reuse beyond it needs separate authorization.
- Terms are engagement-specific and set no precedent for future engagements.
- Evaluation scope is limited to parameters agreed during scoping.
- Evaluation access is time-bound as defined in the evaluation agreement.
Specific limitations and boundaries are documented in the evaluation agreement.
Evaluation
Scope, cost and what comes after
Is the evaluation free?
Evaluation terms, including any associated costs, are defined during evaluation scoping. We work with your procurement team to establish appropriate terms based on scope and requirements.
How large is a typical evaluation dataset?
Dataset size is defined during evaluation scoping based on your technical requirements and validation needs. Evaluation scope varies by engagement, so the size is set per evaluation.
Does evaluation data match production quality?
Evaluation datasets use the same controls and quality procedures as production where supported. Specific quality criteria and validation methods are defined during scoping.
Can we define custom evaluation criteria?
Custom criteria and validation requirements are addressed during evaluation scoping. We work with your technical team to establish appropriate evaluation parameters.
What happens after evaluation?
Evaluation outcomes inform subsequent decisions. Production terms, if applicable, are negotiated separately. The decision to proceed to production is yours.
Can evaluation data be retained?
Data retention terms are defined in the evaluation agreement. Retention scope and conditions are agreed upon during scoping based on your requirements and applicable regulations.
Who reviews evaluation results?
Evaluation results are provided to your designated internal team for technical and business review. Review procedures and deliverables are defined during scoping.
Is there a time limit on evaluation?
Evaluation timelines and duration limits are defined contractually during scoping. We work with your team to establish appropriate timeframes based on project requirements.
What if evaluation does not meet our needs?
The decision to engage further is yours, informed by the evaluation outcome. Feedback on evaluation results is welcomed.
Is evaluation available for all languages and domains?
Language and domain availability for evaluation is discussed during scoping. We assess feasibility based on your requirements and current capabilities.
How is evaluation data delivered?
Delivery format and method are defined during evaluation scoping. Delivery follows the same technical specifications applicable to production where supported.
Can we request changes during evaluation?
Scope changes during evaluation are handled through agreed procedures. Change requests are assessed for impact and require appropriate approval before implementation.
Bring the requirement. The evaluation is scoped from it.
Evaluation scope is defined during the initial consultation from your project requirements. Volume, timeline and usage terms come out of that step.
Speech data overview Service Level Agreement Technical specifications DPA overview Audio data QA and acceptance criteria Improving Whisper for European languages