--- title: "Evaluation Programme | Enterprise Speech Data | YPAI" url: https://ypai.ai/speech-data/evaluation-program/ description: "Structured evaluation of enterprise speech data before production engagement. Scope, controls and terms are defined during consultation." source: "src/copy/routes (route /speech-data/evaluation-program/)" --- # A scoped evaluation before production. The same controls, on a corpus you can judge. > Structured evaluation of enterprise speech data before production engagement. Scope, controls and terms are defined during consultation. Evaluation datasets are scoped, produced under the production controls, delivered with their QA artefacts, and reviewed by your team before any production commitment. Production terms are negotiated separately, after the evaluation is reviewed. Evaluation scorecard ยท what your team judges - Encoding, sample rate and channel structure against the specification agreed in scoping - Acoustic characteristics against the agreed thresholds, with the validation results - Manifests and provenance fields as specified, one record per recording - Alignment with the project requirements the evaluation was scoped against - Sourced through YPAI's own contributor network and platform, documented Volume, timeline and usage terms bounded in the evaluation agreement ## What an evaluation is, and where it ends. Specific terms, dataset parameters and timelines are fixed contractually during evaluation scoping. - **Purpose**: Evaluation precedes production commitment and informs go/no-go decisions - **Scope**: Scope and constraints defined during evaluation scoping - **Controls**: Evaluation datasets use the same controls as production, with any exception named in the agreement - **QA**: Quality assurance and validation procedures applied during evaluation - **Results**: Provided for internal technical and business review - **Production terms**: Negotiated separately after evaluation completion - **Transition**: Production starts on explicit approval from your designated stakeholders - **Terms**: Contractual and engagement-specific ## Validate quality, format and fit before committing to production scale. Evaluation datasets allow enterprise teams to validate speech data quality, format, and fit-for-purpose before committing to production-scale engagement. Technical format compliance, audio quality characteristics, metadata structure, and alignment with project requirements. A bounded corpus with agreed volume, timeline and usage terms. Scope and constraints are defined during evaluation scoping. Evaluation results inform internal technical and business decisions regarding subsequent engagement. Evaluation scope and deliverables are defined contractually during scoping. - **What it validates**: Technical format compliance, audio quality characteristics, metadata structure, and alignment with project requirements. - **Scope and limits**: A bounded corpus with agreed volume, timeline and usage terms. Scope and constraints are defined during evaluation scoping. - **Intended outcome**: Evaluation results inform internal technical and business decisions regarding subsequent engagement. ## Technical and contractual expectations are aligned before the first recording. Evaluation scoping aligns technical requirements and contractual expectations before work begins. Languages, domains, audio formats, and quality criteria are discussed and agreed upon during scoping. Volume limits, timeline boundaries, and usage restrictions are established before evaluation commences. Scoping ensures technical teams and legal/procurement stakeholders share common expectations regarding evaluation deliverables. Scoping discussions address your specific requirements. Parameters are set per evaluation. - **Scope definition**: Languages, domains, audio formats and quality criteria are discussed and agreed during scoping. - **Constraints**: Volume limits, timeline boundaries and usage restrictions are established before the evaluation starts. - **Alignment**: Technical teams and legal or procurement stakeholders share the same expectations of the deliverables. ## The evaluation corpus is produced under the operational controls production uses. Evaluation datasets are produced under the same operational controls as production data. Where a control cannot apply at evaluation scale, it is named in the evaluation agreement. Data is collected through controlled channels with appropriate consent mechanisms and quality validation. Validation and QA procedures are applied during evaluation. Specific QA methods are documented in evaluation deliverables. Evaluation data is sourced through YPAI's own contributor network and platform. Provenance is documented and available for review. Specific controls and validation procedures are defined during evaluation scoping. - **Collection controls**: Data is collected through controlled channels with the consent mechanisms and quality validation production uses. - **Quality assurance**: Validation and QA procedures are applied during the evaluation. The methods are documented in the deliverables. - **Provenance**: Evaluation data is sourced through YPAI's own contributor network and platform. Provenance is documented and available for review. ## Delivered to the agreed specification, reviewed by your team, then a go, a revised scope or a no go. Delivery method, access duration and artefact scope are defined during scoping. Production scope, terms and pricing are negotiated separately afterwards. - **Delivery**: Format, encoding and metadata structure follow the technical specification agreed in scoping. - **Artefacts**: QA summaries, metadata manifests and provenance documentation, where supported and agreed. - **Review**: Your internal team reviews the deliverables. Feedback is documented and addressed per agreed procedure. - **Outcome**: Results inform your internal decision on subsequent engagement. Outcomes are documented for your review. - **Production**: Starts on explicit approval from designated stakeholders, under standard enterprise scoping and contracting. The decision, and what follows each answer - **Go**: Production starts on explicit approval from your designated stakeholders, under standard enterprise scoping and contract terms. - **Revise the scope**: Feedback is documented and addressed per the agreed procedure; the scope is revised in the agreement before anything further is produced. - **No go**: Evaluation access ends as the agreement defines. Terms set no precedent, and reuse beyond the agreed scope needs separate authorisation. ## Five boundaries, stated in the evaluation agreement. Both sides work to the same scope and expectations. - Evaluation informs the decision; production outcomes are agreed separately. - Evaluation data is used within the agreed scope; reuse beyond it needs separate authorisation. - Terms are engagement-specific and set no precedent for future engagements. - Evaluation scope is limited to parameters agreed during scoping. - Evaluation access is time-bound as defined in the evaluation agreement. Specific limitations and boundaries are documented in the evaluation agreement. ## Scope, cost and what comes after ### Is the evaluation free? Evaluation terms, including any cost, are set during evaluation scoping with your procurement team, from the scope and requirements. ### How large is a typical evaluation dataset? Dataset size is defined during evaluation scoping based on your technical requirements and validation needs. Evaluation scope varies by engagement, so the size is set per evaluation. ### Does evaluation data match production quality? Evaluation datasets use the same controls and quality procedures as production, and any control that cannot apply at evaluation scale is named in the evaluation agreement. Specific quality criteria and validation methods are defined during scoping. ### Can we define custom evaluation criteria? Custom criteria and validation requirements are addressed during evaluation scoping. We work with your technical team to establish appropriate evaluation parameters. ### What happens after evaluation? Evaluation outcomes inform subsequent decisions. Production terms, if applicable, are negotiated separately. The decision to proceed to production is yours. ### Can evaluation data be retained? Data retention terms are defined in the evaluation agreement. Retention scope and conditions are agreed upon during scoping based on your requirements and applicable regulations. ### Who reviews evaluation results? Evaluation results are provided to your designated internal team for technical and business review. Review procedures and deliverables are defined during scoping. ### Is there a time limit on evaluation? Evaluation timelines and duration limits are defined contractually during scoping. We work with your team to establish appropriate timeframes based on project requirements. ### What if evaluation does not meet our needs? The decision to engage further is yours, informed by the evaluation outcome. Feedback on evaluation results is welcomed. ### Is evaluation available for all languages and domains? Language and domain availability for evaluation is discussed during scoping. We assess feasibility based on your requirements and current capabilities. ### How is evaluation data delivered? Delivery format and method are defined during evaluation scoping. Delivery follows the same technical specifications applicable to production where supported. ### Can we request changes during evaluation? Scope changes during evaluation are handled through agreed procedures. Change requests are assessed for impact and require appropriate approval before implementation. Bring the requirement. The evaluation is scoped from it. Evaluation scope is defined during the initial consultation from your project requirements. Volume, timeline and usage terms come out of that step. - [Speech data overview](https://ypai.ai/speech-data/) - Service Level Agreement - [Technical specifications](https://ypai.ai/speech-data/technical-specifications/) - [DPA overview](https://ypai.ai/speech-data/dpa/) - [Audio data QA and acceptance criteria](https://ypai.ai/compliance/audio-data-qa/) - [Improving Whisper for European languages](https://ypai.ai/blog/data-engineering/improving-whisper-european-languages/)