Evaluation Program

1. Executive summary

Purpose

Evaluation precedes production commitment and informs go/no-go decisions

Scope

Scope and constraints defined during evaluation scoping

Controls

Evaluation datasets use the same controls as production where supported

QA

Quality assurance and validation procedures applied during evaluation

Results

Provided for internal technical and business review

Production terms

Negotiated separately after evaluation completion

Transition

No automatic transition to production without explicit approval

Terms

Contractual and engagement-specific

2. Purpose of the evaluation

Evaluation datasets allow enterprise teams to validate speech data quality, format, and fit-for-purpose before committing to production-scale engagement.

What evaluation validates: Technical format compliance, audio quality characteristics, metadata structure, and alignment with project requirements.

What evaluation is not: Evaluation is not an open trial, production-scale delivery, or unlimited access. Scope and constraints are defined during evaluation scoping.

Intended outcome: Evaluation results inform internal technical and business decisions regarding subsequent engagement.

Evaluation scope and deliverables are defined contractually during scoping.

3. Evaluation scoping

Evaluation scoping aligns technical requirements and contractual expectations before work begins.

Scope definition: Languages, domains, audio formats, and quality criteria are discussed and agreed upon during scoping.

Constraints: Volume limits, timeline boundaries, and usage restrictions are established before evaluation commences.

Alignment: Scoping ensures technical teams and legal/procurement stakeholders share common expectations regarding evaluation deliverables.

Scoping discussions address your specific requirements. Standard parameters are not published.

4. Controls applied during evaluation

Evaluation datasets are produced using the same operational controls applied to production data where supported.

Collection controls: Data is collected through controlled channels with appropriate consent mechanisms and quality validation.

Quality assurance: Validation and QA procedures are applied during evaluation. Specific QA methods are documented in evaluation deliverables.

Separation: Evaluation data is not sourced from open or crowdsourced channels. Provenance is documented and available for review.

Specific controls and validation procedures are defined during evaluation scoping.

5. Delivery and review

Evaluation data is delivered according to specifications agreed during scoping.

Delivery format: Format, encoding, and metadata structure follow technical specifications agreed upon during scoping.

Accompanying artifacts: QA summaries, metadata manifests, and provenance documentation are provided where supported and agreed.

Review process: Your internal team reviews evaluation deliverables. Feedback is documented and addressed according to agreed procedures.

Delivery method, access duration, and artifact scope are defined during scoping.

6. Transition to production

Evaluation completion does not constitute automatic transition to production engagement.

Evaluation outcomes: Evaluation results inform your internal decision regarding subsequent engagement. Outcomes are documented for your review.

Production terms: Production scope, terms, and pricing are negotiated separately. Evaluation terms do not automatically apply to production.

Approval required: Transition to production requires explicit approval from designated stakeholders. There is no obligation to proceed.

Production engagement follows standard enterprise scoping and contracting processes.

7. Boundaries and limitations

Evaluation programs operate within defined boundaries to ensure appropriate scope and expectations.

• Evaluation does not guarantee production outcomes or specific results.
• Evaluation data may not be reused beyond agreed scope without separate authorization.
• Terms are engagement-specific and do not create precedent for future engagements.
• Evaluation scope is limited to parameters agreed during scoping.
• Evaluation access is time-bound as defined in the evaluation agreement.

Specific limitations and boundaries are documented in the evaluation agreement.

8. Evaluation FAQ

Is the evaluation free?

Evaluation terms, including any associated costs, are defined during evaluation scoping. We work with your procurement team to establish appropriate terms based on scope and requirements.

How large is a typical evaluation dataset?

Dataset size is defined during evaluation scoping based on your technical requirements and validation needs. We do not publish standard sizes as evaluation scope varies by engagement.

Does evaluation data match production quality?

Evaluation datasets use the same controls and quality procedures as production where supported. Specific quality criteria and validation methods are defined during scoping.

Can we define custom evaluation criteria?

Custom criteria and validation requirements are addressed during evaluation scoping. We work with your technical team to establish appropriate evaluation parameters.

What happens after evaluation?

Evaluation outcomes inform subsequent decisions. Production terms, if applicable, are negotiated separately. There is no obligation to proceed to production following evaluation.

Can evaluation data be retained?

Data retention terms are defined in the evaluation agreement. Retention scope and conditions are agreed upon during scoping based on your requirements and applicable regulations.

Who reviews evaluation results?

Evaluation results are provided to your designated internal team for technical and business review. Review procedures and deliverables are defined during scoping.

Is there a time limit on evaluation?

Evaluation timelines and duration limits are defined contractually during scoping. We work with your team to establish appropriate timeframes based on project requirements.

What if evaluation does not meet our needs?

There is no obligation to proceed following evaluation. Evaluation outcomes inform your decision whether to engage further. Feedback on evaluation results is welcomed.

Is evaluation available for all languages and domains?

Language and domain availability for evaluation is discussed during scoping. We assess feasibility based on your requirements and current capabilities.

How is evaluation data delivered?

Delivery format and method are defined during evaluation scoping. Delivery follows the same technical specifications applicable to production where supported.

Can we request changes during evaluation?

Scope changes during evaluation are handled through agreed procedures. Change requests are assessed for impact and require appropriate approval before implementation.

Evaluation scope is defined during initial consultation based on your project requirements.