Skip to content
Docs Docs

Testing Approach

POC testing moves Rilla from local validation to monitored live cohorts, then expands only when playback guardrails, telemetry, and CDN-only comparisons support the next rollout step.

CDN fallback makes testing reversible

Section titled “CDN fallback makes testing reversible”

CDN fallback makes Rilla practical to test in controlled stages. Peer-assisted delivery can be enabled for selected platforms, regions, streams, or cohorts while your CDN remains available when peer delivery is disabled, unavailable, or unsuitable for playback.

This keeps the POC reversible: exposure can increase gradually, monitoring can catch quality changes early, and feature flags or traffic segmentation can return viewers to CDN-only delivery if guardrails are not met. Technical fallback behavior is covered in CDN Fallback.

Local testing before live traffic

Section titled “Local testing before live traffic”

  1. Local playback testing

    Goal: Confirm Rilla can run beside the target player before any live-user exposure.

    Local testing proves that playback, fallback, and player performance remain stable in a representative environment.

    Confirm:

    • Playback starts and continues with Rilla enabled
    • CDN fallback remains available when peer delivery is disabled, unavailable, or bypassed
    • Required request routing, CMCD, and CDN header assumptions are in place
    • SDK events needed for monitoring are emitted
    • Startup, buffering, CPU, and memory remain within expected limits

  2. Telemetry and monitoring check

    Goal: Confirm the POC can compare Rilla-enabled delivery against a CDN-only baseline.

    Measurement validation ensures deflection, fallback, and QoE signals are visible before live exposure increases.

    Confirm visibility into:

    • CDN and peer delivery mix
    • Fallback behavior
    • Startup time, rebuffering, playback errors, and bitrate stability
    • Peer participation and connection success
    • CDN-delivered GB, peer-delivered GB, and deflection during representative windows
    • Alerts for integration or playback issues

  3. A/B testing against CDN-only delivery

    Goal: Measure Rilla-enabled playback against comparable CDN-only traffic.

    Live validation starts with isolated cohorts so delivery gains and quality deltas can be evaluated before exposure increases.

    Required controls:

    • CDN-only control group
    • CDN + Rilla test group
    • Feature flag or traffic segmentation
    • Comparable event windows, regions, platforms, and content profiles
    • Agreed thresholds for deflection, fallback, startup time, rebuffering, and playback errors

Technical reference: Observability and Monitoring.

Controlled, incremental rollout

Section titled “Controlled, incremental rollout”

Rollout expands through explicit controls for capacity, viewer exposure, platform, region, and event window. Feature flags, traffic segmentation, and kill switch controls keep each increase reversible while playback guardrails are monitored.

  1. Internal QA

    Confirm the integration, fallback behavior, request routing, telemetry, dashboards, and alerts on internal devices before live traffic begins.

  2. Limited live cohort

    Enable Rilla for a narrow monitored cohort. Compare deflection, fallback behavior, startup time, rebuffering, and playback errors against the CDN-only control group.

  3. Scaled live cohort

    Increase exposure by platform, region, content type, or event window while quality guardrails remain stable and monitoring remains actionable.

  4. Production readiness decision

    Use the POC evidence to decide whether the delivery path, measurement model, operational controls, and quality results support broader rollout.