HLS/DASH manifest drift
on a primetime live lane.
A working postmortem on a weekday primetime live event where the live manifest packager drifted mid-window — segment numbering jumped 8 steps across a 2-second window, the upstream EXT-X-DATERANGE cues were dropped on the stitched output, and an out-of-window #EXT-X-DISCONTINUITY tag was injected so the player-side hls.js MOOV parse stalled for ~11% of the primetime cohort for ~8 minutes. The Streamwake agentic ops layer caught it from viewer-impact telemetry, ranked it as a packager-segment-drift with 89% confidence, and remediated it with a snapshot+reread plus a reissued-packager split — split explicitly into the acts the agent did autonomously and the acts it surfaced to humans.
Book a technical demo for HLS/DASH manifest drift
Read the postmortem — then bring your own incident to Streamwake.
Two ways to engage on this exact failure pattern: book a 30-minute technical demo where we walk through the probe cascade on your source, or hand us an archived incident and watch the agent diagnose it end-to-end.
Both routes land on the scoping intake form — no SDR gate.
What the cohort saw
The first things to read on any real primetime manifest-drift incident are the cohort-level numbers — how many sessions a parse-fail symptom hit, how long, and where the symptom landed on the packager lane. Three numbers did the heavy lifting here.
Roughly 11% of the eu-west primetime cohort — session-level hls.js MOOV parse stalled against a drifting EXT-X-MEDIA-SEQUENCE anchor and a cueless, out-of-window-discontinuity manifest; players dropped into the startup-stall lane; cohort parse-fail ratio spiked 8× over baseline.
~8 minutes between the first drifting manifest probe at T+0m and the cohort-parse-fail ratio returning within baseline at T+22m. The postmortem window goes longer because the mid-window anchor guardrail ships on the next deploy — not in the same incident window.
ext_x_media_sequence_drift_s: 1.93 on pkg-prime-04 (cdn-A/fra02). The same cohort on the same CDN was reading manifest 200-OK — a healthy HTTP response holding a drifting manifest — ruling out CDN-side failure as the cause.
How Streamwake classified this incident
Three ranked hypotheses, with the top one filing the timeline and the secondary signal carrying the cause. The discontinuity_injection_stitcher and vod_manifest_dropping_cues lanes are included to make explicit that the agent ruled them out — the symptom was on the packager mid-window anchor, not on a stitcher-only discontinuity bug or on the stitched-VOD path.
manifest_packager_segment_drift — the live packager’s mid-window EXT-X-MEDIA-SEQUENCE anchor drifted forward without writing the missing segments. Three signals line up: manifest.ext_x_media_sequence_drift_s 1.93s (baseline 0), manifest.ext_x_daterange_coverage_vs_upstream 0.0 vs 1.0 baseline (four cues dropped), manifest.ext_x_discontinuity_in_window true on segment 041882.
discontinuity_injection_stitcher was the secondary signal ranked at 6% — the discontinuity tag co-occurred, but on the same cohort, with the segment-numbering jump and the cueless coverage. The dismissal rule was “rank the cause on the broader probe pattern, not on the single symptom that landed on the player”; the mid-window anchor drift is the decisive signal pattern.
- Region: eu-west
- Status: resolved (window closed)
- Opened: 2026-08-09 19:14 UTC
- Spread: contained to eu-west — na-east cohort unaffected
Above the 80% threshold the agent treats as a confident top-hypothesis filing. vod_manifest_dropping_cues · 0.05 was cleared explicitly because this is a live HLS lane (not a stitched-VOD seam) and the failure-mode pattern does not match the scte35 ad-marker loss postmortem’s cue_filter_overshoot lane.
Incident timeline
Ten events: detection on the cohort, classification across three ranked hypotheses, four autonomous acts the agent took on its own, three acts it surfaced to humans, the recovery probe, and the resolution. The right-hand “act” tag is what makes this postmortem distinct from a generic write-up — it pins the split between autonomous agentic ops and the work that genuinely needed a person.
Today
- T+0mDetectionby cohort agent · eu-westact · autonomous
Manifest drift & cohort parse-fail spike simultaneously
cohort.cohort_hls_parse_fail_ratio drifted from 0.015 baseline to 0.118 over a 90-second window; cohort.cohort_player_startup_stall_ratio flipped to fail at 0.107 (baseline 0.011); manifest.ext_x_media_sequence_drift_s reported a 1.93s mid-window jump.
Aug 9, 07:14:37 PM - T+1mClassificationby Streamwake reliability agentact · autonomous
Ranked: manifest_packager_segment_drift (0.89) · secondary discontinuity_injection_stitcher (0.06)
Top hypothesis reads 89% confidence. Two ranked alternatives dismissed: discontinuity_injection_stitcher (the discontinuity tag co-occurs with segment-numbering drift and cueless coverage on the same cohort — pattern matches a broader packager drift, not a stitcher-only bug), vod_manifest_dropping_cues (this is a live HLS lane without a cdn-A→cdn-E seam; pattern does not match SCTE-35 VOD-stitcher cue-filter case).
Aug 9, 07:15:37 PM - T+2mAutomated actionby Streamwake reliability agentact · autonomous
Emitted snapshot_manifest_reread_from_clean_source + repackage_title_with_clean_window_anchor
Snapshot the live HLS manifest from a clean source window, re-read the EXT-X-MEDIA-SEQUENCE anchor, and reissue a manifest on the affected lane with the cues restored and the out-of-window discontinuity tag stripped. The reissued manifest preserves the EXT-X-DATERANGE cues from upstream so the SSAI/DSA lane is whole.
Aug 9, 07:16:37 PM - T+3mAutomated actionby Streamwake reliability agentact · autonomous
Reissued-packager split kick-in on the affected lane
Split the affected lane (pkg-prime-04) onto pkg-prime-reissue-01 so the drift window can’t recur on the original primary packager; cohort manifest reads migrate to the new packager within the 90-second probe window.
Aug 9, 07:17:37 PM - T+5mAutomated actionby Streamwake reliability agentact · autonomous
Rebalance hint queued: tighten_mid_window_media_sequence_anchor_check
Raise the mid-window EXT-X-MEDIA-SEQUENCE drift guardrail from "report at 2.0s drift" to "snapshot at 1.0s drift" so the next packager anchor miss is caught before the cohort parse-fail ratio climbs. Staged for the next day’s deployment.
Aug 9, 07:19:37 PM - T+11mSurfaced to humanby agent → operatoract · surfaced to humans
Operator engaged the packager vendor on the mid-window anchor model
Vendor confirmed that the mid-window EXT-X-MEDIA-SEQUENCE anchor used a segment-availability check that ran on the write-side, not the read-side — so a manual replay of the upstream segment could push the anchor forward without writing the missing segments. New policy: anchor on read-side availability, not write-side enqueue. Vendor pushed a config patch to the mid-window anchor.
Aug 9, 07:25:37 PM - T+14mSurfaced to humanby on-callact · surfaced to humans
On-call paged for the manifest-drift root-cause review
Page acknowledged within 85s; reviewer confirmed the agent’s rank and that the snapshot+reread plus the reissued-packager split had prevented the cohort from doubling into a player-startup-stall clip before the reissue landed.
Aug 9, 07:28:37 PM - T+18mSurfaced to humanby reliability teamact · surfaced to humans
Postmortem write-up assigned (this page)
Reliability team assigned the public postmortem; this page is the resulting write-up, with the ranked hypotheses, recommended fix, and agentic-act split pinned from the timeline.
Aug 9, 07:32:37 PM - T+22mAutomated actionby Streamwake reliability agentact · autonomous
Re-probed the cohort; cohort_hls_parse_fail_ratio returned within baseline
cohort.cohort_hls_parse_fail_ratio dropped from 0.118 back to 0.018 over the next 90 seconds; ext_x_media_sequence_drift_s cleared under 0.20s on the next manifest probe; ext_x_daterange_coverage_vs_upstream flipped to 1.0 with all four upstream cues preserved.
Aug 9, 07:36:37 PM - T+35mResolutionby Operator + agentact · autonomous
Incident resolved; anchor-check guardrail + vendor anchor-config patch ship next day
Player-side hls.js MOOV parse stalled cleared for the affected cohort; no further mid-window anchor drift on the reissued packager; the anchor-check guardrail bump and the vendor anchor-config patch both ship in the next deploy window.
Aug 9, 07:49:37 PM
- classify · ranked three hypotheses with confidence in 90s
- snapshot · emitted snapshot_manifest_reread_from_clean_source on the drifting lane
- repackage · emitted repackage_title_with_clean_window_anchor (cues restored, discontinuity stripped)
- reissue · split the affected lane (pkg-prime-04 → pkg-prime-reissue-01)
- rebalance · queued tighten_mid_window_media_sequence_anchor_check for the next deploy
- operator · engaged the packager vendor on the mid-window anchor model
- on-call · paged for the manifest-drift root-cause review
- reliability team · assigned the public postmortem write-up (this page)
Anatomy of the evidence packet
The two packets on the failing source — a manifest-drift probe packet on the drift-window manifest (with the cues missing and the out-of-window discontinuity tag), and the agent timeline response with the ranked hypotheses and the rebalance hints. The probe packet is what the agent decided on; the timeline response is what the agent emitted.
GET /live/event/manifest.m3u8 HTTP/1.1
host: cdn.example.com
accept: application/vnd.apple.mpegurl
----- cycle 0 (T+0m, before snapshot+reread) -----
HTTP/2 200
content-type: application/vnd.apple.mpegurl
server-timing: manifest-fetch;dur=22
x-cdn: cdn-A/fra02
x-packager: pkg-prime-04
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:41879 ← jumped from 41871 over a 2s window (8-step drift, baseline 0)
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.000,
041879.ts
#EXTINF:6.000,
041880.ts
#EXTINF:6.000,
041881.ts
#EXTINF:6.000,
041882.ts ← starts with #EXT-X-DISCONTINUITY (out-of-window)
#EXT-X-DISCONTINUITY
#EXTINF:6.000,
041883.ts
# ext_x_daterange_coverage_vs_upstream: 0.0 (upstream had 4 cues in this window)
# ext_x_media_sequence_drift_s: 1.93 (segment numbering jumped in 1.93s)
# ext_x_discontinuity_in_window: true (out-of-window discontinuity tag on 041882)
# cohort impact over 90s window:
# cohort_hls_parse_fail_ratio: 0.118
# cohort_player_startup_stall_ratio: 0.107 (within baseline 0.011)
# encoder.bitrate_target_delta_pct: 0.30 (within baseline)
# cdn.availability: pass (edge serving manifest green)
----- cycle 1 (T+0m +3s, after snapshot+reread + packager split) -----
HTTP/2 200
content-type: application/vnd.apple.mpegurl
server-timing: manifest-fetch;dur=24
x-cdn: cdn-A/fra02
x-packager: pkg-prime-reissue-01
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:41881 ← reads continuous from prior window
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.000,
041881.ts
#EXTINF:6.000,
041882.ts ← no discontinuity tag, presentation continues
#EXTINF:6.000,
041883.ts
# ext_x_daterange_coverage_vs_upstream: 1.0 (cue points restored)
# ext_x_media_sequence_drift_s: 0.12 (within baseline; packager reads continuous)
# ext_x_discontinuity_in_window: false (no out-of-window tag injected)
# rebalance_hints emitted: snapshot_manifest_reread_from_clean_source,
# repackage_title_with_clean_window_anchor,
# reissue_packager_split_for_affected_lane- EXT-X-MEDIA-SEQUENCE →
41879 (jumped from 41871 in 2s) - ext_x_daterange_coverage_vs_upstream →
0.0 (4 cues dropped) - ext_x_discontinuity_in_window →
true on segment 041882 - cohort_hls_parse_fail_ratio →
0.118 vs 0.015 baseline (90s window) - encoder.bitrate_target_delta_pct → pass. Encoder healthy on the same cohort. cdn.availability → pass. Manifest served green on the same CDN.
{
"stream_id": "cklivemanifestdriftprimetime8821",
"source": "https://cdn.example.com/live/event/manifest.m3u8",
"protocol": "HLS",
"checked_at": "2026-08-09T19:14:37Z",
"ranked_hypotheses": [
{
"rank": 1,
"hypothesis": "manifest_packager_segment_drift",
"confidence": 0.89,
"evidence_signals": [
"manifest.ext_x_media_sequence_drift_s → fail (1.93s jump, baseline 0; 8-step segment-numbering desync over 2s window)",
"manifest.ext_x_daterange_coverage_vs_upstream → fail (0.0 vs 1.0 baseline, 4 cues dropped across the stitch seam)",
"manifest.ext_x_discontinuity_in_window → fail (out-of-window discontinuity tag on segment 041882)",
"cohort.cohort_hls_parse_fail_ratio → fail (0.118 vs 0.015 baseline over 90s window)",
"cohort.cohort_player_startup_stall_ratio → fail (0.107 vs 0.011 baseline over 90s window)",
"encoder.bitrate_target_delta_pct → pass (0.30 within baseline; encoder healthy)",
"cdn.availability → pass (manifest served green on the same cohort/CDN)"
]
},
{
"rank": 2,
"hypothesis": "discontinuity_injection_stitcher",
"confidence": 0.06,
"evidence_signals": [
"discontinuity tag present but co-occurs with segment-numbering drift and cueless coverage on the same cohort — pattern matches a broader packager drift, not a stitcher-only discontinuity bug"
]
},
{
"rank": 3,
"hypothesis": "vod_manifest_dropping_cues",
"confidence": 0.05,
"evidence_signals": [
"this is a live HLS lane not a stitched-VOD lane; SCTE-35 postmortem covers the VOD stitcher cue-filter case explicitly — pattern does not match (no cdn-A→cdn-E seam, no vod.cue_point_coverage probe involvement)"
]
}
],
"agent_rebalance_hints": [
"snapshot_manifest_reread_from_clean_source",
"repackage_title_with_clean_window_anchor",
"reissue_packager_split_for_affected_lane"
],
"surfaced_to_humans": [
{"owner": "packager vendor", "task": "engage on mid-window EXT-X-MEDIA-SEQUENCE anchor model"},
{"owner": "on-call", "task": "page for the manifest-drift root-cause review"},
{"owner": "reliability team", "task": "assign the postmortem write-up"}
]
}- rebalance hint →
snapshot_manifest_reread_from_clean_source - rebalance hint →
repackage_title_with_clean_window_anchor - rebalance hint →
reissue_packager_split_for_affected_lane - surfaced → engaged the packager vendor on the mid-window EXT-X-MEDIA-SEQUENCE anchor model
- surfaced → paged on-call for the manifest-drift root-cause review
Snapshot+reread now · tighten the anchor guardrail in the next deploy
Two changes close the gap the timeline exposed. The first is what the agent already emitted on this incident (the snapshot+reread plus the reissued packager); the second is what ships on the next deploy (the mid-window anchor guardrail). Both are tracked as exported rebalance_hints rows.
Take the drifting packager off the cohort, snap the live HLS manifest from a clean source window whose EXT-X-MEDIA-SEQUENCE anchor is contiguous from the cohort’s last known-good read, restore the upstream EXT-X-DATERANGE cues on the reissue, and strip the out-of-window #EXT-X-DISCONTINUITY tag. The reissued manifest is what the cohort reads for the rest of the window — the drifting primary stays off-lane until the vendor fix lands.
manifest.ext_x_media_sequence_drift_s stays under 0.20s on the next manifest probe for ten consecutive 30-second windows; ext_x_daterange_coverage_vs_upstream flips to 1.0 with all upstream cues preserved; ext_x_discontinuity_in_window stays false on the reissued lane.
The current packager anchor guardrail reports at 2.0s mid-window drift, which let this incident’s 8-step segment-numbering jump reach the cohort before rebalance. The bumped guardrail triggers at 1.0s drift and snapshots the manifest immediately on the affected lane — so the next mid-window anchor miss is caught before the cohort parse-fail ratio climbs. Staged for the next deploy so the operator team can review the false-positive rate.
cohort.cohort_hls_parse_fail_ratio drift on the affected geo drops under 2% over a 24-hour window; manifest.ext_x_media_sequence_drift_s stays under 0.20s on every 5-second probe across a primetime cycle; guardrail_false_positive_rate stays under 4% over the same window.
Want Streamwake to catch this on your primetime cohort?
Sign up, register an HLS probe, and the same manifest.ext_x_media_sequence_drift_s · manifest.ext_x_daterange_coverage_vs_upstream · manifest.ext_x_discontinuity_in_window probes that produced the timeline above run on every refresh — and surface in a Slack channel, a webhook, or the streams dashboard.
- Pick a recent on-call incident — manifest stall, edge miss, player-side stall, or peer congestion.
- We replay it through the same reliability-agent probe cascade used on the postmortem above.
- You walk away with a written what-could-have-been-Automated readout, not a sales deck.
Read the next postmortem
The other live entries cover DRM license-server cold-starts, encoder ABR ladder overshoots, and SCTE-35 cue-loss on the live→VOD stitch seam.