Troubleshoot low-latency HLS
without losing the live edge.
A working guide to the low-latency HLS failure patterns Streamwake catches on real delta playlists — chunked-transfer ingest stalls, partial-segment prefetch terminations, GOP timing drift, manufacturing-server cadence drift, playlist-delta size regressions, and blocking-playlist-request storms — written so an on-call engineer can read it next to a recent chunklist capture and act on it.
What Streamwake checks
Five families of probes, each with a deterministic pass / fail / warn verdict that feeds the timeline. Every check has a name — that's the name you'll see on the agents feed.
- First-chunk TTFB p95 against the phased baseline — flags origin read-through stalls on
Transfer-Encoding: chunkedpartials. - Chunk-completion p95 — surfaces stalled mid-PART writes that never finish a chunk before the next PART lands.
- Pairs with
manufacturing_server.cadence_driftso the stalled socket's downstream effect on the partial-segment emitter is visible.
- Counts INDEPENDENT PARTs the player requested versus PARTs the player actually consumed — terminations mid-stream mark a stale byte stream.
- Reports
EXT-X-PRELOAD-HINTalignment withCAN-PRELOAD-BEFORE. - Flags PARTs marked
INDEPENDENT=NOthat the player attempted to prefetch anyway.
- Every keyframe sits at the start of an
EXT-X-PART— mid-PART IDR is the partial-segment equivalent of a re-buffer event. - Encoder cadence drift > ±0.05s fails the probe — the packager cannot honor PART-TARGET.
- Replays trailing PARTs once cadence crosses the threshold.
- Per-second count of how many PARTs the partial-segment emitter actually shipped, against the expected rate driven by
PART-TARGET. - Hides behind a green
playlist.delta_updates— the playlist is produced, the BYTES trail. - Diff between manufactured and expected rate at p95 — > 15% drift is a fail.
_HLS_msn+_HLS_partmatch the next part on every probe cycle.- Delta bytes p95 within a threshold; > 1KB on a delta cursor is a regression that warns before a full BLOCKING request lands.
- BLOCKING playlist requests per minute vs ceiling — flips to fail the moment the cohort loses confidence in the cursor.
An LL-HLS variant + delta window
A LL-HLS variant with PART-INF, SERVER-CONTROL declaring CAN-BLOCK-RELOAD + PART-HOLD-BACK + CAN-PRELOAD-BEFORE, and a segment of 16 PARTs (one of them an off-cycle 0.660s part, one a 0.990s part — encoder cadence slipping). The blocking-replay trace on the right shows what the server returns when delta should have fired. The probes→fields map links each verdict to a line in the playlist.
#EXTM3U
#EXT-X-VERSION:9
#EXT-X-TARGETDURATION:6
#EXT-X-PART-INF:PART-TARGET=0.33
#EXT-X-SERVER-CONTROL:CAN-BLOCK-RELOAD=YES,PART-HOLD-BACK=1.0,HOLD-BACK=4.0,CAN-PRELOAD-BEFORE=YES
#EXT-X-MEDIA-SEQUENCE:1470
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.000,
seg1470.m4s
#EXT-X-PROGRAM-DATE-TIME:2026-08-16T11:02:10.000Z
#EXT-X-PART:DURATION=0.330,URI="seg1470.0.m4s",INDEPENDENT=YES
#EXT-X-PART:DURATION=0.330,URI="seg1470.1.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.2.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.3.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.4.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.5.m4s"
#EXT-X-PART:DURATION=0.660,URI="seg1470.6.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.7.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.8.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.9.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.10.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.11.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.12.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.13.m4s"
#EXT-X-PART:DURATION=0.330,URI="seg1470.14.m4s"
#EXT-X-PART:DURATION=0.990,URI="seg1470.15.m4s"
#EXTINF:6.000,
seg1471.m4s
#EXT-X-PROGRAM-DATE-TIME:2026-08-16T11:02:16.000Z
#EXT-X-PRELOAD-HINT:TYPE=PART,URI="seg1472.0.m4s"GET /live/event/llhls-variant.m3u8?_HLS_msn=1471&_HLS_part=4
HTTP/2 200
content-type: application/vnd.apple.mpegurl
date: Sat, 16 Aug 2026 11:02:17.412Z
cache-control: max-age=0
transfer-encoding: chunked
server-timing: _HLS_msn=1471; _HLS_part=4; part_drift_ms=212
x-cache: MISS
# delta window should have returned under 200B; instead the server
# shipped 4,920 B — a full BLOCKING reload triggered by a chunked-transfer
# chunk-drop on seg1470.6.m4s. The 212ms part_drift_ms is the cue.- ingest.chunked_transfer_stall → first chunk TTFB + chunk-completion p95
- part.prefetch_termination → ratio of
INDEPENDENT=YESPARTs the player consumed vs requested - gop.boundary_aligned →
EXT-X-PARTstart times vs GOP keyframe timestamps - manufacturing_server.cadence_drift → shipped PARTs/sec vs
EXT-X-PART-INF PART-TARGETimplied rate - playlist.delta_size → response body bytes with
_HLS_msn+_HLS_partcursor - blocking_playlist.storm_ceiling → full BLOCKING GETs per minute vs ceiling
The EXT-X-PART-INF:PART-TARGET=0.33 line tells the player where parts will land, EXT-X-SERVER-CONTROL opens the BLOCKING channel and gates PART-HOLD-BACK + CAN-PRELOAD-BEFORE, and every EXT-X-PART entry is a chunked partial the player can prefetch out of order — flagged with INDEPENDENT=YES when it can stand on its own.
The blocking-replay trace shows the regression in flight: the client had a delta cursor in _HLS_msn=1471, _HLS_part=4, and asked for a delta. The server returned 4,920 bytes — a full BLOCKING window — because the partial-segment emitter stalled on seg1470.6.m4s (the off-cycle 0.660s part). playlist.delta_size flips to warn, and once the cohort sees enough of those warnings, blocking_playlist.storm_ceiling flies.
The agent never reads the bytes itself — it probes what the playlist tells it the bytes would be, and then samples the cadence to look for the regression. The probes→fields map on the left pairs each verdict with one specific element in the playlist or in the delta response, so the line to grep for in a chunklist capture is right there.
A delta response is the new + last-played MEDIA-SEQUENCE worth of PARTs — typically a single new EXT-X-PART line plus the trailing EXT-X-PRELOAD-HINT. A BLOCKING response is the entire served window back to the playback head. The probe counts bytes: > 1KB on a delta cursor with part_drift_ms> 0 means the server failed to honor CAN-BLOCK-RELOAD.
Six ways LL-HLS falls off the live edge
Each row: symptom the agent reports → underlying cause → a fix that holds under the next probe cycle. The probe names are what you'd grep for in the agents feed.
First chunk TTFB hovers at 600ms+ versus a 280ms baseline; chunk-completion p95 climbs into the 1.5–2.0s range. Viewers see a working player that suddenly pauses mid-PART, then jumps forward when the chunk drops land. Live edge creeps from the target 2–3s glass into an 8–10s glass.
The ingest / packaging pipeline publishes the partial segments with Transfer-Encoding: chunked but the first chunk stalls behind a CDN edge read-through or an origin hop. The player can fetch the next PART URI but the body bytes arrive late; manufacturing_server.cadence_drift compounds because the chained partial-segment emitter is waiting on the same stalled socket.
Pin the ingest path to a hold-connection transport, push the manufacturing server onto a dedicated egress so a CDN edge cold-pull cannot stall partial chains, and turn on preemptive PART replay when ingest.chunked_transfer_stall trips a chunk-completion p95 > 1.2s for more than one cadence. Streamwake auto-rebalances the egress preference away from the saturated edge within ~30s of the verdict.
Across the run the agent reports 2 of 8 INDEPENDENT PARTs terminated mid-stream. The client decided the prefetch window was unhealthy, dropped the prefetch, and the player visibly downshifted to a safer buffering posture. Several cohort members see a small rebuffer right at the prefetch drop.
preload.can_preload_hint is still passing at the playlist level, but the BYTE STREAM of the PART never completes. Either ingest.chunked_transfer_stall has a dropped chunk that the client cannot recover from, or the manufacturing server never finished publishing the late-stage parts in the prefetch window and the server returns stale bytes the player will not trust.
Emit the full PART only once manufacturing has acknowledged publish; mark any PART that is mid-publish with the INDEPENDENT=NO attribute and have the player skip the prefetch on those parts. part.prefetch_termination should fall to zero once the publish path is consistently committing ahead of the prefetch window.
PART-DURATION hits 0.660s or 0.990s against a declared PART-TARGET=0.330s. Every off-cycle PART needs a partial append mid-GOP, and the player rolls forward to the next IDR — burning ~250ms of buffer each cycle. Buffer math goes back-of-envelope wrong; ABR descends the ladder.
The encoder is emitting 2.5s GOPs under load instead of the 2.0s GOP locked against PART-TARGET=0.33. The packager splits on inference and the parts occasionally start mid-frame, which is exactly the signature encoder.cadence_part_compatible flags. The packager cannot honor PART-TARGET without a stable GOP underneath.
Lock the encoder cadence so GOP duration divides cleanly into TARGETDURATION (PART-TARGET=0.33 with GOP=2.00 → 6 parts per GOP). The agent replays the trailing partials the moment cadence crosses the ±0.05s threshold; gop.boundary_aligned flips to pass when every PART starts with an IDR.
agent timeline shows manufacturing_server.cadence_drift fail with manufactured_parts_per_sec_p95 = 1.7 against an expected 3.03 (one part every PART-TARGET). The playlist still lines up; the player has not panicked yet — but the publish path is consistently ~430ms behind the encoder.
The LL-HLS packaging server (the manufacturing server) is responsible for chopping the encoded chunks into PARTs and emitting them to the playlist + CDN. Under load the packaging worker pool saturates, the partial-segment publisher stops honoring its SLA, and the playlist cursor stays ahead of the published media. The playlist.delta_updates probe stays green because the playlist is being produced — but the BYTES it references trail by hundreds of milliseconds.
Scale the manufacturing worker pool to keep manufactured_parts_per_sec_p95 inside ±15% of expected_parts_per_sec, and prefetch the next GOP from the encoder into packaging memory any time cadence drift exceeds ±10%. manufacturing_server.cadence_drift should flip back to pass on the next cadence window once the pool recovers.
playlist.delta_updates stays green (deltas honored) but playlist.delta_size flips to warn. The agent timeline p95 for delta bytes lingers at 4,920 on a single probe cycle — most deltas are still ~80 bytes but the over-fetch ratio is climbing. Bandwidth on the live edge roughly doubles inside that window.
A BLOCKING playlist reply slipped into the deltas — usually because ingest.chunked_transfer_stall dropped a partial chunk, the server failed to honor the cursor, or the manufacturing-server stalled and the playlist writer returned the full window the player did not need. The player treats the over-fetched delta as a full reload; CDN cache fill rules no longer apply.
Throttle any delta reply that returns over the delta_size_threshold_b to a single orchestrating client and emit a small delta only, and pair playlist.delta_size with manufacturing_server.cadence_drift so the over-fetch root cause is visible. Streamwake auto-throttles when delta bytes p95 exceeds the threshold for more than two cadences.
blocking_playlist.storm_ceiling fails. Over a 60s window the player cohort issued 18 full BLOCKING playlist GETs against a 4-per-minute ceiling. The CDN edge misses stack up; egress saturates; the cohort is now paying 4–5KB per playlist pull when ~80B was sufficient.
Once playlist.delta_size has been failing for several cycles the cohort loses confidence in the delta cursor — every client correlates the bad delta with a refresh and issues a full BLOCKING reload. The failures feed on each other: more BLOCKING reloads produce more cache misses, cache misses inflate the BLOCKING budget, and the storm takes off.
Inject a server-side cap on BLOCKING reloads per cohort (force re-establish delta cursor on a 410 Gone-style stronger signal), and pair the cap with playlist.delta_size so a delta regression becomes a single coordinated rebalance instead of a storm. Streamwake routes the cohort to a healthier egress and forces a delta cursor replay within a single cadence.
Heuristics: how the agent loop classifies the incident
The probe families above produce verdicts. Three rules in the agentic-ops layer turn a stream of verdicts into an incident classification — and pair the verdict with the smallest-safe auto-rebalance — without a human reading the timeline.
- Rule: first-chunk TTFB p95 across the last N probes vs the phased baseline for the same cohort + export region; chunk-completion p95 above 1.2s for more than one cadence.
- What it surfaces: an ingest / CDN edge read-through stall that drags the partial-segment emitter down with it. Latency creeps from 2–3s to 8–10s before any of the byte probes notice.
- Agent does next: auto-rebalance the egress preference away from the saturated edge within ~30s of the verdict, raise probe cadence for this stream, prepend to the alert feed, and tag an incident "ingest_chain_pressure" — actionable before any bitrate ladder starts slipping.
- Rule: ratio of consumed vs requested INDEPENDENT PARTs over the last N probes; any mid-stream termination that exceeds 5% of the prefetch window.
- What it surfaces: a manufacturing-server stall the playlist itself never flagged, because the playlist was still produced. The client is the only observability that catches it before the ear.
- Agent does next: meta-classify as "byte_stream_pressure", replay the trailing PARTs that were terminated, and notify the publish path to throttle — the player will converge on a healthy prefetch window within one cadence.
- Rule: blocking playlist requests per minute vs ceiling; double-or-more spikes after one delta_size warn cycle flip the verdict.
- What it surfaces: a cohort that has lost confidence in the delta cursor — the moment a few bad deltas are correlated with refresh bumps, every client issues a full reload.
- Agent does next: route the cohort to a healthier egress, force a delta cursor replay, and inject a single coordinated remediation that quenches the storm rather than absorbing more bandwidth — closing the loop that a passive monitoring pipeline cannot.
The failure-mode rows above trace back to the three heuristics: chunked-transfer ingest stalls and manufacturing-server cadence drift both surface as elevated first-chunk TTFB + PART termination ratios; partial-segment prefetch drops map onto the manufacturing-server chain dragging the prefetch window along with it; and playlist-delta size regressions funnel into a blocking-playlist storm the moment the cohort loses confidence in the cursor. A passive-monitor logs the same verdicts and waits for a rebuffer report to fire — by which point you're already paying the cohort-trust cost and writing the postmortem. The agent loop flips the polarity: classify the verdict under one of the three heuristics, rebalance by re-routing the egress preference away from the saturated edge or coordinating a replay of the trailing PARTs, then verify that the next probe cycle clears the cohort off the BLOCKING storm. Each heuristic closes a loop that a passive monitoring pipeline cannot.
Diagnose with Streamwake
Register the LL-HLS source against POST /api/v1/streams with protocol: HLS+LL, then read the agent timeline back through GET /api/v1/agents. The probe verdicts in the timeline are exactly the rows above.
The curl below registers an LL-HLS source URL and asks for a 30-second probe cadence. The HLS+LL protocol routing is what makes the agent run the ingest-chunked, partial-prefetch, preload-hint, GOP-alignment, manufacturing-server, delta-size, and blocking-storm probes on every rebuild.
The cookie is the same better-auth.session_token that gates every /api/v1/* call — see the auth guide for how to mint one.
Once the stream is registered, the agents endpoint returns the per-probe verdicts below. The order of checks mirrors the probe families in the section above — ingest.chunked_transfer_stall first, then prefetch / preload / GOP / manufacturing-server, with the delta + blocking probes last because those are the ones that close the loop with the cohort.
curl -X POST https://streamwake.polsia.io/api/v1/streams \
-H "content-type: application/json" \
-b "better-auth.session_token=<your-session-cookie>" \
-d '{
"sourceUrl": "https://cdn.example.com/live/event/llhls-variant.m3u8",
"protocol": "HLS+LL",
"probeIntervalSeconds": 30,
"agents": [
"ingest.chunked_transfer_stall",
"part.prefetch_termination",
"preload.can_preload_hint",
"gop.boundary_aligned",
"encoder.cadence_part_compatible",
"manufacturing_server.cadence_drift",
"playlist.delta_updates",
"playlist.delta_size",
"blocking_playlist.storm_ceiling"
]
}'curl https://streamwake.polsia.io/api/v1/agents?stream_id=<id> \
-b "better-auth.session_token=<your-session-cookie>"{
"stream_id": "cklivellhlsstorm712",
"source": "https://cdn.example.com/live/event/llhls-variant.m3u8",
"protocol": "HLS+LL",
"playlist_mode": "delta",
"checks": [
{
"probe": "ingest.chunked_transfer_stall",
"result": "warn",
"ttfb_ms_p95": 612,
"chunk_completion_ms_p95": 1820,
"detail": "Transfer-Encoding: chunked; first chunk TTFB p95 612ms vs 280ms baseline; manufacturing server stalled on seg1470.6.m4s (chunk drop)"
},
{
"probe": "part.prefetch_termination",
"result": "fail",
"terminated_parts": 2,
"prefetched_parts": 6,
"detail": "2 of 8 INDEPENDENT PARTs terminated mid-stream — client dropped the prefetch window after seg1470.6.m4s"
},
{
"probe": "preload.can_preload_hint",
"result": "pass",
"detail": "CAN-PRELOAD-BEFORE=YES in SERVER-CONTROL; EXT-X-PRELOAD-HINT TYPE=PART present for seg1472.0.m4s"
},
{
"probe": "gop.boundary_aligned",
"result": "warn",
"gop_period_s": 2.0,
"part_count_per_gop": "5–7",
"detail": "2 of 16 PARTs start mid-GOP — encoder cadence slipped to ~2.5s for two cycles; seg1470.6.m4s and seg1470.15.m4s are the off-cycle parts"
},
{
"probe": "encoder.cadence_part_compatible",
"result": "fail",
"detail": "encoder GOP slipped to ~2.5s for two cycles; PART-DURATION jitter to 0.660 and 0.990 — PART-INF PART-TARGET=0.330 cannot be honored"
},
{
"probe": "manufacturing_server.cadence_drift",
"result": "fail",
"manufactured_parts_per_sec_p50": 2.4,
"manufactured_parts_per_sec_p95": 1.7,
"expected_parts_per_sec": 3.03,
"detail": "partial-segment manufacturing cadence dropped to 1.7 parts/sec at p95; the publish path fell behind the encoder by ~430ms on the captured window"
},
{
"probe": "playlist.delta_updates",
"result": "pass",
"delta_responses": 31,
"delta_bytes_p50": 78,
"delta_bytes_p95": 212,
"detail": "32/32 deltas honored _HLS_msn + _HLS_part cursor"
},
{
"probe": "playlist.delta_size",
"result": "warn",
"delta_bytes_p95": 4920,
"delta_size_threshold_b": 1024,
"detail": "delta returned 4920B on 1/32 probes — chunked-transfer drop + manufacturing-server stall pushed the server into a BLOCKING-window delta reply"
},
{
"probe": "blocking_playlist.storm_ceiling",
"result": "fail",
"blocking_requests_per_min": 18,
"blocking_ceiling_per_min": 4,
"detail": "blocking playlist requests spiked to 18/min after the chunked-transfer stall; clients lost the delta cursor and fell back to full BLOCKING reloads"
},
{
"probe": "llhls.part_holdback_drift",
"result": "pass",
"detail": "PART-HOLD-BACK matches delta cursor; CAN-PRELOAD-BEFORE aligned"
}
]
}Read the HLS / MPEG-DASH pair
The low-latency arm of the protocol-by-protocol guide pair. The HLS guide covers the shared master / variant / tag-continuity probes; the LL-HLS optimization guide covers the part-cadence / delta-vs-blocking / preload / GOP-alignment tuning lane; the MPEG-DASH guide covers the DASH-shaped counter-parts.
Want Streamwake to catch this on its own?
Sign up, register an LL-HLS source with protocol: HLS+LL, and the same ingest-chunked / prefetch / preload / GOP / manufacturing-server / delta-size / blocking-storm probes that produced the timeline above run on every cadence — and surface in a Slack channel, a webhook, or the streams dashboard.
- Stream list reads from
GET /api/v1/streams; per-stream timeline fromGET /api/v1/agents. - Probe verdicts stream into the dashboard within one cadence interval — manual curl not required.
- Self-serve signup at /sign-up — no sales call required for the first stream.
- Pick a recent on-call incident — manifest stall, edge miss, player-side stall, or peer congestion.
- We replay it through the same reliability-agent probe cascade used on the postmortem above.
- You walk away with a written what-could-have-been-Automated readout, not a sales deck.