HLS qualification checklist

Akamai uses the following test cases to qualify HLS ingest.

HLS qualification tests
Test Description For details, see
Basic streams test

Verify if the first line of the bit rate playlist files is #EXTM3U. Check if bit rate playlist files have the tags #EXT-X-TARGETDURATION, #EXT-X-MEDIA-SEQUENCE.

Verify that the number of streams received is the same as the number of streams expected.

 Bitrate and playlist
Bit rate playlist alignment. Verify if the tag EXT_X_MEDIA_SEQUENCE is consistent across playlists (optional). Bitrate and playlist
Bit rate playlist upload interval Verify that the bit rate playlist upload interval is almost equal to the segment upload interval.
Directory rollover Verify if directory rollover happens at consistent intervals.
ENDLIST tag when the stream stops Verify if the EXT-X-ENDLIST is present in the playlist when publishing stops. Marking playlists complete
Error recovery test (failover to a different entry point – DNS lookup test) Check if the transition to the secondary entry point is seamless. DNS lookups, upload retries, and timeouts
Error recovery test (streams reconnect to same entry point) Verify the ts segment IDs on reconnect.
HTTP request type Verify that the HTTP request type for the ts segments is either PUT or POST.
Long duration publish Verify playback and analyze the stream publish after 48 hours of publish.
Master playlist upload interval Verify that the master playlist file does not get uploaded too frequently.
Playback Verify playback on QuickTime, Safari, and iPad. Playback workflow (HLS)
Segment numbering Check that the segment numbering is not close to (2^32)-1.
Segment retry on 5xx error and verify the retry interval Trigger a 5xx response to the encoder by modifying the delivery-hostname for the stream-id.

Trigger a 5xx response by modifying the forward target host and verify encoder behavior.
Segment upload interval Verify if the segment upload interval is almost equal to the advertised values.
Session ID verification. Verify that the master playlist file does not have session IDs mentioned in the bit rate URLs.

Verify that the session IDs are same across all bit rates.
ts segments and bit rate URL Verify that the segment numbers are sequentially increasing.

Verify is response received is 200 by requesting random ts segments (first one, last one, and a middle one).

Verify that EXT-X-MEDIA-SEQUENCE corresponds to first segment in the bit rate playlist.

Verify if any of the #EXTINF has negative values in the bit rate m3u8 files.
User-Agent string Verify if the encoder is sending a valid User-Agent string.
Verify if the session ID and directory path is unique during republish. When an encoder stops the publish and starts another one, the path must be unique.