SyntaxPilot
Courses/System Design/Lesson 31
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

System Design · Lesson 31 of 40

Design Hotstar live streaming for IPL

queued

These are the notes for System Design episode 31, our fifth full design problem. The IPL final, last over, twelve needed off six balls, and tens of millions of people want to watch the same ball live, some on home fibre and some, like Ravi, on a local train whose 4G keeps dropping. Today you design live streaming for that moment: you'll write the requirements, discover that a live match has no file to send and cut it into segments with a playlist (HTTP Live Streaming, HLS), find that one origin would need 50.6 terabits a second and serve through CDN edges instead (100,000 requests, 50 origin fetches), stop the spinner in a tunnel with adaptive bitrate (6 stalls → 0), survive the crowd that arrives when a star batter walks out by pre-warming instead of waiting for autoscaling (20 million failed viewer-minutes → 0), and keep the video alive with a panic switch that turns off the extras. Every demo is plain Python and runs as-is; outputs are real (Python 3.13). The bitrates, server sizes and viewer curve are our own assumptions for the exercise; the real-world records come from the sources at the end.

Live streaming means sending video to viewers while it is still happening, a few seconds behind real life. Example: an IPL match on Hotstar, a YouTube live stream.

The problems

Problem What it teaches
The last over of the final Requirements: live, huge concurrency, every network, no spinner, spikes, video first
A live match has no file Segments and playlists; HLS (RFC 8216)
50 terabits a second from one place Origin vs CDN edge servers; request coalescing
Buffering in a tunnel Buffers, stalls, renditions, adaptive bitrate (ABR)
Millions join in minutes Reactive autoscaling vs pre-warming; headroom; game days
Extras crash the video Panic mode, graceful degradation, critical vs optional features
Put it all together Encoders → origin → CDN → players, plus pre-warmed services

Problem 0: the last over of the final (requirements)

The crowds are real. Hotstar reported 25.3 million concurrent viewers on 10 July 2019 (the previous record was 18.6 million), and TechCrunch reported that the 2023 World Cup final between India and Australia "topped 59 million concurrent viewers" on Disney+ Hotstar. Concurrent viewers means people watching at the same moment.

Functional requirements:

  • Stream the match live, only a few seconds behind the stadium.
  • Play on phones, TVs and laptops, on every kind of network.

Non-functional requirements:

  • Tens of millions of concurrent viewers.
  • No spinner: keep playing, even at a lower quality.
  • Survive spikes: a wicket, a star batter walking out, the last over.
  • Video first: everything else on the screen (recommendations, comments, ads rows) is optional.

A few words first:

  • Bitrate is how many bits of video are sent per second. Example: 1080p video at about 5 Mbit/s (megabits per second); 240p at about 0.4 Mbit/s (our demo's assumptions).
  • Bandwidth (network speed) is how many bits per second a connection can carry right now. Example: home fibre 50 Mbit/s; a weak train signal 1 Mbit/s.
  • 1080p, 720p, 480p, 240p are video qualities named by their height in pixels: higher means sharper and more bits.

Problem 1: a live match has no file (segments and playlists)

Real life: the live commentary notebook

A reporter at the stadium Live video
Can't post you the finished book: the match isn't over There's no complete file: the video keeps growing
Tears off a page every few balls and sends it The encoder cuts the stream into segments, a few seconds each
Pins a list on the door: "latest pages: 3, 4, 5" A playlist names the newest segments; the player re-reads it
  • An encoder is the software that turns the raw camera feed into compressed video files. Example: it takes the stadium feed and writes a 4-second file every 4 seconds.
  • A segment is one small video file covering a few seconds. Example: seg_5.ts (.ts is a common segment file type).
  • A playlist is a small text file listing the segments to play, in order.
  • HTTP Live Streaming (HLS) is the standard that defines this. Its RFC 8216 describes "a protocol for transferring unbounded streams of multimedia data": segments plus playlists, all fetched with ordinary HTTP.
SEG = 4                                   # seconds of video per segment
made = []                                 # segments the encoder has cut

def encoder_tick(second):                 # live camera feed comes in...
    if second % SEG == 0:
        made.append(f"seg_{second // SEG}.ts")   # ...cut every 4 s

def playlist(window=3):                   # the newest few segments
    lines = ["#EXTM3U", f"#EXT-X-TARGETDURATION:{SEG}",
             f"#EXT-X-MEDIA-SEQUENCE:{len(made) - window + 1}"]
    for name in made[-window:]:
        lines += [f"#EXTINF:{SEG}.0,", name]
    return "\n".join(lines)

for s in range(1, 21):                    # 20 seconds of the match
    encoder_tick(s)
print(playlist())
encoder_tick(24)                          # 4 seconds later
print("next fetch adds:", made[-1], "| oldest now:", made[-3])

Output:

#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:3
#EXTINF:4.0,
seg_3.ts
#EXTINF:4.0,
seg_4.ts
#EXTINF:4.0,
seg_5.ts
next fetch adds: seg_6.ts | oldest now: seg_4.ts

The playlist lines, which are the real HLS tags:

Line Meaning
#EXTM3U "This is an HLS playlist" (the file format is called M3U)
#EXT-X-TARGETDURATION:4 Segments are at most 4 seconds long
#EXT-X-MEDIA-SEQUENCE:3 The first segment listed is number 3
#EXTINF:4.0, The next segment is 4.0 seconds long
seg_3.ts The segment's address

The window slides: 4 seconds later the next fetch adds seg_6.ts and the oldest listed is seg_4.ts. The player re-downloads the playlist every few seconds and fetches any new segment.

The big win: segments and playlists are ordinary files over HTTP, and ordinary files can be cached.

Problem 2: 50 terabits a second from one place (CDN)

Real life: newspapers across India

A newspaper Live video
One printing press in Mumbai The origin server: where segments are made
One bundle to each city's depot CDN edge servers copy each segment once
The hawker delivers from the nearby depot Each viewer fetches from the nearest edge
  • The origin is the server (or cluster) that holds the original segments and playlists.
  • A CDN (content delivery network, episode 21) is a set of edge servers spread across many cities that keep copies of files close to users.
  • A cache hit means the edge already has the file; a cache miss means it has to fetch it from the origin first.
import random
random.seed(31)
EDGES = 50                                # edge servers near viewers
edge_cache = [set() for _ in range(EDGES)]
origin_fetches = 0

def get(segment, viewer):
    global origin_fetches
    edge = edge_cache[viewer % EDGES]     # nearest edge for this viewer
    if segment not in edge:               # miss: ask the origin once
        origin_fetches += 1
        edge.add(segment)
    return segment                        # hit: served from the edge

viewers = list(range(100_000))
random.shuffle(viewers)
for v in viewers:
    get("seg_5.ts", v)
print("viewer requests:", len(viewers), " origin fetches:", origin_fetches)

watching, mbps = 25_300_000, 2            # 25.3 million at 2 Mbit/s
print(f"from one origin directly: {watching * mbps / 1e6:.1f} Tbit/s")

Output:

viewer requests: 100000  origin fetches: 50
from one origin directly: 50.6 Tbit/s
  • 100,000 viewers ask for seg_5.ts, spread over 50 edges. Each edge misses once, fetches from the origin, then serves everyone else from its copy: 50 origin fetches.
  • If 25.3 million viewers pulled 2 Mbit/s each straight from one origin: 25.3 million × 2 Mbit/s = 50.6 Tbit/s (terabits per second). No single server or data-centre link comes close.

Request coalescing: in real life thousands of viewers miss on the same edge in the same instant, right when a new segment appears. A good edge sends one request to the origin and makes the others wait for that answer, instead of forwarding thousands. (Our demo runs one request at a time, so it never sees this race; the idea is the same as the cache stampede fix from episode 8.)

Problem 3: buffering in a tunnel (adaptive bitrate)

Real life: pouring through a funnel

Pouring water Streaming video
A bucket into a thin funnel overflows 1080p at 5 Mbit/s into a 1.5 Mbit/s connection: the buffer runs dry
Switch to a smaller cup Adaptive bitrate: pick a quality that fits right now
Cups of every size ready Renditions: every segment encoded in several qualities
  • The buffer is the few seconds of video the player has downloaded ahead of what you're watching.
  • A stall (rebuffering) is when the buffer runs out and the spinner appears.
  • A rendition (or variant stream) is the same video encoded at one particular quality and bitrate.
  • In HLS, a master playlist lists the renditions; each points to its own media playlist. RFC 8216: "Clients should switch between different Variant Streams to adapt to network conditions."
  • Adaptive bitrate (ABR) means the player chooses the rendition for each segment from the speed it's actually measuring.
RENDITIONS = {"1080p": 5.0, "720p": 2.8, "480p": 1.2, "240p": 0.4}  # Mbit/s
SEG = 4                                   # seconds per segment
network = [6, 6, 5, 1.5, 1.0, 1.0, 1.2, 1.5, 4, 6]   # Mbit/s, metro to tunnel

def watch(choose):
    buffer, stalls, picked = 8.0, 0, []   # start with 8 s buffered
    for bw in network:
        q = choose(bw)
        download = SEG * RENDITIONS[q] / bw       # seconds to fetch it
        buffer -= download                        # video plays meanwhile
        if buffer < 0:
            stalls += 1                           # spinner on screen!
            buffer = 0
        buffer += SEG
        picked.append(q)
    return stalls, picked

def fixed(bw):
    return "1080p"

def adaptive(bw):                         # highest that fits 80% of bw
    for q, rate in RENDITIONS.items():
        if rate <= 0.8 * bw:
            return q
    return "240p"

for name, rule in [("fixed 1080p", fixed), ("adaptive", adaptive)]:
    stalls, picked = watch(rule)
    print(f"{name:<12} stalls: {stalls}  {' '.join(picked)}")

Output:

fixed 1080p  stalls: 6  1080p 1080p 1080p 1080p 1080p 1080p 1080p 1080p 1080p 1080p
adaptive     stalls: 0  720p 720p 720p 480p 240p 240p 240p 480p 720p 720p
  • download = SEG * RENDITIONS[q] / bw: a 4-second 1080p segment is 4 × 5 = 20 megabits; at 1 Mbit/s it takes 20 seconds to download, while only 4 seconds of video were in hand.
  • Fixed 1080p: 6 stalls. Adaptive: 0 stalls: it drops to 480p and 240p in the tunnel and climbs back to 720p after.
  • Using only 80% of the measured speed leaves a safety margin, because measurements jump around.

A blurry ball beats a spinner. Viewers forgive lower quality far more than a frozen screen during the last over.

Problem 4: millions join in minutes (pre-warming)

A wicket falls, the star batter walks out, a notification goes out, and everyone opens the app. From Hotstar's AWS re:Invent 2019 talk (as reported by Classmethod): viewers grew at a "1M+ growth rate - per minute", while traditional autoscaling had a "90 seconds reaction time" and hit "Insufficient capacity errors".

  • Autoscaling means automatically adding servers when load rises (and removing them when it falls). Reactive autoscaling waits until it sees the load.
  • Pre-warming means starting the capacity you'll need before the event, based on a prediction.
  • Headroom is extra capacity on top of the prediction, in case the guess is low.
PER_SERVER = 20_000                       # viewers one server can handle
# viewers each minute: toss, first over, then the star walks out to bat
viewers = [2, 4, 7, 11, 16, 20, 21, 22, 22, 22]   # millions

def run(start_servers, reactive):
    servers, dropped = start_servers, 0
    for minute, m in enumerate(viewers):
        need = m * 1_000_000 // PER_SERVER
        if need > servers:
            dropped += (need - servers) * PER_SERVER   # errors, spinners
        if reactive:                      # scale on what we see NOW;
            servers = max(servers, need)  # new servers are ready only
    return dropped                        # from the NEXT minute

peak_guess = 22_000_000 * 1.2             # predicted peak + 20% headroom
print(f"reactive autoscaling: {run(100, True):,} viewer-minutes failed")
prewarmed = int(peak_guess // PER_SERVER)
print(f"pre-warmed {prewarmed:,} servers: {run(prewarmed, False):,} failed")

Output:

reactive autoscaling: 20,000,000 viewer-minutes failed
pre-warmed 1,320 servers: 0 failed
  • Reactive scaling adds servers for the load it sees now, but they're ready only the next minute. Every minute of growth, the newest viewers hit errors: 20 million viewer-minutes failed (a viewer-minute is one viewer for one minute).
  • Pre-warmed for the predicted peak (22 million) plus 20% headroom: 1,320 servers from the start, 0 failed.

Hotstar's engineering blog says it directly: "Given the time it takes to auto-scale, it's a useless strategy when you need deal with spikes", and "Our strategy is to estimate the peak concurrency, then scale up ahead of the event". They also ran game days, "simulations of a real IPL event, before it actually happened", and chaos tests that simulated failures.

Problem 5: extras crash the video (panic mode)

CAPACITY = 100                            # units of work per second
features = [  # (name, cost per viewer, critical?)
    ("video segments",  4, True),
    ("live score",      1, True),
    ("login check",     1, True),
    ("recommendations", 3, False),
    ("ads row",         2, False),
    ("comments",        2, False),
]

def serve(viewers, panic=False):
    on = [f for f in features if f[2] or not panic]
    load = viewers * sum(cost for _, cost, _ in on)
    status = "OK" if load <= CAPACITY else "OVERLOADED -> video fails too"
    names = ", ".join(name for name, _, _ in on)
    print(f"{viewers} viewers, panic={panic}: load {load} -> {status}")
    print("   serving:", names)

serve(7)                                  # a normal over
serve(15)                                 # the last over: everyone joins
serve(15, panic=True)                     # flip the panic switch

Output:

7 viewers, panic=False: load 91 -> OK
   serving: video segments, live score, login check, recommendations, ads row, comments
15 viewers, panic=False: load 195 -> OVERLOADED -> video fails too
   serving: video segments, live score, login check, recommendations, ads row, comments
15 viewers, panic=True: load 90 -> OK
   serving: video segments, live score, login check
  • Every feature costs work per viewer. With everything on, 15 viewers need 195 units against a capacity of 100, and because all features share the same servers, the video fails too.
  • With panic on, only the critical features run (video segments, live score, login): load 90, OK.

Graceful degradation means switching off less important parts so the most important part keeps working. Hotstar's talk listed: "Turn off recomendations or personalization to keep critical components running". Their blog describes panic switches that "indicate to our client applications that they should ease off momentarily", so apps poll less and skip optional calls.

The full design

 stadium cameras ──▶ encoders (4 renditions, 4 s segments) ──▶ origin (segments + playlists)
                                                                     │
                                         CDN edges near viewers ◀────┘
                                                │
   players: master playlist → pick rendition → fetch segments (ABR every segment)

 separate, pre-warmed services: login, subscriptions, live score · panic switches drop extras
  1. Cameras send the feed to encoders, which cut 4-second segments in several renditions and publish segments and playlists to the origin.
  2. CDN edges in many cities fetch each segment once (coalescing simultaneous misses) and serve viewers nearby.
  3. The player reads the master playlist, picks a rendition, fetches segments from the nearest edge, and switches quality every segment.
  4. Login, subscriptions and the live score are separate services, pre-warmed for the predicted peak plus headroom.
  5. Panic switches drop extras when load passes the plan; apps back off.
  6. Everything is load-tested and rehearsed with game days before the final.

Why is live a few seconds behind? The encoder must finish a segment before it can be published, the player keeps a few segments buffered for safety, and the files travel through the CDN. Shorter segments reduce the delay but mean more files and more requests.

The whole chain

  1. A live match has no file → segments + playlist (HLS).
  2. 50.6 Tbit/s from one origin → CDN edges (100,000 requests → 50 fetches).
  3. 6 stalls in a tunnel → adaptive bitrate.
  4. Reactive scaling too slow → pre-warm with headroom.
  5. Extras crash the video → panic mode.
  6. Encoders, origin, CDN, players, pre-warmed services, rehearsals.

Common mistakes (and the real outputs)

Mistake What you see Why Fix
Every viewer pulls from the origin from one origin directly: 50.6 Tbit/s One place can't send to everyone CDN edges cache each segment
Always send 1080p fixed 1080p stalls: 6 Weak networks can't refill the buffer Renditions + adaptive bitrate
Rely on autoscaling for the spike reactive autoscaling: 20,000,000 viewer-minutes failed New servers arrive after the crowd Pre-warm for the predicted peak + headroom
Pre-warm with no headroom pre-warmed 1,100 servers: 10,000,000 failed (peak 25M, not 22M) The prediction was low Add headroom; keep panic switches ready
All features on, always load 195 -> OVERLOADED -> video fails too Optional features steal capacity from video Panic mode: keep only the critical path
Every edge miss goes to the origin Origin overloaded at each new segment Thousands of misses at the same instant Request coalescing at the edge

Mini project: a tiny live-streaming system

An encoder, an origin, five CDN edges, 10,002 players with adaptive bitrate (one on fibre, one on a train through a tunnel, 10,000 on random networks), and a panic switch, in about 70 lines:

# Mini project: a tiny live-streaming system: an encoder with renditions,
# an origin, CDN edges with request coalescing, adaptive players, and a
# panic switch. One process, no real video: segment names stand in.
import random
random.seed(31)

RENDITIONS = {"720p": 2.8, "480p": 1.2, "240p": 0.4}   # Mbit/s
SEG = 4

class Origin:
    def __init__(self):
        self.segments, self.fetches = [], 0
    def encode(self, n):                  # cut segment n in every quality
        self.segments.append(n)
    def get(self, n, q):
        self.fetches += 1
        return f"seg_{n}_{q}.ts"
    def playlist(self, window=3):
        return self.segments[-window:]

class Edge:
    def __init__(self, origin):
        self.origin, self.cache = origin, {}
    def get(self, n, q):
        key = (n, q)
        if key not in self.cache:         # first viewer: one origin fetch;
            self.cache[key] = self.origin.get(n, q)   # the rest share it
        return self.cache[key]

class Player:
    def __init__(self, name, edge, network):
        self.name, self.edge, self.network = name, edge, network
        self.buffer, self.stalls, self.got = 8.0, 0, []
    def play(self, n, t):
        bw = self.network(t)
        q = next((q for q, r in RENDITIONS.items() if r <= 0.8 * bw), "240p")
        self.buffer -= SEG * RENDITIONS[q] / bw
        if self.buffer < 0:
            self.stalls, self.buffer = self.stalls + 1, 0
        self.buffer += SEG
        self.got.append(self.edge.get(n, q))

origin = Origin()
edges = [Edge(origin) for _ in range(5)]
players = [Player("fibre", edges[0], lambda t: 50)]
players += [Player("train", edges[1], lambda t: 1.0 if 3 <= t <= 5 else 6)]
players += [Player(f"fan{i}", random.choice(edges),
                   lambda t: random.choice([1.5, 3, 6, 10]))
            for i in range(10_000)]

PANIC = False
extras = ["recommendations", "ads row", "comments"]
for t in range(8):                        # 8 segments = 32 seconds live
    origin.encode(t)
    if t == 6:
        PANIC = True                      # load past plan: flip the switch
    for p in players:
        p.play(origin.playlist()[-1], t)

print("viewers:", len(players), " origin fetches:", origin.fetches)
for p in players[:2]:
    print(f"{p.name:<6} stalls: {p.stalls}  last: {p.got[-1]}",
          " qualities:", [g.split('_')[2][:-3] for g in p.got])
print("total stalls:", sum(p.stalls for p in players))
print("serving extras:", [] if PANIC else extras)

Output:

viewers: 10002  origin fetches: 83
fibre  stalls: 0  last: seg_7_720p.ts  qualities: ['720p', '720p', '720p', '720p', '720p', '720p', '720p', '720p']
train  stalls: 0  last: seg_7_720p.ts  qualities: ['720p', '720p', '720p', '240p', '240p', '240p', '720p', '720p']
total stalls: 0
serving extras: []
  • 10,002 viewers watched 8 segments, and the origin served only 83 files: at most one per segment, per quality, per edge.
  • The train passenger dropped to 240p in the tunnel (minutes 3 to 5) and back to 720p after, with 0 stalls.
  • At minute 6 the panic switch turned off the extras.

Try extending it: give some fans a network that drops to 0.2 Mbit/s for three segments and count the stalls, then make the starting buffer 12 seconds instead of 8.

Practice

  1. Homework: in the SEGMENTS demo, use 2-second segments. What changes in the playlist?
  2. Homework: in the ABR demo, make the tunnel even slower (the video suggests 0.5 Mbit/s; try 0.3 too). What does adaptive pick? Any stalls?
  3. Homework: in the SPIKE demo, use 0% headroom. Is it still safe? What if the real peak is 25 million?
  4. Homework: why is a live stream always a few seconds behind the TV broadcast?
  5. 40 million viewers, average 1.5 Mbit/s. What total bandwidth must the CDN deliver?
  6. Why does ABR use only 80% of the measured speed?
  7. Which features would you keep in panic mode for an IPL final, and which would you drop?
  8. Why are segments a few seconds long, not 0.1 s or 60 s?
Solutions and hints

1. Change SEG = 4 to SEG = 2. Output:

#EXTM3U
#EXT-X-TARGETDURATION:2
#EXT-X-MEDIA-SEQUENCE:8
#EXTINF:2.0,
seg_8.ts
#EXTINF:2.0,
seg_9.ts
#EXTINF:2.0,
seg_10.ts
next fetch adds: seg_12.ts | oldest now: seg_9.ts

Twice as many segments for the same 20 seconds, so the sequence numbers are higher and the target duration is 2. (The demo jumps straight from second 20 to 24, so the encoder only cut seg_12 and skipped seg_11; ticking every second would add both.) Shorter segments mean less delay but twice as many files and requests.

2. At 0.5 Mbit/s adaptive picks 240p (0.4 fits exactly into 80% of 0.5) with 0 stalls. Change the tunnel values to 0.3. Output:

fixed 1080p  stalls: 6  1080p 1080p 1080p 1080p 1080p 1080p 1080p 1080p 1080p 1080p
adaptive     stalls: 0  720p 720p 720p 480p 240p 240p 240p 480p 720p 720p

No rendition fits 80% of 0.3 Mbit/s, so adaptive falls back to the lowest, 240p. Each 240p segment now takes 5.3 s to download for 4 s of video, so the buffer shrinks, but the buffer built up before the tunnel lasts the three bad segments: 0 stalls. A longer tunnel would stall; real services add even lower renditions (or audio only) for this.

3. With * 1.0: pre-warmed 1,100 servers: 0 failed, but only because the guess was perfect. Change the viewer list's last four values to 23, 25, 25, 25 (the real peak is 25 million). Output:

reactive autoscaling: 23,000,000 viewer-minutes failed
pre-warmed 1,100 servers: 10,000,000 failed

With the 20% headroom kept, the same higher peak gives pre-warmed 1,320 servers: 0 failed. Headroom pays for a wrong guess.

4. The encoder must finish each segment before publishing it, the player keeps a couple of segments buffered, and files pass through the CDN. With 4-second segments and two in the buffer, you're already more than 8 seconds behind.

5. 40 million × 1.5 Mbit/s = 60 million Mbit/s = 60 Tbit/s, spread across many edges in many cities (no single place).

6. Measured speed jumps around from moment to moment. Leaving 20% spare means a small dip doesn't immediately drain the buffer.

7. Keep: video segments, playlists, login/subscription checks for people already watching, the live score. Drop first: recommendations, personalised rows, comments, ads rows that call extra services, "watch next" suggestions.

8. Very short segments mean huge numbers of tiny requests and files (more overhead per second of video). Very long ones mean a big delay before a segment can be published and slow quality switching. A few seconds balances delay against overhead.

Quick quiz

video segments · live score · login
recommendations · ads row · comments
Load: 195 of 100

Load is past the plan in the last over. What do you turn off?

  • A. Video segments
  • B. Login
  • C. Recommendations, ads row, comments
  • D. Nothing, just wait
Answer

C. Keep the critical path (video, score, login) and switch off the extras until load is back under capacity. Turning off video defeats the purpose; waiting lets the overload take the video down too (our demo: OVERLOADED -> video fails too).

Interview questions: "Design live streaming for a cricket final"

1. Clarify requirements. Live or on-demand? Acceptable delay? Devices and networks? Peak concurrency? Login/subscriptions? Then state: live within seconds, tens of millions concurrent, every network, no stalls, survive spikes, video first.

2. Estimate. "25 million viewers × 2 Mbit/s = 50 Tbit/s: only a CDN spread over many cities can deliver that. Spikes of a million+ new viewers a minute: autoscaling can't keep up, so pre-warm."

3. Ingest and encoding. Stadium feed → encoders → several renditions → segments of a few seconds → origin with media playlists and a master playlist (HLS; DASH is the other common standard).

4. Delivery. CDN edges cache segments; request coalescing; possibly several CDNs for capacity and failover.

5. Player. ABR per segment with a buffer; start at a modest quality and climb.

6. Control plane. Login, entitlement (is this user allowed to watch?), live score, ads: separate services, pre-warmed with headroom, protected by rate limits (episode 19) and panic switches.

7. Reliability. Redundant encoders and origins; game days and load tests before the match; graceful degradation runbooks.

Common follow-ups:

  • How do you reduce delay? (Shorter segments, low-latency HLS with partial segments, smaller player buffer: each trades delay for robustness.)
  • Why HTTP and not a custom protocol? (HTTP files work with every CDN, cache and device.)
  • What breaks first at the spike? (Usually the non-video paths: login, home page APIs, recommendations, not the CDN.)
  • How do you decide the pre-warm level? (Past matches, the teams playing, the tournament stage, plus headroom; ladders of capacity tied to expected concurrency.)
  • How would you handle the origin failing mid-match? (A standby origin and encoders in another zone, with playlists switched over.)

Glossary

Term Meaning
Live streaming Sending video while it is happening, a few seconds behind
Concurrent viewers People watching at the same moment
Bitrate Bits of video sent per second, e.g. 2.8 Mbit/s for 720p in our demo
Bandwidth How many bits per second a connection can carry right now
Mbit/s, Tbit/s Megabits (a million bits) and terabits (a million million bits) per second
1080p / 720p / 480p / 240p Video qualities by height in pixels
Encoder Software that compresses the raw camera feed into video files
Segment A small video file of a few seconds, e.g. seg_5.ts
Playlist (media playlist) A text file listing the segments to play, in order
Master playlist A playlist listing the renditions (variant streams)
HLS HTTP Live Streaming, the segments-and-playlists standard (RFC 8216)
Rendition / variant stream The same video at one quality and bitrate
Origin The server holding the original segments and playlists
CDN / edge server Servers in many cities keeping copies of files near users (episode 21)
Cache hit / miss The edge has the file / must fetch it from the origin first
Request coalescing Many simultaneous misses for one file become one origin request
Buffer Video downloaded ahead of what's being watched
Stall / rebuffering The buffer runs out and playback stops (the spinner)
Adaptive bitrate (ABR) The player picks the rendition per segment from the measured speed
Autoscaling (reactive) Automatically adding servers after load rises
Pre-warming Starting the needed capacity before the event
Headroom Extra capacity on top of the predicted peak
Viewer-minute One viewer for one minute; a unit for counting failures
Game day A rehearsal that simulates a real event before it happens
Chaos testing Deliberately causing failures to check the system survives them
Graceful degradation Turning off less important parts so the core keeps working
Panic switch / panic mode A switch that drops optional features and tells apps to back off
Critical path The features the product can't work without (here: video, score, login)

Summary

  • A live match has no file: the encoder cuts it into segments of a few seconds, listed in a sliding playlist (HLS, RFC 8216), fetched over plain HTTP.
  • One origin would need 50.6 Tbit/s for 25.3 million viewers at 2 Mbit/s; CDN edges fetch each segment once (100,000 requests → 50 origin fetches) and coalesce simultaneous misses.
  • Every segment comes in several renditions; adaptive bitrate picks one per segment from the measured speed (6 stalls → 0 in the tunnel).
  • Spikes grow faster than autoscaling reacts (Hotstar: 1M+ viewers a minute vs ~90 s reaction): pre-warm for the predicted peak plus headroom (20,000,000 → 0 failed viewer-minutes), and rehearse with game days.
  • Panic mode drops recommendations, ads rows and comments so video, score and login keep running.

Next episode: you tap Book, and within seconds a nearby driver accepts. How does the app find the closest of thousands of moving cars? Design Uber and Ola ride matching.

Sources: RFC 8216, "HTTP Live Streaming" (rfc-editor.org): "a protocol for transferring unbounded streams of multimedia data"; Media Segments, Media Playlists and Master Playlists; "Clients should switch between different Variant Streams to adapt to network conditions". Classmethod, report of Hotstar's AWS re:Invent 2019 session CMY302: 25.3 million concurrent viewers on 10 July 2019 (previous record 18.6 million); "Insufficient capacity errors", "90 seconds reaction time", "1M+ growth rate - per minute"; "Pre-warm infra before match time"; "Turn off recomendations or personalization to keep critical components running". Hotstar engineering blog, "Scaling Is Not An Accident": "Given the time it takes to auto-scale, it's a useless strategy when you need deal with spikes"; "Our strategy is to estimate the peak concurrency, then scale up ahead of the event"; panic switches; game days and chaos testing. TechCrunch, 19 Nov 2023: the India-Australia World Cup final on Disney+ Hotstar "topped 59 million concurrent viewers". All demo sizes are our own stated assumptions.