Design live commenting
Design live commenting
Ref: Infoq - Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Ref: 1p3a 讨论
Requirements
Functional requirements
user can leave comment, real-time
others can see comments/posts in real-time
only focus on comments
Non-functional requirements
high available
low latency
Traffic Estimates
100 M posts, any posts can receive comments
our system should handle 100k/s comments delivery
High-level Design
Single host pattern
Detailed Design
scale challenge 1: hard to manage multiple connections
Linkedin has built AKKA, which has multiple Actors
each Actor is responsible for communicating via one connection
scale challenge 2: watch multiple videos and comment
Pub/Sub pattern
viewers subscribe to live videos
each host maintain a in-memory subscription table
publish using subscription
scale challenge 3: 10 K Concurrent viewers
Add a Real-time Dispatcher Service
Subscribe Process
Publish(Fan-out) Process
scale challenge 4: 100 k /s
Use a key-value store for subscription (e.g. Redis)
Subscribe Process
user publish comments via dispatcher service
dispatcher service looks up the nodes for those videos
dispatch comments to frontend nodes
nodes fanout to subscribed users
scale challenge 5: 100 messages/s, 10 K viewers, distribution of 1M messages/s
Subscription flow
user first subscribe the video in the frontend
in the frontend, it maintains a subscription table, noting down the
<topic-connectionId>entryfrontend route subscribe requests to dispatcher service
dispatcher service then register
<topic-node>entry in Redis
Publish flow
Last updated
