System Design
  • Introduction
  • Glossary of System Design
    • System Design Basics
    • Key Characteristics of Distributed Systems
    • Scalability - Harvard lecture
      • Scalability for Dummies - Part 1: Clones
      • Scalability for Dummies - Part 2: Database
      • Scalability for Dummies - Part 3: Cache
      • Scalability for Dummies - Part 4: Asynchronism
    • Trade-off
      • CAP Theorem
      • Performance vs scalability
      • Latency vs throughput
      • Availability vs consistency
    • Load Balancing
      • Load balancer
    • Proxies
      • Reverse proxy
    • Cache
      • Caching
    • Asynchronism
    • Processing guarantee in Kafka
    • Database
      • Relational database management system (RDBMS)
      • Redundancy and Replication
      • Data Partitioning
      • Indexes
      • NoSQL
      • SQL vs. NoSQL
      • Consistent Hashing
    • Application layer
    • DNS
    • CDN
    • Communication
      • Long-Polling vs WebSockets vs Server-Sent Events
    • Security
    • Lambda Architecture
  • OOD design
    • Concepts
      • Object-Oriented Basics
      • OO Analysis and Design
      • What is UML?
      • Use Case Diagrams
    • Design a parking lot
  • System Design Cases
    • Overview
    • Design a system that scales to millions of users on AWS
    • Designing a URL Shortening service like TinyURL
      • Design Unique ID Generator
      • Designing Pastebin
      • Design Pastebin.com (or Bit.ly)
    • Design notification system (scott)
      • Designing notification service
    • Designing Chat System
      • Designing Slack
      • Designing Facebook Messenger
    • Design Top K System
    • Designing Instagram
    • Design a newsfeed system
      • Designing Facebook’s Newsfeed
      • Design the data structures for a social network
    • Designing Twitter
      • Design the Twitter timeline and search
      • Designing Twitter Search
    • Design Youtube - Scott
      • Design live commenting
      • Designing Youtube or Netflix
    • Designing a Web Crawler
      • Designing a distributed job scheduler
      • Designing a Web Crawler/Archive (scott)
      • Design a web crawler
    • Designing Dropbox
    • Design Google Doc
    • Design Metrics Aggregation System
      • Design Ads Logging System
    • Design Instacart
    • Design a payment system
      • Airbnb - Avoiding Double Payments in a Distributed Payments System
    • Design Distributed Message Queue
      • Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go
    • Design Distributed Cache
      • Design a key-value cache to save the results of the most recent web server queries
    • Design a scalable file distribution system
    • Design Amazon's sales ranking by category feature
    • Design Mint.com
    • Design Autocomplete System
      • Designing Typeahead Suggestion
    • Designing an API Rate Limiter
      • Designing Rate Limiter
    • Design Google Map
      • Designing Yelp or Nearby Friends
      • Designing Uber backend
    • Designing Ticketmaster
      • Design 12306 - Scott
    • Design AirBnB or a Hotel Booking System
  • Paper Reading
    • MapReduce
  • Other Questions
    • What happened after you input the url in the browser?
Powered by GitBook
On this page
  • Design live commenting
  • Requirements
  • Functional requirements
  • Non-functional requirements
  • Traffic Estimates
  • High-level Design
  • Single host pattern
  • Detailed Design
  • scale challenge 1: hard to manage multiple connections
  • scale challenge 2: watch multiple videos and comment
  • scale challenge 3: 10 K Concurrent viewers
  • scale challenge 4: 100 k /s
  • scale challenge 5: 100 messages/s, 10 K viewers, distribution of 1M messages/s

Was this helpful?

  1. System Design Cases
  2. Design Youtube - Scott

Design live commenting

PreviousDesign Youtube - ScottNextDesigning Youtube or Netflix

Last updated 3 years ago

Was this helpful?

Design live commenting

Ref:

Ref:

Ref:

Ref:

Requirements

Functional requirements

  • user can leave comment, real-time

  • others can see comments/posts in real-time

  • only focus on comments

Non-functional requirements

  • high available

  • low latency

Traffic Estimates

  • 100 M posts, any posts can receive comments

  • our system should handle 100k/s comments delivery

High-level Design

Single host pattern

Detailed Design

scale challenge 1: hard to manage multiple connections

  • Linkedin has built AKKA, which has multiple Actors

  • each Actor is responsible for communicating via one connection

scale challenge 2: watch multiple videos and comment

Pub/Sub pattern

  • viewers subscribe to live videos

  • each host maintain a in-memory subscription table

  • publish using subscription

scale challenge 3: 10 K Concurrent viewers

Add a Real-time Dispatcher Service

Subscribe Process

Publish(Fan-out) Process

scale challenge 4: 100 k /s

Use a key-value store for subscription (e.g. Redis)

Subscribe Process

  • user publish comments via dispatcher service

  • dispatcher service looks up the nodes for those videos

  • dispatch comments to frontend nodes

  • nodes fanout to subscribed users

scale challenge 5: 100 messages/s, 10 K viewers, distribution of 1M messages/s

Subscription flow

  • user first subscribe the video in the frontend

  • in the frontend, it maintains a subscription table, noting down the <topic-connectionId> entry

  • frontend route subscribe requests to dispatcher service

  • dispatcher service then register <topic-node> entry in Redis

Publish flow

Infoq - Streaming a Million Likes/Second: Real-Time Interactions on Live Video
FB - Building Real Time Infrastructure at Facebook
FB - Live Commenting: Behind the Scenes
1p3a 讨论