> For the complete documentation index, see [llms.txt](https://liuyang89116.gitbook.io/system-design/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://liuyang89116.gitbook.io/system-design/chapter-2/distributed_message_queue.md). # Design Distributed Message Queue ## Design Distributed Message Queue > Ref: [original video](https://www.youtube.com/watch?v=iJLL-KPqBpM) ## Problem Statement ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-31386dff16fd4ec6ca247c628659a3b960875c05%2F1.png?alt=media) ### synchronous communication ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-7de9a1ad4c3aad3a724d656229b6fa59bf443d5b%2F2.png?alt=media) * Pros: * easier and faster to implement * Cons: * harder to deal with consumer service failures ### asynchronous communication - add a queue ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-d7a30cad75d7ff6956f5abfaf3989981c32dfe6e%2F3.png?alt=media) *** ## Requirements ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-c770d284efe58644bbc46f60f66993bda5f2b761%2F4.png?alt=media) *** ## High-level Architecture ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-13184b6a344654924d3bac41b407b0d11fefee4a%2F5.png?alt=media) *** ### VIP and Load Balancer ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-bb4deee44774333907d18feb24cd594c3d37331f%2F6.png?alt=media) > What happens if LB goes down? * LB uses primary and secondry nodes. * The primary node accepts connections and serves requests while the secondary node monitors the primary. If primary node doesn't respond, the secondary node takes over. > What if traffic increases and reaches LB's limit? * As for scalablity concerns, a concept fo multiple VIPs can be utilized. * By spreading load balancers across several data centers, we improve both avalibility and performance. ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-1cdaebc9cbfc7f8b119be90a572806704594e6ec%2F7.png?alt=media) *** ### FrontEnd Service ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-24b0b43e0781eba61bdd6de2983b24c3cb192bfb%2F8.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-5cf888cb4027bea084d0fba234365d8484c76089%2F9.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-11d6fc53b9ee268bc572c8a7f0207e29f72330e9%2F10.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-b01bd395de94c41bf41b43213750295b0be88da1%2F11.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-af3fe10dd2c25e2985d4f5c0ae28413f49245af5%2F12.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-a2e2e3565e63da7af4e4077efd67262a059b5fda%2F13.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-07c375081b4a020bfc74cb8273a3cda98dcc956a%2F14.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-01240c69cfe3e09028c4af1c6c4b3ed80820989d%2F15.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-2c32d92a0aa6b7ddf837db363564d6dbd327ddcb%2F16.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-61d217da4319b8cca65dfb164460219f5d8a54df%2F17.png?alt=media) *** ### Metadata Service ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-4389da324042a0e6fd4f7d82c565e4d8b67e3e62%2F18.png?alt=media) *** ### Backend Service ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-571ec20cbbbd1cafb034b2a35c94b743add4d73e%2F19.png?alt=media) #### Option 1: Leader-Follower relationship ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-aa8257008536f847a26f46c807cc4e832e6f7b7f%2F20.png?alt=media) #### Option 2: Small cluster of independent hosts ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-dccf6e479e602f302cd828c0426765d5cbfd0d51%2F21.png?alt=media) ![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-2b3f2ed27b796ec3083a02cabb74c5b032781bae%2F22.png?alt=media) *** ## What else is important? ### Queue creation and deletion * API is a good option to control over queue configuration parameters. Delete queue is a little bit controversial as it may cause a lot of harm and must be executed with caution. ### Message deletion * One option is not to delete a message right after it was consumed. Messages can be deleted several days later by a job. This is used by Kafka. * We need to maintain some kind of an order for messages in the queue and keep track of offset, which is the position of a message within a queue. * Another option is used by Amazon SQS. Messages are also not deleted immediately, but marked as invisible. So other consumers may not get already retrieved message. * Consumer that retrieved the message, needs to then call delete message API to delete the messages from a backend host. * And if the message was not explicitly deleted by a consumer, message becomes visible and may be delivered and processed twice. ### Message replication * Synchronously replication: when backend host receives new message, it waits until data is replicated to other hosts. And only if replication is fully completed, successful response is returned to a producer. * higher durability but with a cost of higher latency for send message operation. * Asynchronous replication: response is returned back to a producer as soon as a message is stored on a single backend host. Message is later replicated to other hosts. * more performant, but not guarantee that message will survive backend host failure. ### Message delivery semantics * At most once: when messages may be lost but are never redelivered. * At least once: when messages are never lost but maybe redelivered. * Exactly once: when each message is delivered once and only once. ### Push vs. pull * Pull model: consumer constantly sends retrieve message requests and when new message is available in the queue, it is sent back to a consumer. * Push model: consumer is not constantly bombarding FrontEnd service with receive calls. Instead, consumer is notified as soon as new message arrives to the queue. * From producer side, pull is easy to implement compared to push. But from a consumer perspective, we need to do more work if we pull. ### FIFO * It's hard to maintain the order. Some system either not guarantee strict order or have limitations around throughput. ### Security * Encryption using SSL over HTTPS helps to protect messages in transit. * We may also encrypt messages while storing them on backend hosts. ### Monitoring * We need to monitor health of our distributed queue system and give customers ability to track state of their queues. * Each service we built has to emit metrics and write log data. *** ## Final look ### Is it scalable? * Yes. Every component is scalable. When load increases, we just add more load balancers, more FrontEnd hosts, more Metadata service cache shards, more backend clusters and hosts. ### Is it highly available? * Yes. No single point of failure. Each component is deployed across several data centers. ### Is it highly performant? * Yes. Each individual microservice needs to be fast. ### Is it durable? * Yes. We replicate data while storing and ensure messages are not lost during the transfer from a producer to a consumer.