10: Publish-Subscriber Pattern

Problem Statements:

Following are some of the challenges or issues raised in a distributed system

1. In traditional communication models, components are tightly coupled, meaning that a change in one component can impact others directly connected to it. This results in a lack of flexibility and makes the system more challenging to maintain and scale.

2. Systems with a direct point-to-point communication model can face scalability challenges. As the number of components increases, the complexity of managing connections between all entities becomes a bottleneck.

3. In some systems, components may need to communicate with each other based on specific events or updates. Traditional request-response models may not be suitable for scenarios where components need to be informed asynchronously.

4. Tight coupling between components can lead to challenges when modifying or updating one part of the system, as it may impact other interconnected components.

5. Some systems require an event-driven architecture where components react to specific events or changes in the system.

The Pub-Sub pattern addresses these challenges by providing a flexible, scalable, and decoupled communication model, making it suitable for various distributed system scenarios.

Publish-Subscriber (Pub-Sub) architecture is a messaging pattern where message senders (publishers) and message receivers (subscribers) are decoupled. Publishers send messages to a central message broker, and subscribers express interest in receiving messages on specific topics.

When a new message is published on a topic, the broker delivers it to all interested subscribers.

Components:

1. Publisher:

   • Entities that generate and send messages to the message broker.

   • Functionality: Publishes messages on specific topics without knowing who the subscribers are.

2. Subscriber:

   • Entities that express interest in receiving messages on specific topics.

   • Functionality: Subscribes to one or more topics and receives messages from the message broker.

3. Message Broker (Pub-Sub Engine):

   • A centralized system that receives messages from publishers and distributes them to interested subscribers.

   • Functionality:

     • Manages topics and subscriptions.

     • Forwards published messages to all relevant subscribers.

     • Ensures decoupling between publishers and subscribers.

Real-Time Use Case: Stock Market Data Streaming

Scenario: In a stock market data streaming application, real-time updates on stock prices need to be disseminated to various entities, such as traders, analysts, and automated trading systems.

Components in the Use Case:

1. Publishers (Stock Exchanges):

   • Role: Each stock exchange acts as a publisher.

   • Functionality: Publishes real-time stock price updates for various stocks.

2. Message Broker (Pub-Sub Engine):

   • Role: Centralized message broker managing the stock market data.

   • Functionality:

     • Receives real-time stock price updates from different stock exchanges.

     • Manages topics for each stock and handles subscriptions from various subscribers.

3. Subscribers:

   • Role: Traders, Analysts, Automated Trading Systems, News Platforms.

   • Functionality: Subscribe to receive real-time updates for specific stocks or market indices.

Workflow:

1. Publishing:

   • Stock exchanges continuously publish real-time stock price updates to the message broker.

   • For example, the New York Stock Exchange publishes updates for Apple Inc. stock on the "AAPL" topic.

2. Subscription:

   • Traders, analysts, and other entities subscribe to topics of interest (e.g., subscribing to "AAPL" for Apple Inc. stock updates).

3. Distribution:

   • The message broker forwards real-time stock price updates to all subscribers interested in the corresponding topics.

   • Subscribers receive timely updates on stock prices without directly interacting with stock exchanges.

Benefits:

• Scalability: The architecture scales well and the number of publishers and subscribers can increase independently.

• Decoupling: Publishers and subscribers operate independently, allowing for flexible and modular system components.

• Real-Time Updates: Enables real-time distribution of information, crucial in time-sensitive scenarios like stock trading.

Challenges:

• Reliability: The message broker must ensure reliable message delivery to subscribers.

• Scalability Challenges: Handling a large number of subscribers and publishers may pose scalability challenges for the message broker.

Conclusion:

The Pub-Sub architecture is well-suited for scenarios where real-time updates need to be disseminated to multiple entities with varying interests, such as stock market data streaming. The decoupling of publishers and subscribers provides flexibility and scalability in handling diverse data dissemination requirements.