WhatsApp / Messaging System

What is a Real-Time Messaging System?

A real-time messaging system enables users to exchange instant text, media, and document messages. Unlike standard HTTP request-response systems, messaging requires a persistent, bi-directional connection between the client and server to push updates instantly.

Real-World Applications

• WhatsApp, Facebook Messenger, Telegram, and Signal.

Core API Contracts (WebSocket Frames)

WebSocket communication uses thin JSON frames instead of standard HTTP request wrappers.

1. Inbound Send Message

{
  "event": "send_message",
  "data": {
    "to": "usr_9981",
    "conversationId": "conv_abc123",
    "content": "Are we meeting today?",
    "type": "text"
  }
}

2. Outbound Message Delivery Frame

{
  "event": "message_delivered",
  "data": {
    "messageId": "snowflake_12893821",
    "conversationId": "conv_abc123",
    "status": "delivered",
    "timestamp": 1782051600
  }
}

Functional Requirements

• 1-on-1 Chats: Real-time message exchange between two online users.
• Group Chats: Shared chat rooms supporting up to 500 members with concurrent typing and distribution.
• Delivery Status Tracking: Support for the complete messaging state machine: Sent (✓), Delivered (✓✓), and Read (✓✓ in blue).
• Online Presence: Track if a user is currently online and when they were "last seen."
• Offline Queuing: If a recipient is offline, queue the message and send a push notification. Deliver the message immediately when they reconnect.
• Media Support: Support sending images, videos, and voice notes.

Non-Functional Requirements

• Ultra-low Latency: End-to-end delivery of a message under 100ms when both users are online.
• Durability: Messages must not be lost. Once the server returns a "Sent" acknowledgment, the message must be safely stored.
• High Concurrency: Support 50 Million concurrent active WebSocket connections.
• Security: Discuss how to support End-to-End Encryption (E2EE) where the server is a blind forwarder.

Capacity Estimates (Back-of-the-Envelope Math)

1. Message Volume:
   • 1 Billion Daily Active Users (DAU).
   • Each user sends 100 messages/day.
   • Total messages per day: $100 ext{ Billion}$.
   • Messages/second: $100 ext{B} div 86,400 ext{s} approx 1.15 ext{ Million messages/second}$.
2. WebSocket Server Connections:
   • 50 Million concurrent connections.
   • Each WebSocket connection on a Linux server takes $approx 10 ext{ KB}$ of memory.
   • Memory needed: $50 ext{M} imes 10 ext{ KB} = 500 ext{ Gigabytes}$ of RAM across all Gateway nodes.
3. Message Storage (Cassandra):
   • Average message text size $approx$ 100 bytes + metadata (100 bytes) = 200 bytes per message.
   • Storage per day: $100 ext{B msgs} imes 200 ext{ bytes} = 20 ext{ Terabytes}$ per day.
   • Storage over 5 years (with backup): $approx 36.5 ext{ Petabytes}$ of data.

How to Approach This Problem

When designing a messaging app, follow this step-by-step mental roadmap:

Step 1: Characterize the System

Recognize that this is a high-concurrency, bi-directional stateful system. Unlike stateless apps, the server must keep a persistent connection open to push notifications.

Step 2: Establish the Client-Server Protocol

Determine the transport layer:

• Why not HTTP polling? (Too slow, high network overhead).
• Why not SSE? (Server-Sent Events is unidirectional; we need client-to-server speed as well).
• Use WebSockets (TCP-based, low-overhead, full-duplex).

Step 3: Handle Connection State

Address how a server knows where a client is:

• Maintain a Connection Routing Cache (Redis) that maps user_id to the IP address of the WebSocket gateway server they are connected to.

Step 4: Map Message Flow with Acknowledgments (ACKs)

Draft the message transfer sequence:

• Client A $ ightarrow$ Gateway Server A $ ightarrow$ Kafka $ ightarrow$ Gateway Server B $ ightarrow$ Client B.
• Ensure you design the explicit client-server ACK handshake to update delivery status.

Core Design Intuition

Real-time messaging requires shifting from a passive server model to an active server model that pushes data.

Stateful Connection Routing Pattern

Key Insights

1. Decouple Ingestion from Delivery: Writing messages to database storage synchronously halts real-time dispatch. Send messages through Kafka for asynchronous writing, while forwarding messages directly to the gateway servers in memory.
2. Ephemeral vs Persistent Store: Track presence and gateway mapping in Redis (in-memory, highly dynamic). Save chat history in Cassandra (NoSQL, optimized for rapid sequential writes and time-ordered lookups).
3. E2EE Support: To implement end-to-end encryption, the server must not store decrypted text. The client encrypts the message using Bob's public key (Signal Protocol) before sending. The server only sees and stores encrypted blobs.

System Architecture & Data Flow

Core Components

1. WebSocket Gateway Servers: Light stateful nodes that maintain persistent TCP connections with clients.
2. Session / Routing Table: Redis instance storing active mapping: user_id -> gateway_ip.
3. Chat Microservice: Coordinates API logic, group creation, and authorization.
4. Kafka Clusters: Ingests sent messages, presence logs, and status receipts.
5. Presence Service: Track online status using periodic heartbeats.
6. NoSQL Database (Cassandra): Stores long-term chat history.

Cassandra Message Storage Schema

PRIMARY KEY ((conversation_id), message_id) WITH CLUSTERING ORDER BY (message_id DESC)

• We partition by conversation_id so all messages in a chat are located on the same physical database node.
• We cluster by message_id (descending) to allow fast pagination of the newest messages in a chat.

Execution Flow: Sending 1-on-1 Message

1. Alice sends message to Bob over WebSocket to Gateway A.
2. Gateway A generates a Snowflake ID and publishes the message to the Kafka topic messages-ingest.
3. An consumer group writes the message to Cassandra.
4. Gateway A checks Redis for Bob's location:
   • Online: Redis returns Gateway B. Gateway A publishes the message to a channel matching Gateway B. Gateway B receives the message and pushes it to Bob's device.
   • Offline: Redis returns nothing. Gateway A triggers the Notification service (FCM/APNs) to push a mobile notification.

Advanced Scaling & Engineering Edge Cases

1. The Delivery Status Machine

To ensure reliable delivery status indicators:

• Sent (✓): When the WebSocket Gateway receives a message from Alice and writes it to Kafka, it returns a send_ack containing the message_id. Alice's client shows a single grey check.
• Delivered (✓✓): When Bob's client receives the message via WebSocket, it sends a delivery_ack back to its gateway. This gateway publishes it to Kafka, which updates the Cassandra row and notifies Alice's gateway to send a frame to Alice. Alice's client displays two grey checks.
• Read (✓✓ Blue): Triggered when Bob opens the chat window. Bob's app sends a read_ack frame, replicating the delivery flow to display the blue checks.

2. Presence Tracking & Heartbeats

Presence systems must scale to millions of updates per second:

• Heartbeat Protocol: Clients send a thin heartbeat packet every 5 seconds to the Presence service.
• Presence Storage: Store in Redis as a hash map: presence:{user_id} -> {status: "online", last_active: timestamp}.
• Offline Mark: If the server receives no heartbeat for 15 seconds, it marks the user as offline.
• Presence Broadcast (Push): When Bob opens a chat with Alice, he subscribes to Alice's presence updates. The server publishes Alice's presence changes to Bob's connection.

3. Scaling Group Chats

• Small Groups (< 100 members): When a message is sent to a group, the server fetches the member list and delivers it to each online member's gateway on the fly.
• Large Groups (500+ members): Do not perform direct real-time loop fan-out inside the WebSocket loop. Use dedicated consumer pools to partition the fan-out task. Store the message once in Cassandra, and write references to each active member's queue.

4. Handling Network Fluctuations & Reconnects

• When a mobile device switches from Wi-Fi to cellular data, the socket drops.
• The client maintains an offline queue. Upon reconnect, it connects to a new gateway, registers its presence, and fetches missed messages using the highest message_id it currently holds.

Trade-offs & Architectural Decisions

Succeeding in the Interview

What Interviewers Look For

1. Deep Understanding of Stateful Gateways: Explain how gateway servers coordinate, how connection tables are tracked in Redis, and what happens when a gateway server crashes.
2. Reliability Mechanics: Clearly describe the ACK protocol. Draw out the sequence of client $ ightarrow$ server $ ightarrow$ server $ ightarrow$ client to show how single/double checks work.
3. End-to-End Encryption: Briefly introducing the concept of E2EE (e.g., using Diffie-Hellman handshakes) demonstrates advanced, modern engineering capabilities.
4. Group Chat Optimizations: Differentiate between 1-on-1 message flow and group message distribution to show you understand fan-out bottlenecks.

Common Mistakes

• Assuming Consistent Hashing Solves Gateway Routing: Consistent hashing helps select a server, but clients disconnect and reconnect constantly. A dynamic connection table in Redis is mandatory.
• Synchronous DB Writes: Writing to the database on the main socket connection path will cause gateway threads to block. Always buffer writes using a message queue.
• Ignoring Mobile Batteries: Running constant polling or complex ping networks will drain a user's phone battery. Use clean, spaced heartbeats.