How to Build a Real-Time Chat App: Scalable Architecture for 10k+ Users

🚀 Quick Answer

• Tech Stack: Node.js/Go (Backend) + React/Angular (Frontend) + Socket.io (WebSocket Library) + Redis (Message Broker) + MongoDB/PostgreSQL.
• Core Mechanism: Use WebSockets (HTML5 protocol) instead of HTTP polling for real-time bi-directional communication without request overhead.
• Broadcasting: Leverage Redis Pub/Sub (Publish-Subscribe) pattern to relay messages to multiple chat rooms efficiently.
• Persistence: Write messages to a durable database asynchronously (off the main thread) to ensure socket throughput isn't throttled by disk I/O.
• Scaling: Handle connections via Stateless WebSocket servers behind a Load Balancer (Nginx/HAProxy) using binary message frames.

---

🎯 Introduction

If you search for how to build a real-time chat app, you’ll find thousands of tutorials showing you how to send a message from User A to User B on the same screen. But that’s "Hello World" complexity. In the real world, scaling that to 10,000 concurrent users with low latency requires a serious architectural shift.

Understanding how to build a real-time chat app (scalable architecture) means abandoning traditional request-response HTTP models and moving to a persistent connection model. The first 100 words of this guide focus on the core problem: synchronous communication needs open TCP pipes, not restart loop headers. Whether you are architecting a customer support platform or an internal team tool, the architecture remains strikingly similar.

-->

🧠 Core Explanation

The essence of real-time communication is bi-directionality. Unlike a traditional REST API where the client must ask (poll) for updates, a chat app needs an open line where the server pushes data instantly.

There are three main paradigms you can use:

1. Long Polling: The old school way (HTTP). Browser sends a request, server holds it open. HTTP limit. Not for "chat" apps.
2. Server-Sent Events (SSE): One-way stream (HTTP). Good for news feeds, bad for chat. You can't easily send a reply.
3. WebSockets: Full-duplex (TCP). Two open channels. The standard for chat.

When analyzing how to build a real-time chat app, you aren't just choosing HTTP; you are choosing an architecture that maintains stateful connections.

🔥 Contrarian Insight

"Do not over-architect your initial chat app with microservices unless you handle Quorum-based distributed locking manually."

Most engineers default to Kafka and RabbitMQ for chat architecture immediately. I’ve seen .NET developers spin up a whole Kubernetes cluster for a chat server. My advice: Start with a single monolithic WebSocket server or a horizontally scalable stateless gateway. The bottleneck in early-stage chat apps isn't the architecture; it's the disk writes and the message complexity. Get it working with Redis Pub/Sub first. Only move to Kafka if you have thousands of messages per second per user.

🔍 Deep Dive / Details

Architecture Pattern: Push to Pull

To build a scalable chat app, you must decouple the protocol layer from the persistence layer.

1. The Connection Layer: The client connects via WebSocket (usually wss://). The Server keeps this connection open.
2. The Transport Layer: When User A sends a message, the Server receives it via WebSocket.
3. The Relay Layer: Instead of finding User B's WebSocket connection and piping it directly (which gets hard across server nodes), the Server publishes the message to Redis (PUBLISH chat:room_1).
4. The Broker Layer: Any other WebSocket server listening to that Redis channel receives the event.
5. The Delivery Layer: That server pushes the message into the specific TCP socket held by User B.
6. The Storage Layer: A background worker/service (queue runner) picks up the message and appends it to PostgreSQL/MongoDB. Crucial: The storage operation happens outside the WebSocket event loop.

🏗️ System Design / Architecture (For Coding Topics)

Data Flow & Trade-offs

• Storage Blocking: If you run db.collection.insert() inside your WebSocket on('message') handler, you will hit 100ms+ latency instantly. Fix: Use a Producer-Consumer pattern. Write to Redis List/Queue, have a side-cron worker save to disk.
• Database Schema: Messages within a chat room need a small footprint. User_ID, Room_ID, Timestamp, Body. Keep indexes minimal.

API Structure

• POST /api/v1/auth → Returns JWT + Session Token.
• GET /api/v1/sessions → Returns a list of active WebSocket connections (even if ephemeral).
• POST /api/v1/chat → For historical data.
• WS /socket.io/ → For real-time streams.

🧑‍💻 Practical Value

Step-by-Step Implementation (Node.js + Socket.io + Redis)

Here is the production-grade setup logic. Do not copy-paste this blindly; understand why we are separating the logic.

1. The Server (Node.js)

const io = require('socket.io')(server);
const redis = require('socket.io-redis');

// This allows socket.io to broadcast across multiple servers for scaling
io.adapter(redis({ host: 'redis-cache', port: 6379 }));

io.on('connection', (socket) => {
  const userId = socket.handshake.auth.token; // From auth header

  // Join a specific chat room
  socket.join(`room-${roomId}`);

  // Listen for private chat requests or public broadcasts
  socket.on('chat message', async (msgPayload) => {
    try {
      // 1. Create message object (without saving yet)
      const messageEvent = {
        id: crypto.randomUUID(),
        roomId: msgPayload.roomId,
        userId: userId,
        text: msgPayload.text,
        timestamp: new Date()
      };

      // 2. Send to ALL users in that room via Socket.io (fast, in-memory or Redis)
      io.to(messageEvent.roomId).emit('chat message', messageEvent);

      // 3. Send to Redis Queue for Persistence (Non-blocking)
      await redisClient.lPush('chat_queue', JSON.stringify(messageEvent));
      
    } catch (error) {
      console.error("Failed to process message", error);
      socket.emit('error', 'Message failed to send');
    }
  });
});

2. The Background Worker (Persistence) This is the part usually skipped in tutorials.

const redis = require('redis');

const subscriber = redis.createClient();
const db = require('./databaseClient'); // Your Mongo/Postgres connection

subscriber.subscribe('chat_queue', () => {
  console.log("Worker listening for messages...");
});

subscriber.on('message', async (channel, message) => {
  const data = JSON.parse(message);
  
  // Save to DB
  await db.messages.create({
    id: data.id,
    roomId: data.roomId,
    userId: data.userId,
    text: data.text,
    timestamp: data.timestamp
  });

  // Optimization: If you need message status, update status here
});

Common Mistakes to Avoid

• Nuclear Spamming: Sending a loop of 10 messages instead of one 10kb message. Browsers throttle this.
• Not Handling Reconnection: What happens if User B drops their Wi-Fi and comes back? You need a "History Sync" logic based on timestamps when they reconnect.
• Ignoring Encryption: Running on standard ws:// is a security liability. You must force wss:// (secure WebSocket) or implement your own TLS termination.

⚔️ Comparison Section

Feature	WebSockets (Recommended)	SSE (Server-Sent Events)
Direction	Bi-directional	One-way (Client -> Server limited)
Implementation	Requires Heart-beats & State	Standard HTTP
Browser Support	Universal	Universal
NAT Traversal	Needs Config (Stunner/TURN)	Works behind Firewalls
Best For	Chat, Gaming, Trading	News feeds, File updates

⚡ Key Takeaways

• Architecture: Use a Pub/Sub pattern (Redis) to scale WebSocket servers horizontally.
• Performance: Always offload database writes to a worker queue to keep latency < 50ms.
• Language: Node.js is the standard for chat due to its non-blocking I/O and the maturity of socket.io. Go is a strong alternative if you need extreme raw TCP performance for massive concurrency.
• Reliability: Implement optimistic UI updates for the sender, but rely on server confirmation for receiving sockets to prevent race conditions or message duplicates.

🔗 Related Topics

• How to Optimize Redis for High Concurrency
• Difference Between WebSocket vs HTTP Long Polling
• Scaling Node.js to 100k Concurrent Connections
• Implementing JWT in Mobile Chat Apps

🔮 Future Scope

Next-gen chat architecture is moving toward Commit-Revoke Streams (used by Discord/Slack). This allows clients to send a message, get a confirmation, reply "confirm", and later "delete" it, without waiting for a second roundtrip for every modification. We are also seeing WebRTC deep integration for voice/video chat apps built on top of the text chat infrastructure.

❓ FAQ

• Q: Is Socket.io a WebSocket implementation? A: Yes. It provides a standard interface around the WebSocket protocol, handling reconnections and fallbacks automatically.
• Q: How to handle "Offline" messages? A: Use a flag is_read = false in the database. When the client reconnects, fetch all messages where is_read = false.
• Q: Can I use PostgreSQL for the queue? A: Yes, via "listen/notify", but Redis is significantly faster for high-frequency pub/sub messaging.
• Q: What about NAT issues? A: Use STUN servers (publicly available like stun:stun.l.google.com:19302) for WebRTC cases, or ensure the Load Balancer maintains WebSocket upgrades (sticky sessions or Redis affinity).

🎯 Conclusion

Building a scalable chat app is a exercise in managing state and throughput. By decoupling the media (WebSockets) from the memory (Redis) and the storage (Database), you create a system that can handle millions of messages. Don't get stuck trying to rig a static HTTP server to feel real-time; embrace WebSockets and Pub/Sub from day one.

---

Would you like a reference architecture diagram for the "Commit-Revoke" pattern mentioned above?