Design: Real-Time Chat Explained
ADVANCED › System Design
Prompt: **design a real-time chat app**. The first decision is transport, and it splits by whether the app is foregrounded. While the chat screen is open, a **persistent WebSocket** gives full-duplex, low-latency delivery over one connection, far cheaper than polling, and unlike Server-Sent Events, it's bidirectional (the client can send too).
But Android's Doze and App Standby suspend background network activity, so a socket dies once the app is backgrounded long enough, and keeping it alive with a permanent foreground service is the wrong trade (battery, and it's not what foreground services are for). The sanctioned way to reach a backgrounded app is a **high-priority FCM message** that wakes the process to fetch the new message or post a notification.
So: WebSocket while foregrounded, FCM as the backgrounded wake path, not two competing systems, one covers where the other can't.
**Optimistic send** is what makes the UI feel instant: the moment the user hits send, insert the message locally as pending and render it immediately, don't wait for a server round trip. The tricky part is retries. If the socket drops mid-send and the client resends, how does the server avoid creating a duplicate message?
The answer is a **stable client-generated id** sent as an idempotency key, the same id on every retry of the same logical message:
data class SendMessageRequest(
val clientMessageId: String = UUID.randomUUID().toString(),
val roomId: String,
val text: String
)
// Server upserts on clientMessageId, a duplicate retry is a silent no-op
The server dedupes on that key and echoes back the canonical server id and sequence number once the message is actually persisted.
**Ordering** across participants can't rely on each device's own clock, phones drift, users change time zones, and clocks can just be wrong. The server assigns an authoritative **sequence number** (or server timestamp) when it persists a message, and every client reorders to match that, regardless of the order messages actually arrived on the wire:
data class ChatMessage(val id: String, val serverSeq: Long, val text: String)
val ordered = messages.sortedBy { it.serverSeq }
This also gives you gap detection for free: if a client sees sequence 100 then 103, it knows 101 and 102 are missing (a dropped socket message, say) and can request exactly that range as a targeted backfill, instead of reloading the whole conversation.
As with every offline-capable design, **Room is the single source of truth** for the message list, the UI never binds directly to the raw socket stream. Each message also carries a **delivery state**: pending (queued locally, not yet ACKed), sent (server ACKed), delivered (recipient device received it), read (recipient viewed it). The sender's own client sets pending; the server ACK drives sent; the recipient's device emits delivered or read receipts back through the same channel.
@Dao
interface MessageDao {
@Query("SELECT * FROM messages WHERE roomId = :roomId ORDER BY serverSeq ASC")
fun observeMessages(roomId: String): Flow<List<MessageEntity>>
}
Socket events and FCM-triggered fetches both just write into this DAO, the UI stays agnostic to which path delivered the data.
TCP can leave a socket **half-open**, the connection looks alive locally but the other side is long gone, common after a radio handoff. OS-level TCP keepalive is too slow and unreliable on mobile to catch this promptly, so chat apps send their own **app-level ping or pong heartbeat** and tear down the socket if a pong doesn't arrive in time.
When a reconnect is needed, don't hammer the server immediately, especially after an outage when thousands of clients drop at once. Use **exponential backoff with jitter**:
fun reconnectDelay(attempt: Int): Long {
val base = minOf(30_000L, 1_000L * (1L shl attempt))
val jitter = (Math.random() * 1_000).toLong()
return base + jitter
}
The cap keeps delay bounded, and the random jitter spreads reconnect attempts out in time so they don't all land on the server in the same instant, a thundering herd.
Messages sent while offline need to survive the app being killed, that means a **persisted send queue** in Room with a pending status, not an in-memory list. When connectivity returns, flush the queue, over the live socket if it's connected, or via a WorkManager job constrained to NetworkType.CONNECTED as the durable fallback:
val flushWork = OneTimeWorkRequestBuilder<SendQueueWorker>()
.setConstraints(Constraints(requiredNetworkType = NetworkType.CONNECTED))
.build()
WorkManager.getInstance(context).enqueueUniqueWork(
"flush_send_queue", ExistingWorkPolicy.KEEP, flushWork
)
enqueueUniqueWork with KEEP stops duplicate flush jobs piling up if connectivity flaps on and off. Because every queued message already carries its clientMessageId from the optimistic-send step, replays from this flush are safe, the server just no-ops the ones it already has.
For scrolling **history**, the same pagination rule from the feed lesson applies, and matters even more here because messages arrive continuously while you scroll: OFFSET-based paging shifts under you as new messages land, causing skipped or duplicated rows. **Cursor or keyset pagination**, anchored to a message id or timestamp boundary, doesn't:
@GET("rooms/{id}/messages")
suspend fun getMessages(
@Path("id") roomId: String,
@Query("before") beforeId: String?,
@Query("limit") limit: Int = 50
): List<Message>
A RemoteMediator writing into Room, with PagingSource reading from Room, gives you the same offline-first paged history as the feed design, backed by the same Room-as-SSOT principle.
Two closing details tie the design together. First, **dedup**: the same message can legitimately arrive twice, once over the live socket, again from an FCM-triggered fetch after a reconnect. An idempotent **upsert keyed by the server message id** collapses both deliveries into one row, so the UI never shows a duplicate:
@Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun upsert(message: MessageEntity) // second delivery is a no-op
Second, **attachments**: never cram a photo or video into a WebSocket frame or an FCM payload (FCM caps out around 4KB). Upload the blob separately over resumable HTTP, then send a small control message carrying its URL, that keeps the real-time channel lean and gives you upload progress and retry for free.
Putting it together: WebSocket-plus-FCM transport, Room as SSOT with delivery states, server sequence numbers for order, client ids for idempotency, cursor pagination for history, and backoff for reconnects, that's a complete answer.