Threads, Executors & Thread Pools Explained
KOTLIN › JVM Concurrency
Coroutines run on threads, and interviewers who have been doing this a while will ask about the layer underneath. Start with why a thread is expensive, because every design decision here follows from it.
A JVM thread costs three things:
- **Memory.** Each one reserves a stack, typically 512KB to 1MB of address space on Android. A thousand threads is a gigabyte of reservation before any of them does anything. - **Scheduling.** Threads are scheduled by the operating system, so switching between two of them is a kernel context switch: save registers, swap page tables, flush parts of the cache. - **Creation.** Starting one is a system call, not an allocation. Roughly a millisecond, which is an eternity in a loop.
A thread also has a defined lifecycle, and the state names come up in questions:
| State | Meaning | |---|---| | NEW | created, start() not yet called | | RUNNABLE | eligible to run (running, or waiting for a CPU) | | BLOCKED | waiting to acquire a monitor lock | | WAITING | in wait(), join(), or park() with no timeout | | TIMED_WAITING | the same with a deadline, including sleep() | | TERMINATED | finished |
Note that RUNNABLE covers both "executing" and "ready but waiting for a core", because the JVM cannot tell them apart. And BLOCKED specifically means a monitor lock, so a thread waiting on IO shows as RUNNABLE, which surprises people reading their first thread dump.
Because threads are expensive, you pool them. The abstraction is ExecutorService, and the first thing to get right is what you submit to it.
Runnable { doWork() } // returns nothing, cannot throw checked exceptions
Callable<Int> { computeValue() } // returns a value, may throw
Callable exists because Runnable.run() returns void and cannot declare exceptions, so there is nowhere to put a result or a failure. Submitting a Callable gives you back a Future:
val future: Future<Int> = executor.submit(Callable { computeValue() })
val result = future.get() // BLOCKS until done
val quick = future.get(2, SECONDS) // or blocks with a deadline
future.cancel(true) // true: interrupt the running thread
Future.get() is the crux of why this API aged badly: **it blocks**. There is no way to be notified when the value is ready, so composing two asynchronous operations means blocking a thread to wait for the first, which defeats the point of the pool.
// Two network calls, one thread blocked the whole time
val a = executor.submit(Callable { api.first() }).get()
val b = executor.submit(Callable { api.second(a) }).get()
That single limitation is the reason for callbacks, then CompletableFuture, then RxJava, then coroutines. Being able to say that lineage in one sentence is a good answer to "why coroutines?"
Note also the exception behaviour: a failure inside a Callable is stored and rethrown from get() wrapped in an ExecutionException. Submit and never get(), and the exception vanishes entirely.
Executors offers convenient factory methods, and one of them is a well-known footgun.
Executors.newFixedThreadPool(4) // exactly 4 threads, unbounded queue
Executors.newSingleThreadExecutor() // 1 thread: tasks run in submission order
Executors.newCachedThreadPool() // unbounded threads, no queue
Executors.newScheduledThreadPool(2) // delayed and periodic tasks
**newCachedThreadPool creates threads without limit.** Its queue is a SynchronousQueue, which holds nothing, so every task that arrives while all existing threads are busy starts a **new** thread. Feed it a burst of a thousand tasks and it tries to create a thousand threads, which on Android means OutOfMemoryError: pthread_create failed rather than a graceful slowdown.
The mirror-image trap is in newFixedThreadPool: its queue is **unbounded**, so tasks never get rejected, they just accumulate. A producer faster than four consumers grows the queue until the heap runs out. One fails loudly by exhausting threads, the other quietly by exhausting memory.
Shutting down has two verbs and both matter:
executor.shutdown() // no new tasks; queued ones still run
executor.shutdownNow() // also interrupt running tasks, return the queued ones
executor.awaitTermination(5, SECONDS) // wait for the above to finish
Neither blocks by itself, which is why awaitTermination exists. And forgetting to shut down at all leaves non-daemon threads alive, which keeps the JVM running: a common cause of a command-line program that finishes its work and never exits.
Both factory methods are thin wrappers over one class, and knowing its parameters lets you answer questions the factories cannot.
ThreadPoolExecutor(
corePoolSize = 4, // threads kept alive even when idle
maximumPoolSize = 8, // ceiling, but see below
keepAliveTime = 60, // how long non-core idle threads survive
unit = TimeUnit.SECONDS,
workQueue = LinkedBlockingQueue(100),
handler = ThreadPoolExecutor.AbortPolicy(),
)
The submission algorithm is where the interview value is, because it is not what people assume:
1. Fewer than corePoolSize threads exist? **Create a new thread**, even if others are idle. 2. Otherwise, **try to queue** the task. 3. Queue full? Create a thread, up to maximumPoolSize. 4. Queue full and at maximum? **Reject** it, via the rejection handler.
Step 2 before step 3 is the surprise. **The pool prefers queueing to growing**, which means that with an unbounded queue maximumPoolSize is never reached at all: step 3 is unreachable because the queue is never full.
// maximumPoolSize is dead code here: LinkedBlockingQueue() is unbounded
ThreadPoolExecutor(2, 50, 60, SECONDS, LinkedBlockingQueue())
That is exactly the configuration people write when they want "2 threads normally, up to 50 under load", and it gives them 2 threads and an ever-growing queue. Bounding the queue is what makes the maximum mean anything.
The four rejection policies: AbortPolicy throws (the default), CallerRunsPolicy runs it on the submitting thread (a crude but effective brake), DiscardPolicy drops it silently, DiscardOldestPolicy drops the head of the queue.
Pool sizing has two standard answers, and the reasoning matters more than the formulas.
**CPU-bound work: threads = cores, or cores + 1.**
val n = Runtime.getRuntime().availableProcessors()
Executors.newFixedThreadPool(n + 1)
More threads than cores cannot compute faster; they only add context switches and cache pressure. The + 1 is a hedge against an occasional page fault leaving a core idle.
**IO-bound work: threads = cores x (1 + wait / compute).**
If a task spends 90ms waiting on the network and 10ms parsing, the ratio is 9, so a 4-core device wants around 4 x 10 = 40 threads. The intuition: a thread parked on a socket uses no CPU, so you need enough of them that some are always ready to run when the cores are free.
This is precisely why Dispatchers.Default is capped at the core count and Dispatchers.IO at 64. Same reasoning, decided for you.
Two caveats worth raising unprompted, because they show you have measured rather than memorised:
- **availableProcessors() on Android can lie.** It reports currently-online cores, and the kernel takes cores offline to save power, so the value can change between calls and be lower than the physical count. - **big.LITTLE means cores are not equal.** Four efficiency cores plus four performance cores is not eight equivalent workers, so a formula assuming uniform cores overestimates throughput.
The honest senior answer: use the formula as a starting point and measure, because the real constraint is usually the downstream service or the disk, not the CPU.
CompletableFuture was the JVM's answer to Future.get() blocking: a future you can attach callbacks to instead of waiting on.
CompletableFuture
.supplyAsync { api.user(id) } // runs on the common pool
.thenApply { user -> user.name } // transform, no blocking
.thenCompose { name -> lookupAsync(name) } // chain another future
.exceptionally { e -> "unknown" } // recover
.thenAccept { println(it) } // terminal side effect
The pairing to know is thenApply versus thenCompose: thenApply maps the value, thenCompose flattens a future of a future. It is exactly map against flatMap, and getting a CompletableFuture<CompletableFuture<T>> is the sign you reached for the wrong one.
CompletableFuture.allOf(a, b, c).join() waits for several, which is the awaitAll equivalent.
Two reasons it is still worth knowing on an Android interview, given nobody writes it in new Kotlin code. First, it appears in Java libraries you have to interoperate with, and kotlinx-coroutines-jdk8 provides await() to bridge it. Second, comparing it with coroutines is a natural question, and the answer is specific:
- **Cancellation** is structural in coroutines and manual in CompletableFuture: cancelling one does not reliably cancel the work feeding it. - **Error handling** splits into exceptionally, handle and whenComplete rather than ordinary try/catch. - **The code reads inside out.** thenCompose chains are callbacks with better syntax; a suspend function is sequential code.
That last point is the whole pitch for coroutines: they give you CompletableFuture's non-blocking composition with the readability of blocking code.
Everything above is what coroutine dispatchers are built on, and being able to connect the two layers is the point of this topic.
Dispatchers.Default and Dispatchers.IO share a pool whose sizing decisions are exactly the formulas from earlier: core count for computation, 64 for blocking work. The library made the choice so you do not have to, and limitedParallelism(n) is how you narrow it for a subsystem without allocating threads of your own.
When you genuinely have an ExecutorService, the bridge is one call:
val executor = Executors.newSingleThreadExecutor()
val dispatcher = executor.asCoroutineDispatcher()
withContext(dispatcher) { nativeCodec.decode(frame) }
dispatcher.close() // you own these threads
And in the other direction, to call a suspend function from Java or from a callback-based executor world, future { } and asCompletableFuture() in kotlinx-coroutines-jdk8 convert a Deferred into something Java can consume.
The comparison an interviewer is usually fishing for:
| | Thread pool | Coroutines | |---|---|---| | Unit of work | Runnable on a thread | continuation on the heap | | Waiting | thread blocked | thread released | | Composition | Future.get() or callbacks | sequential suspend calls | | Cancellation | Future.cancel, interrupts | structural, propagates to children | | Cost of 100k | infeasible | routine |
The sentence that ties it together: **coroutines did not replace thread pools, they multiplexed onto them.** There are still exactly as many threads as before, and a blocking call still occupies one.
Android has its own history with this API, and each piece of it is fair interview ground.
**AsyncTask is dead**, deprecated in API 30, and knowing why is a decent proxy for understanding the platform. It leaked the Activity through its inner-class reference, it had no lifecycle awareness so onPostExecute ran against a destroyed view, its default executor became serial in API 11 so parallel work silently queued, and its error handling had nowhere to put an exception.
**HandlerThread** is still the right tool when something needs a dedicated thread with a message queue: a camera callback thread, or a sensor listener that must not run on main.
val thread = HandlerThread("sensor").apply { start() }
val handler = Handler(thread.looper)
// ...
thread.quitSafely()
**The libraries you already use are executor-backed.** Room takes a query executor and a transaction executor; WorkManager takes one in its Configuration; Retrofit uses OkHttp's dispatcher, which is itself a thread pool with a per-host limit. Being able to say "I would raise Room's query executor" is a more concrete answer than "I would add threads".
**Thread priority** is Android-specific and genuinely useful:
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND)
That is android.os.Process, not java.lang.Thread.setPriority, and it maps to the Linux scheduler so it actually takes effect. Background-priority threads are grouped into a cgroup that receives a small share of CPU, which is why a background thread doing heavy work does not starve rendering.
Finally, **daemon threads**: a JVM exits when only daemon threads remain, so a non-daemon pool that is never shut down keeps a command-line process alive forever. On Android the process model makes this less visible but no less real.
A pool introduces a hazard that a single thread does not: **your task now runs on a thread you did not create, alongside other tasks on other threads.** Everything the memory-model topic covers applies immediately.
// Broken: counter is shared across pool threads with no synchronisation
var counter = 0
repeat(1000) { executor.submit { counter++ } }
executor.shutdown()
executor.awaitTermination(1, MINUTES)
println(counter) // some number under 1000
Two separate problems, and naming both is the senior answer: counter++ is not atomic, so updates are lost, and there is no happens-before edge, so the reading thread may not see other threads' writes at all.
There is one guarantee a pool does give you, and it is easy to miss:
// Actions before submit() happen-before the task executes
val config = loadConfig()
executor.submit { use(config) } // guaranteed to see the fully built config
Submitting a task to an executor creates a happens-before edge, exactly like Thread.start(). So handing data **into** a task is safe. Sharing mutable data **between** tasks is not.
Two practical notes that show up in real code. Give your threads names via a ThreadFactory, because "pool-3-thread-7" in a crash report tells you nothing:
ThreadFactory { r -> Thread(r, "image-decode").apply { isDaemon = true } }
And remember that an exception in a task submitted with execute reaches the thread's uncaught handler and kills that thread, while one submitted with submit is swallowed into the Future and disappears unless someone calls get(). Silent task failure is nearly always this.
Pull it together into the answers worth having ready.
**Why are threads expensive?** A reserved stack of around a megabyte, kernel scheduling with real context switches, and a system call to create one. That cost is why pools exist and why coroutines exist.
**How do you size a pool?** Cores, or cores plus one, for CPU work; cores multiplied by one plus the wait-to-compute ratio for IO. Then measure, because the real limit is usually the service you are calling.
**What is wrong with newCachedThreadPool?** Unbounded thread creation, because its queue holds nothing. And its sibling trap: newFixedThreadPool has an unbounded queue, so it exhausts memory instead of threads.
**Why does maximumPoolSize sometimes do nothing?** Because the pool queues before it grows, so an unbounded queue makes the maximum unreachable.
**Why did we move on from Future?** get() blocks, so composing two async operations wastes a thread. CompletableFuture fixed the composition and kept the callback shape; coroutines gave you the composition with sequential syntax.
**Do coroutines replace all of this?** No. They multiplex onto it. Dispatchers.Default and IO are thread pools sized by the formulas above, and blocking inside a coroutine still occupies a real thread from one of them.
The thing to avoid is dismissing the whole layer as legacy. It is what everything you use is built on, and "I would raise Room's query executor" or "that pool has an unbounded queue" are the kinds of specific answer that only come from knowing it.