These are the notes for Java episode 41. Our chai stall has been hiring a brand-new cook for every single order (new Thread per task). With 20,000 orders, hiring and letting go of cooks takes whole seconds while the real work takes almost nothing. We invent the fix: a thread pool, a small team of reusable threads behind an ExecutorService. The pool never lets its program end until we call shutdown(). Then we want results back: a Callable returns a value and a Future is the token for it, but get() blocks. Finally CompletableFuture lets us say what should happen next (thenApply, thenAccept, thenCombine, exceptionally) without standing at the counter. Every example was run with javac 21.
Threaded programs can print in a different order, and timings differ on every machine. The outputs below are real runs; your millisecond numbers and line order may be different.
The problem: a new cook for every order
Real-life example: hire a cook, fire a cook. An order arrives (a task), the owner hires a new cook (new Thread), the cook makes one chai (run()), and leaves (the thread ends). Next order: hire again.
public class Main {
public static void main(String[] args) throws InterruptedException {
for (int order = 1; order <= 5; order++) {
int n = order;
Thread cook = new Thread(() -> System.out.println(
"Order " + n + " made by " + Thread.currentThread().getName()));
cook.start(); // hire a new cook
cook.join(); // ...who leaves after one chai
}
}
}
Output:
Order 1 made by Thread-0
Order 2 made by Thread-1
Order 3 made by Thread-2
Order 4 made by Thread-3
Order 5 made by Thread-4
Thread.currentThread().getName() returns the name of the thread running that line. Five orders, five different threads: every one was created for a single task and then ended. join() (episode 39) waits until a thread is finished.
Scale it up: 20,000 orders
public class Main {
public static void main(String[] args) throws InterruptedException {
long start = System.currentTimeMillis();
for (int order = 1; order <= 20_000; order++) {
Thread cook = new Thread(() -> { int bill = 15 + 5; });
cook.start();
cook.join();
}
long ms = System.currentTimeMillis() - start;
System.out.println("20000 orders, 20000 cooks hired: " + ms + " ms");
}
}
Output (one run; your number will differ):
20000 orders, 20000 cooks hired: 2799 ms
System.currentTimeMillis() reads the clock in milliseconds; the difference of two readings is the time in between. The work itself (15 + 5) costs almost nothing, so nearly all of those seconds are spent creating and ending threads.
Why a new thread is expensive
| Step | What happens |
|---|---|
new Thread() + start() |
the JVM asks the operating system for a real thread |
| own stack | the thread gets its own stack: memory for its method calls (often up to about 1 MB reserved) |
| run the task | a tiny bit of work |
| thread ends | the OS thread and its stack are cleaned up |
Creating and ending a thread costs far more than a small task. And with one thread per request, 10,000 users at the same moment means 10,000 threads: the machine can run out of memory. The fix: keep a few threads and reuse them.
Thread pools and ExecutorService
Real-life example: a kitchen team and an order rail.
| In the kitchen | In Java | Meaning |
|---|---|---|
| a team of 3 cooks, hired once | thread pool | a fixed set of threads, created once and reused |
| the order rail, slips waiting in a line | task queue | submitted tasks wait until a thread is free |
| the head cook who takes orders | ExecutorService |
the object you hand tasks to with submit(task) |
Executorsis a helper class with factory methods that build pools:newFixedThreadPool(n)(exactly n threads),newSingleThreadExecutor()(one thread, tasks run in order),newCachedThreadPool()(grows and shrinks as needed),newVirtualThreadPerTaskExecutor()(Java 21: very cheap virtual threads).submit(task)puts a task in the queue and returns immediately.
The pool that never ends
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Main {
public static void main(String[] args) {
ExecutorService kitchen = Executors.newFixedThreadPool(3); // 3 cooks
for (int order = 1; order <= 6; order++) {
int n = order;
kitchen.submit(() -> System.out.println(
"Order " + n + " made by " + Thread.currentThread().getName()));
}
System.out.println("All orders handed to the kitchen");
}
}
Output (one real run; the line order changes from run to run), and then the program never ends: the online compiler stops it with Time Limit Exceeded:
All orders handed to the kitchen
Order 1 made by pool-1-thread-1
Order 4 made by pool-1-thread-1
Order 5 made by pool-1-thread-1
Order 6 made by pool-1-thread-1
Order 2 made by pool-1-thread-2
Order 3 made by pool-1-thread-3
Only pool-1-thread-1, -2 and -3 appear: six orders, at most three threads, reused. But the pool's threads stay alive, waiting for the next task, and the JVM exits only when every normal (non-daemon) thread has finished. So the program runs forever.
shutdown and awaitTermination
| Method | Meaning |
|---|---|
shutdown() |
accept no new tasks; finish the queued ones; then the threads end |
awaitTermination(5, TimeUnit.SECONDS) |
wait until all tasks are done, at most 5 seconds; returns true if they finished |
shutdownNow() |
try to stop running tasks (interrupts them) and return the tasks that never started |
TimeUnit is an enum of time units: SECONDS, MILLISECONDS, MINUTES and so on.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class Main {
public static void main(String[] args) throws InterruptedException {
long start = System.currentTimeMillis();
ExecutorService kitchen = Executors.newFixedThreadPool(3); // 3 cooks
for (int order = 1; order <= 20_000; order++)
kitchen.submit(() -> { int bill = 15 + 5; });
kitchen.shutdown(); // no new orders
kitchen.awaitTermination(5, TimeUnit.SECONDS); // wait for the rest
long ms = System.currentTimeMillis() - start;
System.out.println("20000 orders, 3 cooks: " + ms + " ms");
}
}
Output (one run):
20000 orders, 3 cooks: 48 ms
The same 20,000 tasks in a small fraction of a second instead of whole seconds, and the program ends properly. That's why servers always use thread pools.
Since Java 19,
ExecutorServiceisAutoCloseable, sotry (var pool = Executors.newFixedThreadPool(3)) { ... }callsclose(), which shuts down and waits, automatically (try-with-resources, episode 27).
Getting a result back: Callable and Future
Runnable can't return a value
public class Main {
public static void main(String[] args) {
Runnable makeChai = () -> {
System.out.println("Making chai...");
return 15; // the bill
};
new Thread(makeChai).start();
}
}
Compiler output:
Main.java:5: error: incompatible types: bad return type in lambda expression
return 15; // the bill
^
unexpected return value
1 error
Runnable.run() returns void, so a Runnable can do work but can't give anything back.
Real-life example: the food-court token.
| At the food court | In Java | Meaning |
|---|---|---|
| an order with a result (a dosa, a bill) | Callable<V> |
a task whose call() returns a value of type V (and may throw exceptions) |
| the token you get at once | Future<V> |
an object standing for a result that will arrive later |
| wait at the counter, then collect | future.get() |
wait until the result is ready, then return it |
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws Exception {
ExecutorService kitchen = Executors.newFixedThreadPool(2);
Callable<Integer> makeChai = () -> {
Thread.sleep(1000); // making the chai
return 15; // the bill
};
Future<Integer> token = kitchen.submit(makeChai);
System.out.println("Got a token. Ready? " + token.isDone());
int bill = token.get(); // wait for the result
System.out.println("Ready? " + token.isDone() + ". Bill: Rs " + bill);
kitchen.shutdown();
}
}
Output:
Got a token. Ready? false
Ready? true. Bill: Rs 15
submit(callable)returns aFuture<Integer>immediately: the token.isDone()tells you whether the result is ready, without waiting.get()waits (blocks) until the result is ready.get(2, TimeUnit.SECONDS)waits at most 2 seconds, then throwsTimeoutException.cancel(true)tries to cancel the task.
When the task fails: ExecutionException
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws Exception {
ExecutorService kitchen = Executors.newFixedThreadPool(2);
Future<Integer> token = kitchen.submit(() -> {
if (true) throw new IllegalStateException("Milk is over");
return 15;
});
System.out.println("Waiting for the chai...");
try {
int bill = token.get();
} finally {
kitchen.shutdown();
}
}
}
Output:
Waiting for the chai...
Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Milk is over
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at Main.main(Main.java:12)
Caused by: java.lang.IllegalStateException: Milk is over
at Main.lambda$main$0(Main.java:7)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
The exception happened in the pool's thread, but it reaches you when you call get(), wrapped in a java.util.concurrent.ExecutionException. The Caused by: line shows the real exception. Get it in code with e.getCause(). (The if (true) only lets the compiler accept the return 15 after the throw.)
The catch: get() blocks
import java.util.concurrent.*;
public class Main {
static int cook(String item, int ms) throws InterruptedException {
Thread.sleep(ms);
System.out.println(item + " ready");
return 10;
}
public static void main(String[] args) throws Exception {
ExecutorService kitchen = Executors.newFixedThreadPool(2);
Future<Integer> chai = kitchen.submit(() -> cook("chai", 2000));
Future<Integer> samosa = kitchen.submit(() -> cook("samosa", 2000));
int total = chai.get() + samosa.get(); // main just stands and waits
System.out.println("Main: now I can pack. Bill: Rs " + total);
System.out.println("Main: next customer, please!");
kitchen.shutdown();
}
}
Output:
chai ready
samosa ready
Main: now I can pack. Bill: Rs 20
Main: next customer, please!
Chai and samosa were made at the same time (two threads), but main stood at the counter for two seconds, doing nothing, before it could serve the next customer. Blocking means the calling thread stops until the result arrives. In a server or an app, a blocked thread means slow responses or a frozen screen.
CompletableFuture: "call me when it's ready"
Real-life example: ordering on a delivery app. You don't wait at the stall; you say make it, then pack it, then deliver it, and get on with your day.
| On the app | In Java | Meaning |
|---|---|---|
| make the chai, in the background | CompletableFuture.supplyAsync(supplier) |
start a task that supplies (returns) a value, on another thread |
| then pack it | .thenApply(fn) |
when the result arrives, turn it into a new result |
| then deliver it | .thenAccept(consumer) |
when the result arrives, use it (returns nothing) |
| before closing, make sure it's done | .join() |
wait for the result, like get() but without a checked exception |
Asynchronous means the work finishes later and the next step runs on its own, while the caller carries on.
import java.util.concurrent.CompletableFuture;
public class Main {
static void sleep(int ms) {
try { Thread.sleep(ms); } catch (InterruptedException e) { }
}
public static void main(String[] args) {
CompletableFuture<Void> order = CompletableFuture
.supplyAsync(() -> { sleep(500); return "chai"; }) // make it
.thenApply(item -> item + " in a kulhad") // then pack it
.thenAccept(item -> System.out.println("Delivered: " + item)); // deliver
System.out.println("Main: next customer, please!");
order.join(); // before closing, finish the order
}
}
Output:
Main: next customer, please!
Delivered: chai in a kulhad
main served the next customer first. The chain then ran by itself. join() at the very end keeps the program alive until the order is finished.
Without an executor argument,
supplyAsyncruns on the shared common pool (ForkJoinPool.commonPool()), whose threads are daemon threads: ifmainends first, unfinished work is simply dropped. Alwaysjoin()(or wait some other way) before the program ends, or pass your own executor.
Two tasks at once: thenCombine
import java.util.concurrent.*;
public class Main {
static String cook(String item, int ms) {
try { Thread.sleep(ms); } catch (InterruptedException e) { }
return item;
}
public static void main(String[] args) {
ExecutorService kitchen = Executors.newFixedThreadPool(2);
long start = System.currentTimeMillis();
CompletableFuture<String> chai =
CompletableFuture.supplyAsync(() -> cook("chai", 1000), kitchen);
CompletableFuture<String> samosa =
CompletableFuture.supplyAsync(() -> cook("samosa", 1000), kitchen);
String plate = chai.thenCombine(samosa, (a, b) -> a + " + " + b).join();
long sec = Math.round((System.currentTimeMillis() - start) / 1000.0);
System.out.println("Plate: " + plate + ", ready in about " + sec + " second");
kitchen.shutdown();
}
}
Output:
Plate: chai + samosa, ready in about 1 second
- The second argument of
supplyAsync(supplier, kitchen)is the executor to run on: our own pool, so we control the threads. a.thenCombine(b, (x, y) -> ...)waits for both and combines their results. One second, not two: they were cooked side by side.- For many futures,
CompletableFuture.allOf(f1, f2, f3)completes when all are done;anyOf(...)when the first one is.
Handling failure: exceptionally
import java.util.concurrent.CompletableFuture;
public class Main {
static boolean milkLeft = false;
public static void main(String[] args) {
String drink = CompletableFuture
.supplyAsync(() -> {
if (!milkLeft) throw new IllegalStateException("Milk is over");
return "chai";
})
.exceptionally(e -> "lemon tea, because: " + e.getCause().getMessage())
.join();
System.out.println("Served: " + drink);
}
}
Output:
Served: lemon tea, because: Milk is over
exceptionally(fn) is the backup plan: if any earlier step threw, it receives the exception and returns a replacement value. The exception arrives wrapped in a CompletionException, so e.getCause() is the real one. Related: handle((value, error) -> ...) runs on success and failure; whenComplete observes without changing the result.
Future vs CompletableFuture
Future |
CompletableFuture |
|
|---|---|---|
| get the result | get(): you wait |
thenApply / thenAccept: it calls you |
| combine two results | get both, by hand | thenCombine, allOf, anyOf |
| failures | ExecutionException from get() |
exceptionally, handle in the chain |
| complete it yourself | no | complete(value) |
Common mistakes (and what you see)
| Mistake | What you see | Why | Fix |
|---|---|---|---|
Executors.newFixedThreadPool(3) with no shutdown() |
program never ends; Time Limit Exceeded | pool threads wait for more tasks | shutdown(), then awaitTermination (or try-with-resources) |
Runnable r = () -> { return 15; }; |
error: incompatible types: bad return type in lambda expression / unexpected return value |
run() returns void |
use Callable<Integer> |
int bill = future.get(); when the task threw |
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Milk is over |
the task's exception is wrapped | catch it; read getCause() |
f.get(); g.get(); h.get(); all over the main thread |
no error: slow program, frozen screen | get() blocks the caller |
chain with thenApply / thenCombine, join() once at the end |
a new Thread for every request |
no error: slow, then possibly OutOfMemoryError: unable to create native thread |
threads are expensive | a thread pool |
supplyAsync(...) with no join() before main ends |
the last step may never run | common-pool threads are daemon threads | join() at the end, or your own executor |
Mini project: three orders, one kitchen
Three dishes are cooked at the same time by a pool of three cooks, each packed as soon as it's ready, and the stall waits once, at the end, for all of them with allOf.
import java.util.List;
import java.util.concurrent.*;
public class Main {
static String cook(String dish, int ms) {
try { Thread.sleep(ms); } catch (InterruptedException e) { }
return dish;
}
public static void main(String[] args) {
ExecutorService kitchen = Executors.newFixedThreadPool(3);
List<String> menu = List.of("chai", "samosa", "poha");
List<CompletableFuture<String>> orders = menu.stream()
.map(d -> CompletableFuture.supplyAsync(() -> cook(d, 400), kitchen)
.thenApply(x -> x + " packed"))
.toList();
CompletableFuture.allOf(orders.toArray(new CompletableFuture[0])).join(); // wait for all
for (CompletableFuture<String> o : orders) System.out.println(o.join());
kitchen.shutdown();
System.out.println("All " + orders.size() + " orders delivered");
}
}
Output:
chai packed
samosa packed
poha packed
All 3 orders delivered
menu.stream().map(...).toList() (streams, episode 37) turns each dish into a CompletableFuture<String>. allOf needs an array, hence toArray(new CompletableFuture[0]). After allOf(...).join(), every o.join() returns at once.
Practice
- Time 10,000 tasks: new threads vs a pool of 4.
- A
Callablethat returns the sum 1 to 1,000,000. - A chain: fetch a price → add 18% GST → print the bill.
- Combine two prices; fall back to 0 if one shop fails.
Solution 1
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
long t0 = System.currentTimeMillis();
for (int i = 0; i < 10_000; i++) {
Thread t = new Thread(() -> { int x = 2 + 2; });
t.start();
t.join();
}
long threads = System.currentTimeMillis() - t0;
long t1 = System.currentTimeMillis();
ExecutorService pool = Executors.newFixedThreadPool(4);
for (int i = 0; i < 10_000; i++) pool.submit(() -> { int x = 2 + 2; });
pool.shutdown();
pool.awaitTermination(10, TimeUnit.SECONDS);
long pooled = System.currentTimeMillis() - t1;
System.out.println("new threads: " + threads + " ms");
System.out.println("pool of 4: " + pooled + " ms");
}
}
Output (one run; your numbers will differ, the gap will not):
new threads: 4271 ms
pool of 4: 53 ms
Solution 2
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws Exception {
ExecutorService pool = Executors.newSingleThreadExecutor();
Callable<Long> sum = () -> {
long s = 0;
for (int i = 1; i <= 1_000_000; i++) s += i;
return s;
};
Future<Long> result = pool.submit(sum);
System.out.println("Sum 1..1,000,000 = " + result.get());
pool.shutdown();
}
}
Output:
Sum 1..1,000,000 = 500000500000
long, not int: the sum is far bigger than Integer.MAX_VALUE (episode 2).
Solution 3
import java.util.concurrent.CompletableFuture;
public class Main {
static int fetchPrice(String item) {
try { Thread.sleep(300); } catch (InterruptedException e) { }
return item.equals("thali") ? 200 : 50;
}
public static void main(String[] args) {
CompletableFuture
.supplyAsync(() -> fetchPrice("thali"))
.thenApply(price -> price + price * 18 / 100) // add 18% GST
.thenAccept(total -> System.out.println("Thali bill with GST: Rs " + total))
.join();
}
}
Output:
Thali bill with GST: Rs 236
Solution 4
import java.util.concurrent.CompletableFuture;
public class Main {
static int price(String shop, boolean online) {
if (!online) throw new IllegalStateException(shop + " is offline");
return 120;
}
public static void main(String[] args) {
CompletableFuture<Integer> a = CompletableFuture
.supplyAsync(() -> price("Shop A", true))
.exceptionally(e -> 0);
CompletableFuture<Integer> b = CompletableFuture
.supplyAsync(() -> price("Shop B", false))
.exceptionally(e -> {
System.out.println("Fallback: " + e.getCause().getMessage());
return 0;
});
int total = a.thenCombine(b, Integer::sum).join();
System.out.println("Total: Rs " + total);
}
}
Output:
Fallback: Shop B is offline
Total: Rs 120
Integer::sum is a method reference (episode 36) for (x, y) -> x + y.
Quick quiz
ExecutorService pool = Executors.newFixedThreadPool(2);
pool.submit(() -> System.out.println("chai"));
System.out.println("done");
This prints chai and done, but the program never ends. Why?
A. submit needs a Callable B. the pool's threads wait for more tasks C. println is not thread-safe D. main must call get()
Answer
B. A fixed pool keeps its threads alive, waiting for the next task, so the JVM never exits. Add pool.shutdown(); after the last submit.
Interview questions
- Why use a thread pool instead of
new Thread()? Creating and ending OS threads is expensive; a pool reuses a few threads, limits how many run at once, and queues the rest. - What is
ExecutorService? An interface for running tasks on managed threads:submit,invokeAll,invokeAny,shutdown,awaitTermination. shutdown()vsshutdownNow()?shutdownstops accepting tasks and lets queued ones finish;shutdownNowinterrupts running tasks and returns the ones never started.RunnablevsCallable?Runnable.run()returns nothing and can't throw checked exceptions;Callable.call()returns a value and may throw.- What is a
Future? A handle to a result that arrives later:get(),get(timeout),isDone(),cancel(). - What is
ExecutionException? The wrapperFuture.get()throws when the task failed;getCause()is the original exception. - Why is
CompletableFuturebetter thanFuturefor pipelines? It can be chained and combined without blocking (thenApply,thenCompose,thenCombine,allOf) and handles errors in the chain. thenApplyvsthenCompose?thenApplymaps the value with a normal function;thenComposeis for a function that itself returns aCompletableFuture(it flattens, likeflatMap).get()vsjoin()? Both wait;get()throws checkedExecutionException/InterruptedException,join()throws uncheckedCompletionException.- Which pool does
supplyAsyncuse by default? The commonForkJoinPool; pass your own executor for blocking work such as I/O. - What are virtual threads? (Java 21) Very lightweight threads managed by the JVM, cheap enough to use one per task:
Executors.newVirtualThreadPerTaskExecutor().
Glossary
| Term | Meaning |
|---|---|
| Task | a unit of work to run, e.g. one order |
Thread.currentThread().getName() |
the name of the thread running this line |
System.currentTimeMillis() |
the current time in milliseconds; subtract two readings to time code |
| Operating system thread | a real thread managed by the OS; each Java platform thread is one |
| Stack (of a thread) | memory that holds a thread's method calls and local variables |
| Thread pool | a fixed set of threads created once and reused for many tasks |
| Task queue | the line where submitted tasks wait for a free thread |
Executor / ExecutorService |
interfaces for running tasks on managed threads; ExecutorService adds shutdown and results |
Executors |
helper class with factory methods that build pools |
newFixedThreadPool(n) |
a pool with exactly n threads |
submit(task) |
hand a task to the executor; returns a Future |
| Non-daemon thread | a normal thread; the JVM waits for all of them before exiting |
| Daemon thread | a background thread that doesn't keep the JVM alive |
shutdown() |
stop accepting tasks; finish queued ones; let the threads end |
awaitTermination(t, unit) |
wait up to t for all tasks to finish |
TimeUnit |
an enum of time units: SECONDS, MILLISECONDS, ... |
Callable<V> |
a task whose call() returns a value of type V and may throw |
Future<V> |
a handle to a result that will arrive later |
get() |
wait for a future's result (blocks) |
isDone() |
whether a future's result is ready |
ExecutionException |
thrown by get() when the task failed; wraps the real exception |
| Blocking | the calling thread stops until something finishes |
| Asynchronous | the work finishes later and the next step runs on its own |
CompletableFuture<V> |
a future you can chain, combine and complete |
supplyAsync |
start a task that returns a value, on another thread |
thenApply |
turn a result into a new result when it arrives |
thenAccept |
use a result when it arrives (returns nothing) |
thenCombine |
wait for two futures and combine their results |
allOf / anyOf |
a future that completes when all / any of the given futures do |
join() |
wait for a CompletableFuture's result, without checked exceptions |
exceptionally |
a backup value if an earlier step failed |
CompletionException |
the unchecked wrapper around a failure inside a CompletableFuture |
| Common pool | the shared ForkJoinPool that async tasks use by default |
| Virtual thread | (Java 21) a very lightweight JVM-managed thread |
Summary
- A new
Threadper task is slow and risky at scale: most of the time goes into creating and ending threads. - A thread pool (
ExecutorServicefromExecutors) reuses a few threads; submit tasks withsubmit, and alwaysshutdown()(or the program never ends). Runnablereturns nothing; aCallablereturns a value, andsubmitgives back aFuturetoken.get()waits, and a failure comes asExecutionException.get()blocks.CompletableFuturechains steps instead:supplyAsync→thenApply→thenAccept, combine withthenCombine/allOf, recover withexceptionally, andjoin()once at the end.- Next episode: modern Java: the Date/Time API,
var, text blocks, sealed classes and pattern matching.