Being a software developer has changed how I understand lunch.
Sandwiches
On Saturday I made myself a sandwich, and then my kids wanted one, and then my partner wanted one. For each:
- Get bread, toppings, knives, plates, cutting board
- Wash veggies, slice cheese, assemble sandwich
- Deliver sandwich!
- Return unused toppings
- Wash & return dishes
Making all the sandwiches in one go would have been faster. And if I’m being honest, sometimes I don’t do the dishes right away.
HTTP Requests
At my job we’ve got a bunch of computers running a bunch of JVMs to serve HTTP requests. For each call:
- Decode HTTP headers and select a target action
- Decode the request by parsing JSON or protocol buffers
- Invoke the action!
- Encode response, transmit it
- Garbage collect all the objects created along the way
I’m on the JVM so I don’t write any code to collect the garbage; the GC fairy that does that for me.
Thrash!
If everything is in its place it’s about 4 minutes to deliver a sandwich. But if the plates and cutting board are dirty in the sink when my kids demand lunch – that slows things down.
The total amount of work hasn’t gotten larger! But sandwiches take longer to arrive because the work washing dishes now precedes everything else.
Unintuitively, the amount of work may actually have decreased. By letting the dishes pile up I can do them all in a batch. That’s less time filling the sink!
GC pauses impact HTTP performance similarly. The per-call latency grows but the overall work is lower because garbage is collected in bigger batches.
Actual GC Advice
When you’re operating a whole bunch of computers sometimes you’ll see latency suffer due to GC pauses. Google and StackOverflow offer many pages describing magic parameters for tuning the garbage collector.
Don’t tune the garbage collector. Tune the workload instead.
The easiest way to do this is to distribute the work across more computers. If you’ve got a fancy-enough load balancer, adding more computers might be sufficient. Otherwise you might need to do computer science and software engineering to decompose your work so it can run in parallel.
If you monitor your program’s GC pauses, you’ll be able to scale it up before you run into problems.
And if you’re making so many sandwiches that you can’t keep up with the dishes, hire help: you’re running a restaurant!