Go goroutine leaks and pprofchannels, mutexes, and exit paths
Starting a goroutine is easy. The hard part was deciding who changes the data, who finishes the work, and when a function may return.
Starting a goroutine is easy. The hard part was deciding who changes the data, who finishes the work, and when a function may return.
In the previous post, total allocations stayed at 256MiB however I set GOGC and GOMEMLIMIT. Reusing short-lived buffers with sync.Pool is one way to shrink it.
The container limit was 1GB, yet the process was OOMKilled with an 800MB heap. Go has a garbage collector, so how did memory run out?
Put go in front of a call and it runs concurrently. So do 100,000 goroutines mean 100,000 threads? Does GOMAXPROCS=1 mean one task at a time?
Interfaces, slices, defer, embedding, context: looking back after five posts, most of the pitfalls came from two axes. One is "everything is copied"; the other is "object orientation without classes."
I thought putting a one-hour context.WithTimeout on a request meant it would wait up to an hour. It died at 30 seconds. The context made it all the way through; it just wasn't the only timer. And once I fixed the timer, the real problem turned out to be somewhere else.
I was already taking ctx as the first argument and passing it along. What I had never checked was whether canceling actually stops the goroutine. After stalling on that question in an interview, I went back to context from the start.
Validation was dying on timeouts. Within 30 minutes I had an answer, "it's N+1, batch it," and it was only half right. Reaching the final design took two changes of direction, and since the process taught me more than the code, I'm writing it down.
Go has no classes and no inheritance, yet it's hard to call it non-object-oriented. The official FAQ answers "Is Go an object-oriented language?" with "Yes and no." This post explains why by following what Go put where inheritance used to be.
Go dropped exceptions and filled the gap with defer and errors as values. All three take seconds to learn and keep producing traps for years. I reorganized them around one question: exactly when, and what, does each one hold on to?
The Go bugs I keep hitting come from holding something different from what I think I'm holding. A slice is a window, not data. A map doesn't guarantee addresses. A string isn't a character array. This post opens up the layout of all three.
You return a nil pointer and err != nil comes out true, because an interface value has two slots: a type and the data. From that two-slot layout to method sets, where interfaces should live, and the cost of dynamic dispatch.
Following how compiled, statically typed code turns into runtime wins, using Go: memory layout, compiler optimizations, escape analysis, the GC, and the scheduler, each checked with tools.