Author: Michael Knyszek
Updated: 8 February 2021
Go's tracing garbage collector runs concurrently with the application, and thus requires an algorithm to determine when to start a new cycle. In the runtime, this algorithm is referred to as the pacer. Until now, the garbage collector has framed this process as an optimization problem, utilizing a proportional controller to achieve a desired stopping-point (that is, the cycle completes just as the heap reaches a certain size) as well as a desired CPU utilization. While this approach has served Go well for a long time, it has accrued many corner cases due to resolved issues, as well as a backlog of unresolved issues.
I propose redesigning the garbage collector's pacer from the ground up to capture the things it does well and eliminate the problems that have been discovered. More specifically, I propose:
- Including all non-heap sources of GC work (stacks, globals) in pacing decisions.
- Reframing the pacing problem as a search problem,
- Extending the hard heap goal to the worst-case heap goal of the next GC,
(1) will resolve long-standing issues with small heap sizes, allowing the Go
garbage collector to scale down and act more predictably in general.
(2) will eliminate offset error present in the current design, will allow
turning off GC assists in the steady-state, and will enable clearer designs for
setting memory limits on Go applications.
(3) will enable smooth and consistent response to large changes in the live heap
size with large GOGC values.
Since version 1.5 Go has had a tracing mark-sweep garbage collector (GC) that is able to execute concurrently with user goroutines. The garbage collector manages several goroutines called "workers" to carry out its task. A key problem in concurrent garbage collection is deciding when to begin, such that the work is complete "on time." Timeliness, today, is defined by the optimization of two goals:
- The heap size relative to the live heap at the end of the last cycle, and
- A target CPU utilization for the garbage collector while it is active.
These two goals are tightly related. If a garbage collection cycle starts too late, for instance, it may consume more CPU to avoid missing its target. If a cycle begins too early, it may end too early, resulting in GC cycles happening more often than expected.
Go's garbage collector sets a fixed target of 30% CPU utilization (25% from GC
workers, with 5% from user goroutines donating their time to assist) while the
GC is active.
It also offers a parameter to allow the user to set their own memory use and CPU
trade-off: GOGC.
GOGC is a percent overhead describing how much additional memory (over the
live heap) the garbage collector may use.
A higher GOGC value indicates that the garbage collector may use more memory,
setting the target heap size higher, and conversely a lower GOGC value sets
the target heap size lower.
The process of deciding when a garbage collection should start given these
parameters has often been called "pacing" in the Go runtime.
To attempt to reach its goals, Go's "pacer" utilizes a proportional controller to decide when to start a garbage collection cycle. The controller attempts to find the correct point to begin directly, given an error term that captures the two aforementioned optimization goals.
It's worth noting that the optimization goals are defined for some steady-state. Today, the steady-state is implicitly defined as: constant allocation rate, constant heap size, and constant heap composition (hence, constant mark rate). The pacer expects the application to settle on some average global behavior across GC cycles.
However, the GC is still robust to transient application states. When the GC is in some transient state, the pacer is often operating with stale information, and is actively trying to find the new steady-state. To avoid issues with memory blow-up, among other things, the GC makes allocating goroutines donate their time to assist the garbage collector, proportionally to the amount of memory that they allocate. This GC assist system keeps memory use stable in unstable conditions, at the expense of user CPU time and latency.
The GC assist system operates by dynamically computing an assist ratio. The assist ratio is the slope of a curve in the space of allocation time and GC work time, a curve that the application is required to stay under. This assist ratio is then used as a conversion factor between the amount a goroutine has allocated, and how much GC assist work it should do. Meanwhile, GC workers generate assist credit from the work that they do and place it in a global pool that allocating goroutines may steal from to avoid having to assist.
Since version 1.5, the pacer has gone through several minor tweaks and changes in order to resolve issues, usually adding special cases and making its behavior more difficult to understand, though resolving the motivating problem. Meanwhile, more issues have been cropping up that are diagnosed but more difficult to tackle in the existing design. Most of these issues are listed in the GC pacer meta-issue.
Even more fundamentally, the proportional controller at the center of the pacer is demonstrably unable to completely eliminate error in its scheduling, a well-known issue with proportional-only controllers.
Another significant motivator, beyond resolving latent issues, is that the Go
runtime lacks facilities for dealing with finite memory limits.
While the GOGC mechanism works quite well and has served Go for a long time,
it falls short when there's a hard memory limit.
For instance, a consequence of GOGC that often surprises new gophers coming
from languages like Java, is that if GOGC is 100, then Go really needs 2x more
memory than the peak live heap size.
The garbage collector will not automatically run more aggressively as it
approaches some memory limit, leading to out-of-memory errors.
Conversely, users that know they'll have a fixed amount of memory up-front are
unable to take advantage of it if their live heap is usually small.
Users have taken to fooling the GC into thinking more memory is live than their
application actually needs in order to let the application allocate more memory
in between garbage collections.
Simply increasing GOGC doesn't tend to work in this scenario either because of
the previous problem: if the live heap spikes suddenly, GOGC will result in
much more memory being used overall.
See issue #42430 for more details.
The current pacer is not designed with these use-cases in mind.
There is some nuance to these definitions.
Firstly, 
is used in place of GOGC because it makes
the math easier to understand.
Secondly, 
may vary throughout the sweep phase, but
effectively becomes fixed once a GC cycle starts.
Stacks may not shrink, only grow during this time, so there's a chance any value
used by the runtime during a GC cycle will be stale.
also includes space that may not be actively used for
the stack.
That is, if an 8 KiB goroutine stack is actually only 2 KiB high (and thus only
2 KiB is actually scannable), for consistency's sake the stack's height will be
considered 8 KiB.
Both of these estimates introduce the potential for skew.
In general, however, stacks are roots in the GC and will be some of the first
sources of work for the GC, so the estimate should be fairly close.
If that turns out not to be true in practice, it is possible, though tricky to
track goroutine stack heights more accurately, though there must necessarily
always be some imprecision because actual scannable stack height is rapidly
changing.
Thirdly, 
acts similarly to .
The amount of global memory in a Go program can change while the application is
running because of the plugin package.
This action is relatively rare compared to a change in the size of stacks.
Because of this rarity, I propose allowing a bit of skew.
At worst (as we'll see later) the pacer will overshoot a little bit.
Lastly, 
is the amount of heap memory known to be live
to the runtime the instant after a garbage collection cycle completes.
Intuitively, it is the bottom of the classic GC sawtooth pattern.
Like in the previous definition of the
pacer,
the runtime sets some target heap size for the GC cycle based on GOGC.
Intuitively, this target heap size is the targeted heap size at the top of the
classic GC sawtooth pattern.
The definition I propose is very similar, except it includes non-heap sources of
GC work.
Let 
be the heap goal for GC 
("N"
stands for "Next GC").
The old definition makes the assumption that non-heap sources of GC work are negligible. In practice, that is often not true, such as with small heaps. This definition says that we're trading off not just heap memory, but all memory that influences the garbage collector's CPU consumption.
From a philospical standpoint wherein GOGC is intended to be a knob
controlling the trade-off between CPU resources and memory footprint, this
definition is more accurate.
This change has one large user-visible ramification: the default GOGC, in most
cases, will use slightly more memory than before.
This change will inevitably cause some friction, but I believe the change is worth it. It unlocks the ability to scale down to heaps smaller than 4 MiB (the origin of this limit is directly tied to this lack of accounting). It also unlocks better behavior in applications with many, or large goroutine stacks, or very many globals. That GC work is now accounted for, leading to fewer surprises.
Unlike the current pacer, I propose that instead of finding the right point to start a GC such that the runtime reaches some target in the steady-state, that the pacer instead searches for a value that is more fundamental, though more indirect.
Before continuing I want take a moment to point out some very fundamental and necessary assumptions made in both this design and the current pacer. Here, we are taking a "macro-economic" view of the Go garbage collector. The actual behavior of the application at the "micro" level is all about individual allocations, but the pacer is concerned not with the moment-to-moment behavior of the application. Instead, it concerns itself with broad aggregate patterns. And evidently, this abstraction is useful. Most programs are not wildly unpredictable in their behavior; in fact it's somewhat of a challenge to write a useful application that non-trivially has unpredictable memory allocation behavior, thanks to the law of large numbers. This observation is why it is useful to talk about the steady-state of an application at all.
The pacer concerns itself with two notions of time: the time it takes to allocate from the GC trigger point to the heap goal and the time it takes to find and perform all outstanding GC work. These are only notions of time because the pacer's job is to make them happen in the same amount of time, relative to a wall clock. Since in the steady-state the amount of GC work (broadly speaking) stays fixed, the pacer is then concerned with figuring out how early it should start such that it meets its goal. Because they should happen in the same amount of time, this question of "how early" is answered in "bytes allocated so far."
So what's this more fundamental value? Suppose we model a Go program as such:
the world is split in two while a GC is active: the application is either
spending time on itself and potentially allocating, or on performing GC work.
This model is "zero-sum," in a sense: if more time is spent on GC work, then
less time is necessarily spent on the application and vice versa.
Given this model, suppose we had two measures of program behavior during a GC
cycle: how often the application is allocating, and how rapidly the GC can scan
and mark memory.
Note that these measure are not peak throughput.
They are a measure of the rates, in actuality, of allocation and GC work happens
during a GC.
To give them a concrete unit, let's say they're bytes per cpu-seconds per core.
The idea with this unit is to have some generalized, aggregate notion of this
behavior, independent of available CPU resources.
We'll see why this is important shortly.
Lets call these rates 
and 
respectively.
In the steady-state, these rates aren't changing, so we can use them to predict
when to start a garbage collection.
Coming back to our model, some amount of CPU time is going to go to each of
these activities.
Let's say our target GC CPU utilization in the steady-state is
.
If 
is the number of CPU cores available and
is some wall-clock time window, then
bytes will be allocated and 
bytes will be scanned in that window.
Notice that ratio of "bytes allocated" to "bytes scanned" is constant in the
steady-state in this model, because both and
are constant.
Let's call this ratio 
.
To make things a little more general, let's make also a
function of utilization 
, because part of the Go garbage
collector's design is the ability to dynamically change CPU utilization to keep
it on-pace.
The big idea here is that this value, 
is a
conversion rate between these two notions of time.
Consider the following: in the steady-state, the runtime can perfectly back out
the correct time to start a GC cycle, given that it knows exactly how much work
it needs to do.
Let 
be the trigger point for GC cycle
.
Let 
be the size of the live scannable heap at the
end of GC .
More precisely, is the subset of
that contains pointers.
Why include only pointer-ful memory? Because GC work is dominated by the cost of
the scan loop, and each pointer that is found is marked; memory containing Go
types without pointers are never touched, and so are totally ignored by the GC.
Furthermore, this does include non-pointers in pointer-ful memory, because
scanning over those is a significant cost in GC, enough so that GC is roughly
proportional to it, not just the number of pointer slots.
In the steady-state, the size of the scannable heap should not change, so
remains constant.
That's nice, but we don't know while the runtime is
executing.
And worse, it could change over time.
But if the Go runtime can somehow accurately estimate and predict
then it can find a steady-state.
Suppose we had some prediction of for GC cycle
called 
.
Then, our trigger condition is a simple extension of the formula above.
Let 
be the size of the Go live heap at any given time.
is thus monotonically increasing during a GC cycle, and
then instantaneously drops at the end of the GC cycle.
In essence is the classic GC sawtooth pattern.
Note that this formula is in fact a condition and not a predetermined trigger
point, like the trigger ratio.
In fact, this formula could transform into the previous formula for
if it were not for the fact that
actively changes during a GC cycle, since the rest of
the values are constant for each GC cycle.
A big question remains: how do we predict ?
To answer that, we first need to determine how to measure
at all.
I propose a straightforward approximation: each GC cycle, take the amount of
memory allocated, divide it by the amount of memory scanned, and scale it from
the actual GC CPU utilization to the target GC CPU utilization.
Note that this scaling factor is necessary because we want our trigger to use an
value that is at the target utilization, such that the
GC is given enough time to only use that amount of CPU.
This note is a key aspect of the proposal and will come up later.
What does this scaling factor look like? Recall that because of our model, any
value of has a 
factor in the
numerator and a factor in the denominator.
Scaling from one utilization to another is as simple as switching out factors.
Let 
be the actual peak live heap size at the end
of a GC cycle (as opposed to , which is only a target).
Let 
be the GC CPU utilization over cycle
and be the target utilization.
Altogether,
Now that we have a way to measure , we could use this
value directly as our prediction.
But I fear that using it directly has the potential to introduce a significant
amount of noise, so smoothing over transient changes to this value is desirable.
To do so, I propose using this measurement as the set-point for a
proportional-integral (PI) controller.
This means that the PI controller is always chasing whatever value was measured
for each GC cycle.
In a steady-state with only small changes, this means the PI controller acts as
a smoothing function.
The advantage of a PI controller over a proportional controller is that it guarantees that steady-state error will be driven to zero. Note that the current GC pacer has issues with offset error. It may also find the wrong point on the isocline of GC CPU utilization and peak heap size because the error term can go to zero even if both targets are missed. The disadvantage of a PI controller, however, is that it oscillates and may overshoot significantly on its way to reaching a steady value. This disadvantage could be mitigated by overdamping the controller, but I propose we tune it using the tried-and-tested standard Ziegler-Nichols method. In simulations (see the simulations section) this tuning tends to work well. It's worth noting that PI (more generally, PID controllers) have a lot of years of research and use behind them, and this design lets us take advantage of that and tune the pacer further if need be.
Why a PI controller and not a PID controller? Firstly, PI controllers are simpler to reason about. Secondly, the derivative term in a PID controller tends to be sensitive to high-frequency noise, and the entire point here is to smooth over noise. Furthermore, the advantage of the derivative term is a shorter rise time, but simulations show that the rise time is roughly 1 GC cycle, so I don't think there's much reason to include it. Adding the derivative term though is trivial once the rest of the design is in place, so the door is always open.
By focusing on this value, we've now reframed the pacing
problem as a search problem instead of an optimization problem.
That raises question: are we still reaching our optimization goals? And how do
GC assists fit into this picture?
The good news is that we're always triggering for the right CPU utilization.
Because being scaled for the target GC CPU utilization
and picks the trigger, the pacer will naturally start at
a point that will generate a certain utilization in the steady-state.
Following from this fact, there is no longer any reason to have the target GC CPU utilization be 30%. Originally, in the design for the current pacer, the target GC CPU utilization, began at 25%, with GC assists always extending from that, so in the steady-state there would be no GC assists. However, because the pacer was structured to solve an optimization problem, it required feedback from both directions. That is, it needed to know whether it was actinng too aggressively or not aggressively enough. This feedback could only be obtained by actually performing GC assists. But with this design, that's no longer necessary. The target CPU utilization can completely exclude GC assists in the steady-state with a mitigated risk of bad behavior.
As a result, I propose the target utilization be reduced once again to 25%, eliminating GC assists in the steady-state (that's not out-pacing the GC), and potentially improving application latency as a result.
Idle-priority GC workers are extra workers that run if the application isn't
utilizing all GOMAXPROCS worth of parallelism.
The scheduler schedules "low priority" background workers on any additional CPU
resources, and this ultimately skews utilization measurements in the GC.
In today's pacer, they're somewhat accounted for.
In general, the idle workers skew toward undershoot: because the pacer is not
explicitly aware of the idle workers, GC cycles will complete sooner than it
might expect.
However if a GC cycle completes early, then in theory the current pacer will
simply adjust.
From its perspective, scanning and marking objects was just faster than
expected.
The new proposed design must also account for them somehow.
I propose including idle priority worker CPU utilization in both the measured
utilization, , and the target utilization,
, in each GC cycle.
In this way, the only difference between the two terms remains GC assist
utilization, and while idle worker CPU utilization remains stable, the pacer may
effectively account for it.
Unfortunately this does mean idle-priority GC worker utilization becomes part of the definition of a GC steady-state, making it slightly more fragile. The good news is that that was already true. Due to other issues with idle-priority GC workers, it may be worth revisiting them as a concept, but they do appear to legitimately help certain applications, particularly those that spend a significant chunk of time blocked on I/O, yet spend some significant chunk of their time awake allocating memory.
This discussion of GC assists brings us to the existing issues around pacing decisions made while the GC is active (which I will refer to as the "GC assist pacer" below). For the most part, this system works very well, and is able to smooth over small hiccups in performance, due to noise from the underlying platform or elsewhere.
Unfortunately, there's one place where it doesn't do so well: the hard heap goal. Currently, the GC assist pacer prevents memory blow-up in pathological cases by ramping up assists once either the GC has found more work than it expected (i.e. the live scannable heap has grown) or the GC is behind and the application's heap size has exceeded the heap goal. In both of these cases, it sets a somewhat arbitrarily defined hard limit at 1.1x the heap goal.
The problem with this policy is that high GOGC values create the opportunity
for very large changes in live heap size, because the GC has quite a lot of
runway (consider an application with GOGC=51100 has a steady-state live heap
of size 10 MiB and suddenly all the memory it allocates is live).
In this case, the GC assist pacer is going to find all this new live memory and
panic: the rate of assists will begin to skyrocket.
This particular problem impedes the adoption of any sort of target heap size, or
configurable minimum heap size.
One can imagine a small live heap with a large target heap size as having a
large effective GOGC value, so it reduces to exactly the same case.
To deal with this, I propose modifying the GC assist policy to set a hard heap
goal of 
.
The intuition behind this goal is that if all the memory allocated in this GC
cycle turns out to be live, the next GC cycle will end up using that much
memory anyway, so we let it slide.
But this hard goal need not be used for actually pacing GC assists other than in extreme cases. In fact, it must not, because an assist ratio computed from this hard heap goal and the worst-case scan work turns out to be extremely loose, leading to the GC assist pacer consistently missing the heap goal in some steady-states.
So, I propose an alternative calculation for the assist ratio. I believe that the assist ratio must always pass through the heap goal, since otherwise there's no guarantee that the GC meets its heap goal in the steady-state (which is a fundamental property of the pacer in Go's existing GC design). However, there's no reason why the ratio itself needs to change dramatically when there's more GC work than expected. In fact, the preferable case is that it does not, because that lends itself to a much more even distribution of GC assist work across the cycle.
So, I propose that the assist ratio be an extrapolation of the current steady-state assist ratio, with the exception that it now include non-heap GC work as the rest of this document does.
That is,
This definition is intentially roundabout. The assist ratio changes dynamically as the amount of GC work left decreases and the amount of memory allocated increases. This responsiveness is what allows the pacing mechanism to be so robust.
Today, the assist ratio is calculated by computing the remaining heap runway and the remaining expected GC work, and dividing the former by the latter. But of course, that's not possible if there's more GC work than expected, since then the assist ratio could go negative, which is meaningless.
So that's the purpose defining the "max scan work" and "extrapolated runway": these are worst-case values that are always safe to subtract from, such that we can maintain roughly the same assist ratio throughout the cycle (assuming no hiccups).
One minor details is that the "extrapolated runway" needs to be capped at the hard heap goal to prevent breaking that promise, though in practice this will almost. The hard heap goal is such a loose bound that it's really only useful in pathological cases, but it's still necessary to ensure robustness.
A key point in this choice is that the GC assist pacer will only respond to changes in allocation behavior and scan rate, not changes in the size of the live heap. This point seems minor, but it means the GC assist pacer's function is much simpler and more predictable.
Not every problem listed in issue #42430 is resolved by this design, though many are.
Notable exclusions are:
- Mark assists are front-loaded in a GC cycle.
- The hard heap goal isn't actually hard in some circumstances.
- Existing trigger limits to prevent unbounded memory growth.
(1) is difficult to resolve without special cases and arbitrary heuristics, and I think in practice it's OK; the system was fairly robust and will now be more so to this kind of noise. That doesn't mean that it shouldn't be revisited, but it's not quite as big as the other problems, so I leave it outside the scope of this proposal.
(2) is also tricky and somewhat orthogonal. I believe the path forward there involves better scheduling of fractional GC workers, which are currently very loosely scheduled. This design has made me realize how important dedicated GC workers are to progress, and how GC assists are a back-up mechanism. I believe that the fundamental problem there lies with the fact that fractional GC workers don't provide that sort of consistent progress.
For (3), I propose we retain the limits, translated to the current design.
For reference, these limits are 
as the
upper-bound on the trigger ratio, and 
as
the lower-bound.
The upper bound exists to prevent ever starting the GC too late in low-activity scenarios. It may cause consistent undershoot, but prevents issues in GC pacer calculations by preventing the calculated runway from ever being too low. The upper-bound may need to be revisited when considering a configurable target heap size.
The lower bound exists to prevent the application from causing excessive memory growth due to floating garbage as the application's allocation rate increases. Before that limit was installed, it wasn't very easy for an application to allocate hard enough for that to happen. The lower bound probably should be revisited, but I leave that outside of the scope of this document.
To translate them to the current design, I propose we simply modify the trigger condition to include these limits. It's not important to put these limits in the rest of the pacer because it no longer tries to compute the trigger point ahead of time.
Like today, the pacer has to start somewhere for the first GC.
I propose we carry forward what we already do today: set the trigger point at
7/8ths of the first heap goal, which will always be the minimum heap size.
If GC 1 is the first GC, then in terms of the math above, we choose to avoid
defining 
, and instead directly define
The definition of 
is necessary for the GC assist
pacer.
Furthermore, the PI controller's state will be initialized to zero otherwise.
These choices are somewhat arbitrary, but the fact is that the pacer has no knowledge of the progam's past behavior for the first GC. Naturally the behavior of the GC will always be a little odd, but it should, in general, stabilize quite quickly (note that this is the case in each scenario for the simulations.
This document uses the term "GC CPU utilization" quite frequently, but so far has refrained from defining exactly how it's measured. Before doing that, let's define CPU utilization over a GC mark phase, as it's been used so far. First, let's define the mark phase: it is the period of wall-clock time between the end of sweep termination and the start of mark termination. In the mark phase, the process will have access to some total number of CPU-seconds of execution time. This CPU time can then be divided into "time spent doing GC work" and "time spent doing anything else." GC CPU utilization is then defined as a proportion of that total CPU time that is spent doing GC work.
This definition seems straightforward enough but in reality it's more
complicated.
Measuring CPU time on most platforms is tricky, so what Go does today is an
approximation: take the wall-clock time of the GC mark phase, multiply it by
GOMAXPROCS.
Call this 
.
That's our GC CPU utilization.
This approximation is mostly accurate in the common case, but is prone to skew in various scenarios, such as when the system is CPU-starved. This fact can be problematic, but I believe it is largely orthogonal to the content of this document; we can work on improving this approximation without having to change any of this design. It already assumes that we have a good measure of CPU utilization.
The alternatives considered for this design basically boil down to its individual components.
For instance, I considered grouping stacks and globals into the current formulation of the pacer, but that adds another condition to the definition of the steady-state: stacks and globals do not change. That makes the steady-state more fragile.
I also considered a design that was similar, but computed everything in terms of
an "effective" GOGC, and "converted" that back to GOGC for pacing purposes
(that is, what would the heap trigger have been had the expected amount of live
memory been correct?).
This formulation is similar to how Austin formulated the experimental
SetMaxHeap API.
Austin suggested I avoid this formulation because math involving GOGC tends to
have to work around infinities.
A good example of this is if runtime.GC is called while GOGC is off: the
runtime has to "fake" a very large GOGC value in the pacer.
By using a ratio of rates that's more grounded in actual application behavior
the trickiness of the math is avoided.
I also considered not using a PI controller and just using the measured
value directly, assuming it doesn't change across GC
cycles, but that method is prone to noise.
Pros:
- The steady-state is now independent of the amount of GC work to be done.
- Steady-state mark assist drops to zero if not allocating too heavily (a likely
latency improvement in many scenarios) (see the "high
GOGC" scenario in simulations). - GC amortization includes non-heap GC work, and responds well in those cases.
- Eliminates offset error present in the existing design.
Cons:
- Definition of
GOGChas changed slightly, so aGOGCof 100 will use slightly more memory in nearly all cases. 
is a little bit unintuitive.
This pacer redesign will be implemented by Michael Knyszek.
- The existing pacer will be refactored into a form fit for simulation.
- A comprehensive simulation-based test suite will be written for the pacer.
- The existing pacer will be swapped out with the new implementation.
The purpose of the simulation infrastructure is to make the pacer, in general, more testable. This lets us write regression test cases based on real-life problems we encounter and confirm that they don't break going forward. Furthermore, with fairly large changes to the Go compiler and runtime in the pipeline, it's especially important to reduce the risk of this change as much as possible.
This change will not modify any user-visible APIs, but may have surprising
effects on application behavior.
The two "most" incompatible changes in this proposal are the redefinition of the
heap goal to include non-heap sources of GC work, since that directly influences
the meaning of GOGC, and the change in target GC CPU utilization.
These two factors together mean that, by default and on average, Go applications
will use slightly more memory than before.
To obtain previous levels of memory usage, users may be required to tune down
GOGC lower manually, but the overall result should be more consistent, more
predictable, and more efficient.
In order to show the effectiveness of the new pacer and compare it to the current one, I modeled both the existing pacer and the new pacer and simulated both in a variety of scenarios.
The code used to run these simulations and generate the plots below may be found at github.com/mknyszek/pacer-model.
The model of each pacer is fairly detailed, and takes into account most details like allocations made during a GC being marked. The one big assumption it makes, however, is that the behavior of the application while a GC cycle is running is perfectly smooth, such that the GC assist pacer is perfectly paced according to the initial assist ratio. In practice, this is close to true, but it's worth accounting for the more extreme cases. (TODO: Show simulations that inject some noise into the GC assist pacer.)
Another caveat with the simulation is the graph of "R value" (that is,
), and "Alloc/scan ratio."
The latter is well-defined for all simulations (it's a part of the input) but
the former is not a concept used in the current pacer.
So for simulations of the current pacer, the "R value" is backed out from the
trigger ratio: we know the runway, we know the expected scan work for the
target utilization, so we can compute the that the
trigger point encodes.
Perfectly steady heap size.
The simplest possible scenario.
Current pacer:
New pacer:
Notes:
- The current pacer doesn't seem to find the right utilization.
- Both pacers do reasonably well at meeting the heap goal.
Jittery heap size and alloc/scan ratio.
A mostly steady-state heap with a slight jitter added to both live heap size and the alloc/scan ratio.
Current pacer:
New pacer:
Notes:
- Both pacers are resilient to a small amount of noise.
Small step in alloc/scan ratio.
This scenario demonstrates the transitions between two steady-states, that are not far from one another.
Current pacer:
New pacer:
Notes:
- Both pacers react to the change in alloc/scan rate.
- Clear oscillations in utilization visible for the new pacer.
Large step in alloc/scan ratio.
This scenario demonstrates the transitions between two steady-states, that are further from one another.
Current pacer:
New pacer:
Notes:
- The old pacer consistently overshoots the heap size post-step.
- The new pacer minimizes overshoot.
Large step in heap size with a high GOGC value.
This scenario demonstrates the "high GOGC problem" described in the GC pacer
meta-issue.
Current pacer:
New pacer:
Notes:
- The new pacer's heap size stabilizes faster than the old pacer's.
- The new pacer has a spike in overshoot; this is by design.
- The new pacer's utilization is independent of this heap size spike.
- The old pacer has a clear spike in utilization.
Oscillating alloc/scan ratio.
This scenario demonstrates an oscillating alloc/scan ratio. This scenario is interesting because it shows a somewhat extreme case where a steady-state is never actually reached for any amount of time. However, this is not a realistic scenario.
Current pacer:
New pacer:
Notes:
- The new pacer tracks the oscillations worse than the old pacer. This is likely due to the error never settling, so the PI controller is always overshooting.
Large amount of goroutine stacks.
This scenario demonstrates the "heap amortization problem" described in the GC pacer meta-issue for goroutine stacks.
Current pacer:
New pacer:
Notes:
- The old pacer consistently overshoots because it's underestimating the amount of work it has to do.
- The new pacer uses more memory, since the heap goal is now proportional to stack space, but it stabilizes and is otherwise sane.
Large amount of global variables.
This scenario demonstrates the "heap amortization problem" described in the GC pacer meta-issue for global variables.
Current pacer:
New pacer:
Notes:
- This is essentially identical to the stack space case.
High alloc/scan ratio.
This scenario shows the behavior of each pacer in the face of a very high alloc/scan ratio, with jitter applied to both the live heap size and the alloc/scan ratio.
Current pacer:
New pacer:
Notes:
- In the face of a very high allocation rate, the old pacer consistently overshoots, though both maintain a similar GC CPU utilization.