Initialize Channel queue capacity lazily to conserve memory

[jgaskins]

For some use cases, initializing a channel buffer eagerly (as it is currently implemented) can avoid reallocs, resulting in great performance, especially for smaller ephemeral channels that may very well use the full channel depth. Unfortunately, the tradeoff is a lot of memory usage for other use cases.

For example, in my NATS client, a Channel(NATS::Message) is used to pipe messages off the wire to a message subscriber. The capacity of this channel must be large enough to handle spikes (if not, NATS disconnects the client). The Go client sets the channel capacity to 512k per subscriber. I use 64k for the Crystal client, and even that uses a lot of memory.

To illustrate, I set up 1k subscriptions (simulating duplicate subscriptions for concurrent message handling, underlying subscriptions as the implementation of higher-level messaging operations inside the client, and general sprawl of subscribers):

require "nats"

nats = NATS::Client.new
at_exit { nats.close }

1_000.times do
  nats.subscribe "foo" do |msg|
    pp msg
  end
end
nats.flush

gets

It uses 4GB of RAM:

With this patch, the same code uses 99.5% less memory:

Memory will grow in size up to the high water mark as messages get buffered during normal execution, but almost none of these channels will ever be saturated and many may never use the buffer at all, so fully allocating the capacity for every single channel can be extreme overkill.

[straight-shoota]

There is an issue about this: #11572. We should continue the general discussion there before considering this PR.
Please add a comment with your reasoning to the issue.