Make HTTP2 window size configurable

[FinleyMcIlwaine]

http2 uses a default connection window size of 1MB per connection and 256KB per stream, which can cause data to pile up on a receiver's heap if they can't keep up. We've observed this in the stress tests. The standard fix for this is to reduce the window size. http2 makes this configurable, although a brief test where we tried to reduce the window size on the client led to bus errors in the server. We should make this configurable in grapesy as well, and see if we can avoid/fix the bus errors.

[edsko]

I just noticed confBufferSize in the settings, maybe we should be using that instead?

[FinleyMcIlwaine]

I have confirmed that the recent patch to http-semantics fixes the bus errors/segfaults we saw here when adjusting the window size. I have also confirmed that lowering the window size does indeed lower heap residency in the receiver for our stress test.

The default connection window size in http2 is relatively large (1048576 bytes), so the stress test was is actually limited by the initial window size in Network.HTTP2.H2.Settings.defaultSettings, which is also a relatively large 262144 bytes. With those default window sizes, in our server streaming RPC stress test, we see this:

Halving that initial window size manually in grapesy with something like this in connectInsecure

...
          , HTTP2.Client.settings =
              (HTTP2.Client.settings HTTP2.Client.defaultClientConfig)
                { HTTP2.Client.initialWindowSize = 262144 `div` 2
                }
...

the residency drops:

Halving it again:

So making the http2 window size configurable does seem like it would be a good improvement.

[edsko]

Ok, awesome. Good to know that our understanding of the memory behaviour is correct now. Let's add a grapesy flag that allows to set these two parameters, and when we document memory behaviour in the yet-to-be-written manual, we should mention this.

[edsko]

Relevant: kazu-yamamoto/http2#121 (comment)

[edsko]

We need four parameters to be configurable:

• max concurrent streams
• stream window size
• connection size
• number of workers

and we should also pick our own defaults independent from the defaults from http2 (or network-control).

We need to ensure that we document in the Haddocks the relationship between these three parameters.

[edsko]

Increasing priority to high due to #169 (comment) .