ATS does not propagate response header if response is sent and connection closed while post data tunnel is active

[shinrich]

I've put up some fixes for this, but sadly was more difficult than I first thought. So I'm filing the issue, so we don't forget this issue.

In the current code ATS will not try to read from the origin until it has completed the post tunnel. If the origin gets bored and goes home, the post tunnel will fail on the write to origin. ATS will not read from the origin connection. Instead it will send back a canned 502 response.

At least one of the cases our customer see is the origin returning 413 Request Entity Too Large. Moving forward with HTTP/2 not supporting 100-continue, I expect we will run into this case more often.

I tried to solve this by launching a server response header read in parallel with the post tunnel. But tracking down all the edge cases was causing a whack-a-mole of crashes. Perhaps we should use the option to read the entire post body into memory. Then read from origin while writing from buffer. No tunnel, maybe this is easier to handle.

Alternatively, maybe from the tunnel failure, we can try to read from the origin assuming the socket has not yet been closed.

I had to move onto other things, but we will need to deal with this eventually.