Mishandling of headers timeout when sending out a big body

[vrurg]

The setup (encountered while implementing uploads for Google Cloud Storage API):

1. Instantiated Cro::HTTP::Client, HTTP 1.1, not persistent.
2. A file of at least few hundred megabytes in size. What is considered 'sufficiently big' depends on connection bandwidth.
3. Best observed with request body implemented as a supply reading and emitting file's data in chunks.
4. Setting the timeout to an unreasonable low value helps with observing the problem.

What happens:

1. The headers timeout is setup before the request is sent, i.e. before $pipeline.send-request($request-object) is done.
2. Since the outgoing transfer takes really long the time out goes off before the entire body sending is completed.
3. I'm not sure what exactly happens internally at this point, but the exception thrown doesn't cause the body supply to be aborted. An attempt to re-use the same instance of Cro::HTTP::Client for another request either resumes the previous request sending its data, or simply await until the sending is complete. Either way, I observe the body supply to continue its read/emit cycle.
4. The second request normally gets aborted with a consequent timeout exception.

So far, I'm trying to figure out a workaround which doesn't require re-instantiating the client per each API request. Instantiation is needed because Google Cloud API specifics make it necessary to remove any body serializers/parsers except for blob and supply fallbacks. Lack of knowing of Cro's guts and lack of time doesn't let me to debug deeper and, perhaps, find a solution for this. But overall the problem makes a lot of heavy-weight use cases very tricky to implement and might result users rejecting Cro as the tool of their choice.

[wbiker]

I think I run into this problem. I was using Cro::HTTP::Client to post data to a influxdb. Actually, I wanted to replace Curl with it, but run in the problem that receiving the response blocks until this exception shows up:
Exceeded timeout for headers when attempting to access http://...

It hangs when the response of the influx request is handled. The data which are transfers are no much. A few kb. TRACE shows this:

[TRACE(anon 2)] Cro::TCP::Connector EMIT TCP Message
  48 54 54 50 2f 31 2e 31 20 32 30 34 20 4e 6f 20  HTTP/1.1 204 No 
  43 6f 6e 74 65 6e 74 0d 0a 58 2d 49 6e 66 6c 75  Content..X-Influ
  78 64 62 2d 42 75 69 6c 64 3a 20 4f 53 53 0d 0a  xdb-Build: OSS..
  58 2d 49 6e 66 6c 75 78 64 62 2d 56 65 72 73 69  X-Influxdb-Versi
  6f 6e 3a 20 76 32 2e 37 2e 31 32 0d 0a 44 61 74  on: v2.7.12..Dat
  65 3a 20 46 72 69 2c 20 32 37 20 4a 75 6e 20 32  e: Fri, 27 Jun 2
  30 32 35 20 31 32 3a 34 35 3a 32 33 20 47 4d 54  025 12:45:23 GMT
  0d 0a 0d 0a                                      ....

When I create a new Cro::HTTP::Client instance for each request (thanks vrurg) it works:

[TRACE(anon 14)] Cro::TCP::Connector EMIT TCP Message
  48 54 54 50 2f 31 2e 31 20 32 30 34 20 4e 6f 20  HTTP/1.1 204 No 
  43 6f 6e 74 65 6e 74 0d 0a 58 2d 49 6e 66 6c 75  Content..X-Influ
  78 64 62 2d 42 75 69 6c 64 3a 20 4f 53 53 0d 0a  xdb-Build: OSS..
  58 2d 49 6e 66 6c 75 78 64 62 2d 56 65 72 73 69  X-Influxdb-Versi
  6f 6e 3a 20 76 32 2e 37 2e 31 32 0d 0a 44 61 74  on: v2.7.12..Dat
  65 3a 20 46 72 69 2c 20 32 37 20 4a 75 6e 20 32  e: Fri, 27 Jun 2
  30 32 35 20 31 32 3a 35 33 3a 32 34 20 47 4d 54  025 12:53:24 GMT
  0d 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c  ..Connection: cl
  6f 73 65 0d 0a 0d 0a                             ose....

However, feels a bit odd and will use curl instead again.