TidHTTP TransferEncoding - Chunked

[msd 5]

Hello,

 

I have a little bit specific situation with one web service,

 

It receives HTTP requests over the HTTP 1.0 protocol but the problem is when TransferEncoding is Chunked.

 

Can I force TidHTTP not to send TransferEncoding is Chunked?

 

I try with 
HTTPClient.Request.TransferEncoding := 'gzip, deflate, br'; 
but I get chunked on the webserver side.

[Angus Robertson 574]

Chunking an HTTP response is a server decision, there is no way a client can stop it.  Some web applications are simply written to return the response as lots of small chunks instead of building the response locally and sending it once the length is known. 

 

One web site I access daily recently changed so that all file downloads now come as a single chunk, this is annoying since there is no HTTP content size header and you've no idea how much is arriving. 

 

The only real solution is to access the site through an HTTP proxy that unchunks the response and fowards it as sized content, the ICS proxy does this. 

 

Angus

 

[msd 5]

Hello Angus,

 

Could you please, if you have little free time, put here some sample pieces of source code for posting data / JSON in a way that you were described with ICS?

 

Thanks for any assistance and support in advance...

[Angus Robertson 574]

I thought you were writing an HTTP client, not a server, how would code to create chunks help you? 

 

ICS does not contain any server code or samples to create chunked data, it is normally created by the application. 

 

The ICS HTTP client and HTTP proxy both unchunk received data into a stream.  The point I was making was the proxy component can accept chunked data and forward it as unchunked, which I thought was your original question.  I'd have thought the Indy client would unchunk as well. 

 

Angus

 

[Remy Lebeau 1393]

On 2/24/2022 at 5:10 AM, msd said:

Can I force TidHTTP not to send TransferEncoding is Chunked?

TIdHTTP only supports RECEIVING chunked data, it does not SEND chunked data (you would have to chunk the data manually). And, even if it did support SENDING chunked data, it certainly would not do that in an HTTP 1.0 request, since chunking didn't exist until HTTP 1.1.  And per the HTTP 1.1 spec, clients are discouraged from SENDING chunked requests unless they know beforehand via outside means that the server can actually receive chunked requests,

Quote

I try with 
HTTPClient.Request.TransferEncoding := 'gzip, deflate, br'; 
but I get chunked on the webserver side.

That is not possible with TIdHTTP alone.  Something else must be going on, for instance if your request is passing through a proxy that reformats your request into a chunked request.

 

You need to explain your situation better, and provide actual logs of the HTTP traffic you are seeing, if you can.

Edited February 25, 2022 by Remy Lebeau

[Remy Lebeau 1393]

8 hours ago, Angus Robertson said:

I'd have thought the Indy client would unchunk as well.

It does unchunk a chunked response, yes.  It does not send chunked requests, though.