Jump to content
Mocte Sandoval

POSTing binary data with TIdHTTP

Recommended Posts

Posted (edited)

Hi to all

 

I don't ask here often but now I need some help with Indy ( which I don't use often either btw ).

 

The scenario: 

 

I need to upload "documents" ( mostly PDF, Tiff, Jpg ) usually from a file to a third party HTTP server, I am doing that perfectly fine with code like this ( just a snippet ) :

var
  Params: TIdMultipartFormDataStream;
Begin
  ...
    Params.AddFile(fileName, fullFilePath, 'application/octet-stream');
    http.Post('http://x.x.x.x:8996/service', Params, responseStream);
  ...
End

The endpoint from the server side receives the file and answers with code 201 which is expected.

 

Now the problem, there are a lot of these "documents" stored on files on disk, each of these files contain a bunch of "documents" on a propietary format, which I'm succesfully extracting as an array of bytes, I can save them as individual files with TFileStream and view them with external tools, but I need to send these individual files to the http server, so for speeding up the process I want to avoid saving them to disk, so what i would like is to send the array of bytes straight to the server, here is an snippet from the code I am using so far:

procedure SendFromBytes( const buffer: array of byte );
var
  ...
  data, 
  idBoundary: String;
  ...
Begin
  ...
     idBoundary := '---someboundaryId';

      StreamResp := TStringStream.Create('',TEncoding.UTF8);
      streamSrc  := TMemoryStream.create;

      http.Request.ContentType := sContentTypeFormData + idBoundary;

      data := idBoundary + CRLF +
        'Content-Disposition: form-data; name="'+name+'"' + CRLF + CRLF;
      streamSrc.Write(data[1], Length(data));
      streamSrc.WriteBuffer(Buffer[0], length(Buffer));

      data := CRLF +  idBoundary;
      streamSrc.Write(data[1], Length(data));
      http.Post('http://x.x.x.x:8996/service', StreamSrc, streamResp);
...
End;

But here I'm getting 400 Bad request, so I'm not sending the message as is expected by the server 

 

Is there an Indy guru here who can help me find the cause? 

 

Edit: I'm on Delphi XE3

Edit 2: spelling on subject

Edited by Mocte Sandoval

Share this post


Link to post
1 hour ago, Attila Kovacs said:

What happens if you set the stream's position to zero before posting?

Just tested with the same result (400),  I didn't show it here but I have a TIdLogFile for logging the message, the data seems to be all in place, I suspect I am constructing the message in some way different from how TIdMultipartFormDataStream is processed by TidHttp and therefore it is possible I'm not sticking to the http spec for a multipart/form-data post but I'm failing to see where.
 

Share this post


Link to post
Posted (edited)

Hm. Maybe the missing closing '--' suffix on the last boundary.

Edited by Attila Kovacs

Share this post


Link to post
Posted (edited)

Thank you Attila, I stripped that in my search of the correct way, added again with no luck.

 

I'm attaching a sample of what the log looks like, I deleted a great part of its contents because the documents are confidential data but maybe some expert eyes can notice something I'm missing.

 

BTW I think I missed the last line on the log  {"error":"multipart: NextPart: EOF"}  definitely there is something missing there.

test.txt

Edited by Mocte Sandoval
spelling

Share this post


Link to post
Posted (edited)

umm, if  "boundary=---2-1d1a304dc9" in the header, shouldn't be the boundary -----2-1d1a304dc9 in the body? And closing with -----2-1d1a304dc9--?

Or just omit a -- in the header.

Edited by Attila Kovacs

Share this post


Link to post
Posted (edited)

Ok, I got rid of all initial "-"  so boundary is now e.g. boundary=4024b236a3a1 on header,  --4024b236a3a1 in body, and closing --4024b236a3a1--

 

And all is working great now ! 

 

Thank you so much Attila this was driving me nuts. 

 

Edit: How can I mark this topic as resolved?

Edited by Mocte Sandoval
Mark as solved

Share this post


Link to post

Glad to hear that.

 

3 minutes ago, Mocte Sandoval said:

How can I mark this topic as resolved?

You can't.

And you will get it resolved the second time tomorrow by others 😉

It's ok.

  • Like 1

Share this post


Link to post
Posted (edited)
6 hours ago, Mocte Sandoval said:

Now the problem, there are a lot of these "documents" stored on files on disk, each of these files contain a bunch of "documents" on a propietary format, which I'm succesfully extracting as an array of bytes, I can save them as individual files with TFileStream and view them with external tools, but I need to send these individual files to the http server, so for speeding up the process I want to avoid saving them to disk, so what i would like is to send the array of bytes straight to the server, here is an snippet from the code I am using so far

Why not continue using TIdMultipartFormDataStream for that?  It has an overloaded AddFormField() method that lets you specify a TStream for the input data.  You could store your bytes in a TMemoryStream or TBytesStream, or use Indy's own TIdMemoryBufferStream, etc.  For example:

procedure SendFromBytes( const buffer: array of byte );
var
  Strm: TIdMemoryBufferStream;
  Params: TIdMultipartFormDataStream;
begin
  ...
  Strm := TIdMemoryBufferStream.Create(PByte(buffer), Length(buffer));
  try
    Params := TIdMultipartFormDataStream.Create;
    try
      Params.AddFormField('name', 'application/octet-stream', '', Strm, 'filename');
      http.Post('http://x.x.x.x:8996/service', Params, responseStream);
    finally
    end;
  finally
    Strm.Free;
  end;
  ...
end;
36 minutes ago, Mocte Sandoval said:

Ok, I got rid of all initial "-"  so boundary is now e.g. boundary=4024b236a3a1 on header,  --4024b236a3a1 in body, and closing --4024b236a3a1--

Yes, whatever value is specified in the "boundary" header, in the body it needs to have a leading "--" in front of each new MIME part, and a trailing "--" at the end of the last MIME part.  TIdMultipartFormDataStream handles that internally for you.

Edited by Remy Lebeau

Share this post


Link to post
11 minutes ago, Remy Lebeau said:

Why not continue using TIdMultipartFormDataStream for that?  It has an overloaded AddFormField() method that lets you specify a TStream for the input data.  You could store your bytes in a TMemoryStream or TBytesStream, or use Indy's own TIdMemoryBufferStream, etc.

Yes, whatever value you specify in the "boundary" header, you need to add a leading "--" to in the body, and a trailing "--" at the end.

 

Good to know, I'll give it a try tomorrow, thank you Remy

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×