Mocte Sandoval 0 Posted July 9, 2019 (edited) Hi to all I don't ask here often but now I need some help with Indy ( which I don't use often either btw ). The scenario: I need to upload "documents" ( mostly PDF, Tiff, Jpg ) usually from a file to a third party HTTP server, I am doing that perfectly fine with code like this ( just a snippet ) : var Params: TIdMultipartFormDataStream; Begin ... Params.AddFile(fileName, fullFilePath, 'application/octet-stream'); http.Post('http://x.x.x.x:8996/service', Params, responseStream); ... End The endpoint from the server side receives the file and answers with code 201 which is expected. Now the problem, there are a lot of these "documents" stored on files on disk, each of these files contain a bunch of "documents" on a propietary format, which I'm succesfully extracting as an array of bytes, I can save them as individual files with TFileStream and view them with external tools, but I need to send these individual files to the http server, so for speeding up the process I want to avoid saving them to disk, so what i would like is to send the array of bytes straight to the server, here is an snippet from the code I am using so far: procedure SendFromBytes( const buffer: array of byte ); var ... data, idBoundary: String; ... Begin ... idBoundary := '---someboundaryId'; StreamResp := TStringStream.Create('',TEncoding.UTF8); streamSrc := TMemoryStream.create; http.Request.ContentType := sContentTypeFormData + idBoundary; data := idBoundary + CRLF + 'Content-Disposition: form-data; name="'+name+'"' + CRLF + CRLF; streamSrc.Write(data[1], Length(data)); streamSrc.WriteBuffer(Buffer[0], length(Buffer)); data := CRLF + idBoundary; streamSrc.Write(data[1], Length(data)); http.Post('http://x.x.x.x:8996/service', StreamSrc, streamResp); ... End; But here I'm getting 400 Bad request, so I'm not sending the message as is expected by the server Is there an Indy guru here who can help me find the cause? Edit: I'm on Delphi XE3 Edit 2: spelling on subject Edited July 9, 2019 by Mocte Sandoval Share this post Link to post
Attila Kovacs 629 Posted July 9, 2019 What happens if you set the stream's position to zero before posting? Share this post Link to post
Mocte Sandoval 0 Posted July 9, 2019 1 hour ago, Attila Kovacs said: What happens if you set the stream's position to zero before posting? Just tested with the same result (400), I didn't show it here but I have a TIdLogFile for logging the message, the data seems to be all in place, I suspect I am constructing the message in some way different from how TIdMultipartFormDataStream is processed by TidHttp and therefore it is possible I'm not sticking to the http spec for a multipart/form-data post but I'm failing to see where. Share this post Link to post
Attila Kovacs 629 Posted July 9, 2019 (edited) Hm. Maybe the missing closing '--' suffix on the last boundary. Edited July 9, 2019 by Attila Kovacs Share this post Link to post
Mocte Sandoval 0 Posted July 9, 2019 (edited) Thank you Attila, I stripped that in my search of the correct way, added again with no luck. I'm attaching a sample of what the log looks like, I deleted a great part of its contents because the documents are confidential data but maybe some expert eyes can notice something I'm missing. BTW I think I missed the last line on the log {"error":"multipart: NextPart: EOF"} definitely there is something missing there. test.txt Edited July 9, 2019 by Mocte Sandoval spelling Share this post Link to post
Attila Kovacs 629 Posted July 9, 2019 (edited) umm, if "boundary=---2-1d1a304dc9" in the header, shouldn't be the boundary -----2-1d1a304dc9 in the body? And closing with -----2-1d1a304dc9--? Or just omit a -- in the header. Edited July 9, 2019 by Attila Kovacs Share this post Link to post
Mocte Sandoval 0 Posted July 9, 2019 (edited) Ok, I got rid of all initial "-" so boundary is now e.g. boundary=4024b236a3a1 on header, --4024b236a3a1 in body, and closing --4024b236a3a1-- And all is working great now ! Thank you so much Attila this was driving me nuts. Edit: How can I mark this topic as resolved? Edited July 9, 2019 by Mocte Sandoval Mark as solved Share this post Link to post
Attila Kovacs 629 Posted July 10, 2019 Glad to hear that. 3 minutes ago, Mocte Sandoval said: How can I mark this topic as resolved? You can't. And you will get it resolved the second time tomorrow by others 😉 It's ok. 1 Share this post Link to post
Remy Lebeau 1398 Posted July 10, 2019 (edited) 6 hours ago, Mocte Sandoval said: Now the problem, there are a lot of these "documents" stored on files on disk, each of these files contain a bunch of "documents" on a propietary format, which I'm succesfully extracting as an array of bytes, I can save them as individual files with TFileStream and view them with external tools, but I need to send these individual files to the http server, so for speeding up the process I want to avoid saving them to disk, so what i would like is to send the array of bytes straight to the server, here is an snippet from the code I am using so far Why not continue using TIdMultipartFormDataStream for that? It has an overloaded AddFormField() method that lets you specify a TStream for the input data. You could store your bytes in a TMemoryStream or TBytesStream, or use Indy's own TIdMemoryBufferStream, etc. For example: procedure SendFromBytes( const buffer: array of byte ); var Strm: TIdMemoryBufferStream; Params: TIdMultipartFormDataStream; begin ... Strm := TIdMemoryBufferStream.Create(PByte(buffer), Length(buffer)); try Params := TIdMultipartFormDataStream.Create; try Params.AddFormField('name', 'application/octet-stream', '', Strm, 'filename'); http.Post('http://x.x.x.x:8996/service', Params, responseStream); finally end; finally Strm.Free; end; ... end; 36 minutes ago, Mocte Sandoval said: Ok, I got rid of all initial "-" so boundary is now e.g. boundary=4024b236a3a1 on header, --4024b236a3a1 in body, and closing --4024b236a3a1-- Yes, whatever value is specified in the "boundary" header, in the body it needs to have a leading "--" in front of each new MIME part, and a trailing "--" at the end of the last MIME part. TIdMultipartFormDataStream handles that internally for you. Edited July 10, 2019 by Remy Lebeau Share this post Link to post
Mocte Sandoval 0 Posted July 10, 2019 11 minutes ago, Remy Lebeau said: Why not continue using TIdMultipartFormDataStream for that? It has an overloaded AddFormField() method that lets you specify a TStream for the input data. You could store your bytes in a TMemoryStream or TBytesStream, or use Indy's own TIdMemoryBufferStream, etc. Yes, whatever value you specify in the "boundary" header, you need to add a leading "--" to in the body, and a trailing "--" at the end. Good to know, I'll give it a try tomorrow, thank you Remy Share this post Link to post