Multipart/form-data vs x-www-form-urlencoded (Indy HTTP server)

[Joe Sansalone 5]

Hi,

 

I have a Indy HTTP server application.

Currently receiving requests in the format: application/x-www-form-urlencoded.

(everything works fine, I use the parsed params).

 

I'm now also receiving Posts from SendGrid: "the post will be multipart/form-data with the email content URL encoded, but the attachments will be in multipart/form-data".

 

Using the TIdHTTPRequestInfo.PostStream, I'm able to view the payload that they are sending me.

 

1. Do I need to URLDecode the email content or is it already decoded?

 

2. Are there any functions/classes to help parse the form-data?

 it looks like this:

 

--XYzZY

Content-Disposition: form-data; name="from"

 

Joe S. <joes1234@gmail.com>

--XYzZY

Content-Disposition: form-data; name="text"

 

Hello,

 

I'm testing this email.

Joe

--XYzZY

Content-Disposition: form-data; name="to"

 

mb12345@parsetest.projectone.ca

--XYzZY

Content-Disposition: form-data; name="subject"

 

Subject123

 

 

[Remy Lebeau 1260]

2 hours ago, Joe Sansalone said:

1. Do I need to URLDecode the email content or is it already decoded?

That depends on how you are parsing it.  TIdHTTPServer does not parse 'multipart/form-data' for you (that is on the todo list), so you have to parse the raw data yourself.  What does your code look like for that?

2 hours ago, Joe Sansalone said:

2. Are there any functions/classes to help parse the form-data?

You can use Indy's TIdMessageDecoderMIME class to help you parse 'multipart/form-data' from the ARequestInfo.PostStream data.

[Joe Sansalone 5]

I'm simply doing a 

   aRequestInfo.PostStream.Position := 0;

   StringList.LoadFromStream(aRequestInfo.POstStream, TEncoding.UTF8);

 

to get the content into a string list.

 

Then, I'll be looping through each line of the stringlist looking for

Content-Disposition: form-data; name="xxx" and getting the NAME of the parameter (i.e. xxx).

Then skipping a line and taking every line until --XYzZY, to get the VALUE of the name parameter.

 

I think it will work for anything without attachments.

[Remy Lebeau 1260]

2 hours ago, Joe Sansalone said:

I'm simply doing a 

   aRequestInfo.PostStream.Position := 0;

   StringList.LoadFromStream(aRequestInfo.POstStream, TEncoding.UTF8);

 

to get the content into a string list.

That will only work if the email is entirely textual, no binary attachments.  And also, only if the sender actually sends the text data as UTF-8, which is not a guarantee.

 

Try this instead:

var
  MsgEnd: Boolean;
  Decoder, NewDecoder: TIdMessageDecoder;
  Line, Boundary, BoundaryStart, BoundaryEnd: String;
  Dest: TStream;
begin
  ...

  Boundary := ExtractHeaderSubItem(ARequestInfo.ContentType, 'boundary', QuoteHTTP);
  BoundaryStart := '--' + Boundary;
  BoundaryEnd := BoundaryStart + '--';

  repeat
    Line := ReadLnFromStream(ARequestInfo.PostStream, -1, True);
    if Line = BoundaryEnd then Exit;
  until Line = BoundaryStart;

  Decoder := TIdMessageDecoderMIME.Create(nil);
  try
    MsgEnd := False;
    repeat
      TIdMessageDecoderMIME(Decoder).MIMEBoundary := Boundary;
      Decoder.SourceStream := ARequestInfo.PostStream;
      Decoder.FreeSourceStream := False;

      Decoder.ReadHeader;
      case Decoder.PartType of
        mcptText, mcptAttachment:
        begin
          Dest := TMemoryStream.Create;
          try
            NewDecoder := Decoder.ReadBody(Dest, MsgEnd);
            try
              // use Dest as needed...
            finally
              Decoder.Free;
              Decoder := NewDecoder;
            end;
          finally
            Dest.Free;
          end;
        end;
        mcptIgnore:
        begin
          FreeAndNil(Decoder);
          Decoder := TIdMessageDecoderMIME.Create(nil);
        end;
        mcptEOF:
        begin
          MsgEnd := True;
        end;
      end;
    until (Decoder = nil) or MsgEnd;
  finally
    Decoder.Free;
  end;
end;

2 hours ago, Joe Sansalone said:

Then, I'll be looping through each line of the stringlist looking for

Content-Disposition: form-data; name="xxx" and getting the NAME of the parameter (i.e. xxx).

Then skipping a line and taking every line until --XYzZY, to get the VALUE of the name parameter.

What you SHOULD be doing is actually parsing the MIME data according to the official MIME RFC specifications.

[Joe Sansalone 5]

Thank you for the code.

I realize that I was doing a quick "custom" version just for text and utf8. (and boundary was the same string always coming from SendGrid).

It was my attempt to get something working (my application simply needed some Form-data fields coming in).

 

We are lucky to have you in this forum!  

I'll try out your code. Thanks.