Delphi webbroker handling large base64

[borni69 1]

Hi , We  have a Delphi webbroker system that receives pdf files from a web client in a base64 format.

The server is just receiving them and returning them, it's  like a proxy.

We do this , because the PDF is created on the client site, but we can not download directly from the client, so after creation we send it to the server and back.

This works perfectly for smaller pdf

But when the base64 is getting bigger we are getting out of memory.

The code we use are 

procedure TWebModule1.WebModule1pdfproxyAction(Sender: TObject;
  Request: TWebRequest; Response: TWebResponse; var Handled: Boolean);
var
 acontentType : String;
 abase64      : String;
 afileName    : String;
 FileContent  : TmemoryStream;

begin
  // Pdf proxy
   acontentType := Request.ContentFields.Values['contenttype'];
   abase64      := Request.ContentFields.Values['base64'];
   afileName    := Request.ContentFields.Values['fileName'];

 if ( acontentType > '') and ( abase64 > '') and ( afileName > '') then
 begin
      FileContent      := TmemoryStream.Create;
      FileContent.write(TNetEncoding.Base64.DecodeStringToBytes(abase64),
        length(TNetEncoding.Base64.DecodeStringToBytes(abase64))  );
      FileContent.Position := 0; // Go to the start of the stream

      Response.SetCustomHeader('Content-Disposition', 'attachment; filename='+ TNetEncoding.URL.Encode(afileName));
      Response.ContentStream := FileContent;
      Response.ContentType := 'application/pdf';
      Response.SendResponse;
 end
  else
    Response.Content:=unitGetJsonText.createJsonStatusResult
                          ( Response , TjsonStatus.error ,'error missing content type / base64 /afileName','');
end;

Would there be a way to store the incoming data  as a file and then return it or other ideas for handle a case like this ?

 

 

Thanks in advance

[Guest]

1 hour ago, borni69 said:

Would there be a way to store the incoming data  as a file and then return it or other ideas for handle a case like this ?

It would understand the problem and its details.

 

Base64 is 3 to 4 compression/encoding, means 1mb file will become 1.33mb, so to get out of memory error it that pdf should be really huge like more than a giga bytes in size, 

So first please add some logging and reporting functionality to that code, to get better understanding what is going on, from there you can decide, and don't only log the size also the content type in the header.

 

After that you can search for better approach for encoding without copy the whole thing few times.

[Attila Kovacs 629]

wow, I didn't know this trick: somestring > '' (Edit: I see, opcode is same as with <>)

 

here you are decoding twice, you should optimize this, it hurts

FileContent.write(TNetEncoding.Base64.DecodeStringToBytes(abase64),
        length(TNetEncoding.Base64.DecodeStringToBytes(abase64))  );

Edited March 4, 2021 by Attila Kovacs

[borni69 1]

Updated,

 

I will try to see what changes this is making to the system..

 

thanks..

 

 

 

var
 acontentType : String;
 abase64      : String;
 afileName    : String;
 FileContent  : TmemoryStream;
 abuffer      : Tbytes;

begin
  // Pdf proxy

   acontentType := Request.ContentFields.Values['contenttype'];
   abase64      := Request.ContentFields.Values['base64'];
   afileName    := Request.ContentFields.Values['fileName'];



 if ( acontentType > '') and ( abase64 > '') and ( afileName > '') then
 begin
      FileContent      := TmemoryStream.Create;
      abuffer :=TNetEncoding.Base64.DecodeStringToBytes(abase64);
      FileContent.write(abuffer, length(abuffer)  );
      FileContent.Position := 0; // Go to the start of the stream

      Response.SetCustomHeader('Content-Disposition', 'attachment; filename='+ TNetEncoding.URL.Encode(afileName));
      Response.ContentStream := FileContent;
      Response.ContentType := 'application/pdf';
      Response.SendResponse;
 end
  else
    Response.Content:=unitGetJsonText.createJsonStatusResult
                          ( Response , TjsonStatus.error ,'error missing content type / base64 /afileName','');
end;

 

Edited March 4, 2021 by borni69