PDF File Send as Base64 from c# to Delphi REST

[mazluta 0]

I write RESTserver with Delphi 10.3 The Service accepts 3 parameters. One of them is the string hold base64 string of PDF file.

When I create the Base64 String with Delphi as :

function  ConvertPdfFileToBase64D(PdfFileName : String; var Base64Str : String) : Boolean;
var
  success   : Boolean;
  b64       : String;
  fBytes    : TBytes;
  fSize     : Integer;

  function FileToBytes(const AFileName: string; var Bytes: TBytes): Boolean;
  var
    Stream: TFileStream;
  begin
    if not FileExists(AFileName) then
    begin
      Result := False;
      Exit;
    end;
    Stream := TFileStream.Create(AFileName, fmOpenRead);
    try
      fSize := Stream.Size;
      SetLength(Bytes, fSize);
      Stream.ReadBuffer(Pointer(Bytes)^, fSize);
    finally
      Stream.Free;
    end;
    Result := True;
  end;
begin
  Result := False;
  Base64Str := '';
  if FileToBytes(PdfFileName,fBytes) then
  begin
    //Base64Str := TNetEncoding.Base64.EncodeBytesToString(fBytes, fSize);
    Base64Str := TNetEncoding.Base64.EncodeBytesToString(fBytes);
    Result := True;
  End;
end;

when i get this base64 in the Delphi REST Api I decode the string and successfully save a PDF file.

when i send base64 string created in Visual Studio C# like :

Byte[] fileBytes = File.ReadAllBytes(@textBox1.Text);
var content = Convert.ToBase64String(fileBytes);

I get different base64 strings.

what is the right way to send base64 string as PDF file from C# to Delphi REST API

[Remy Lebeau 1396]

Why are you using Base64 at all? REST runs over HTTP, and HTTP handles binary data without needing to encode it.

[Dave Nottage 557]

3 minutes ago, Remy Lebeau said:

Why are you using Base64 at all? REST runs over HTTP, and HTTP handles binary data without needing to encode it.

Perhaps the file content is being passed as a property of the JSON payload?

[Remy Lebeau 1396]

8 hours ago, mazluta said:

when i send base64 string created in Visual Studio C# like :

...

I get different base64 strings.

Can you provide a concrete example of such a difference? 

[mazluta 0]

He remy.

1. I will prepare a text file that holds what c# sends and what Delphi Rest reads.

2. how do I send PDF as binary data from C# to Delphi Rest?

3. dose it matter if the C# is AnyCPU or X64 and the rest is x86?

 

Thanks, Yossi

[mvanrijnen 123]

Are the totally different, or:

- Is it just the padding that differs

- Is the .Net B64 a long string, and the Delphi B64 is multilne (a string with CR/LF's in it?) 

 

Can you dump here the 2 different strings?, use a testfile!  

Edited October 26, 2023 by mvanrijnen

[Brian Evans 105]

A lot of file types. including PDFs, start with a known byte sequence which also mean they start with a known BASE64 sequence. So, to start with compare the first couple bytes to JVBERi0 .

Format Bytes             Chars    BASE64
PDF    25 50 44 46 2D    %PDF-    JVBERi0

 

[mazluta 0]

Ok, it looks like I figured it out.

but I am not sure it works for every c# version 🙂 (can't trust Windows developer).

 

The Base64FormattingOptions_None.txt file contains the text of base64 created in C# before sending it to Delphi Rest.

 

The ThisWhatDelphiRestGet.txt file contains the base64 text that Delphi rest moves to my var after Http Request envoke.

 

The LogBase64Different.txt contains the differences Including Char No# and The Char ItSelf.

 

It looks like DELPHI REST replaced the ASCII 43 to 32 or

the HTTP request of the C# sent ASCII 32 instead of ASCII 43.

 

So, If The Request comes from C#, I just convert all appearances of ASCII char 32 to 43, then save The PDF, and all work.

 

      SetLength(MyBytesArr,Length(PdfBase64));
      MyBytesArr := TEncoding.ASCII.GetBytes(PdfBase64);
      for CurChar := 1 to Length(PdfBase64) do
      begin
        if Ord(MyBytesArr[CurChar]) = 32 then
           MyBytesArr[CurChar] := byte(43);
      end;
      PdfBase64 := TEncoding.ASCII.GetString(MyBytesArr);
 

 

Can I Trust That Solution?

Or I Missed Something?

 

Thanks, Yossi

Base64FormattingOptions_None.txt

ThisWhatDelphiRestGet.txt

LogBase64Different.txt

Edited October 28, 2023 by mazluta

[mazluta 0]

On 10/26/2023 at 9:21 AM, Remy Lebeau said:

Why are you using Base64 at all? REST runs over HTTP, and HTTP handles binary data without needing to encode it.

Hi Remy 

I Could not figure out how to Send binary data from C# to Delphi REST?

[mvanrijnen 123]

THe ThisWhatDelphiRestGet.txt file, is that the immediate output of your Delphi method?

Because i thought (ot ouf my headnot checked) that the  TNetEncoding.Base64.EncodeBytesToString method returns CR/LF's in its result?

 

see here:

program TestBase64Encoding;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.IOUtils,
  System.NetEncoding;

const
  cSourceURL = 'https://file-examples.com/storage/fe1134defc6538ed39b8efa/2017/10/file-sample_150kB.pdf';
  cInputFilename = 'C:\SWDev\_TestData\file-sample_150kB.pdf';
  cOutputFilename = 'C:\SWDev\_TestData\file-sample_150kB.b64.delphi.txt';
  cOutputFilename2 = 'C:\SWDev\_TestData\file-sample_150kB.b64.delphi-CRLF.txt';
begin
  try
    TFile.WriteAllText(cOutputFilename, TNetEncoding.Base64.EncodeBytesToString(TFile.ReadAllBytes(cInputFilename)));
    TFile.WriteAllText(cOutputFilename2, TNetEncoding.Base64.EncodeBytesToString(TFile.ReadAllBytes(cInputFilename)).Replace(#13#10, '', [rfReplaceAll]));
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

 

.Net

using System.IO;
using System.Text;
using System.Runtime.CompilerServices;

namespace ConsoleApp
{
    internal class Program
    {
        private const string cSourceURL = @"https://file-examples.com/storage/fe1134defc6538ed39b8efa/2017/10/file-sample_150kB.pdf";
        private const string cInputFilename = @"C:\SWDev\_TestData\file-sample_150kB.pdf";
        private const string cOutputFilename = @"C:\SWDev\_TestData\file-sample_150kB.b64.dotnet.txt";

        static void Main(string[] args)
        {
            File.WriteAllText(cOutputFilename, Convert.ToBase64String(File.ReadAllBytes(cInputFilename)));
        }
    }
}

 

You can see that the CRLF stripped  version is the same as the .Net version of the outputfile.

Somethings mixes up the data for TS,

 

 

Result:

 

file-sample_150kB.b64.delphi.txt

ile-sample_150kB.b64.delphi-CRLF.txt

file-sample_150kB.b64.dotnet.txt

 

Edited October 28, 2023 by mvanrijnen

[mazluta 0]

Hi mvanrijnen

i dont think it's the CRLF.

The + sign is replace with the Space sign (43 --> 32)

[Remy Lebeau 1396]

11 hours ago, mazluta said:

It looks like DELPHI REST replaced the ASCII 43 to 32 or

the HTTP request of the C# sent ASCII 32 instead of ASCII 43.

That should only happen if you are sending the data in the URL query string, or in the HTTP body in 'application/x-www-webform-urlencoded' format. You need to show your actual C# and Delphi codes for the REST request, you are clearly not setting up and/or processing the request correctly. DO NOT employ the workaround you have described, that is the wrong solution. You need to fix the underlying bug in your code that is messing up the data in the first place.

 

Edited October 28, 2023 by Remy Lebeau

[mvanrijnen 123]

13 hours ago, mazluta said:

Hi mvanrijnen

i dont think it's the CRLF.

The + sign is replace with the Space sign (43 --> 32)

Yes, but the question is where the data changes? As i demonstrated that Delphi & .Net creates the same b64 data.

 

[mazluta 0]

Hi Remy Lebeau

 

i think your answer is the right answer.

 

this is the C# Code

 

private void button1_Click(object sender, EventArgs e)
        {
            string requestUrl = "http://185.185.135.XXX:YYYY/MyTest";
            HttpWebRequest request = HttpWebRequest.CreateHttp(requestUrl);
            request.Method = "POST";

            // Optionally, set properties of the HttpWebRequest, such as:
            request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
            request.ContentType = "application/x-www-form-urlencoded";
            // Could also set other HTTP headers such as Request.UserAgent, Request.Referer,
            // Request.Accept, or other headers via the Request.Headers collection.

            //Byte[] fileBytes = File.ReadAllBytes(@textBox1.Text, System.Text.Encoding.ASCII);
            Byte[] fileBytes = File.ReadAllBytes(@textBox1.Text);
            var content = Convert.ToBase64String(fileBytes,Base64FormattingOptions.None);

            // Set the POST request body data. In this example, the POST data is in 
            // application/x-www-form-urlencoded format.
            string postData = "DevApp=C#&Base64Type=base64&username=mazluta&base64=" + content;

            using (var writer = new StreamWriter(request.GetRequestStream()))
            {
                writer.Write(postData);
            }

            // Submit the request, and get the response body from the remote server.
            string responseFromRemoteServer;
            using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                using (StreamReader reader = new StreamReader(response.GetResponseStream()))
                {
                    responseFromRemoteServer = reader.ReadToEnd();
                    fileBytes = Convert.FromBase64String(responseFromRemoteServer);
                    File.WriteAllBytes(@"c:\a\mytest.pdf", fileBytes);
                    MessageBox.Show("File c:\\a\\mytest.pdf signed and saved");
                }
            }
        }

 

 

This is The Delphi Rest Code :

 

procedure TMyWebModule.MyWebModuleactSignAction(Sender: TObject;
  Request: TWebRequest; Response: TWebResponse; var Handled: Boolean);
Var
  RequestObject: TJSONObject;
  PdfBase64      : string;
  RspBase64      : string;
  UserName       : string;
  Base64Type     : string;
  DevApp         : string;
  CachPdfPath    : String;
  CachFilesPath  : String;
  TmpPdfFile     : String;
  TmpTxtFile     : String;
  DstPdfFileName : String;
  SignParam      : TSignParam;
  PfxFileName    : String;
  PfxPassword    : String;
  UserDataRec    : TUserDataRec;
  ParamArea      : TParamArea;
  UserSignData   : TUserSignData;
  //aPdfBase64     : AnsiString;
  aRspBase64     : PAnsiChar;
  aTmpPdfFile    : AnsiString;
  aTmpTxtFile    : AnsiString;
  SaveTextList   : TStringList;
  MyBytesArr     : TBytes;
  CurChar        : Integer;
begin
( JustWriteToLog(' ');
  JustWriteToLog('======');
  JustWriteToLog('reqeust=base64 at : ' + formatdatetime('dd/mm/yyyy hh:nn:ss:zzz', now));
  JustWriteToLog(' ');

  UserName   := Request.ContentFields.Values['UserName'];
  Base64Type := Request.ContentFields.Values['Base64Type'];
  PdfBase64  := Request.ContentFields.Values['Base64'];
  DevApp     := Request.ContentFields.Values['DevApp'];

  JustWriteToLog(' ');
  JustWriteToLog('UserName='+UserName);
  JustWriteToLog(' ');
  JustWriteToLog('Base64Type='+Base64Type);
  JustWriteToLog(' ');
  JustWriteToLog('DevApp='+DevApp);
  JustWriteToLog(' ');
  JustWriteToLog('Base64='+PdfBase64);
  JustWriteToLog(' ');

  UserDataRec  := dm_DB.GetUserData(UserName);
  ParamArea    := dm_DB.LoadAppParams;
  JustWriteToLog('User request = ' + UserName);
  JustWriteToLog('Start PDF 50 char = ' + Copy(PdfBase64, 1, 50));

  if UserDataRec.Found then
    JustWriteToLog('User Found')
  else
    JustWriteToLog('User not found');

  if ParamArea.Found then
    JustWriteToLog('ParamArea Record found');

  If not ParamArea.Found Then
  begin
    Response.Content := 'Fail load Server params';
    Response.StatusCode := 400;
    exit;
  end;

  UserSignData := dm_DB.GetUserSignRec(UserDataRec, ParamArea);
  If not UserSignData.Found Then
  begin
    Response.Content := 'Fail load UserSignData params';
    Response.StatusCode := 400;
    exit;
  end;

  JustWriteToLog('Find User SignData, PfxFile = ' + UserSignData.User_PfxFileName);
  JustWriteToLog('Find User SignData, LogoFile = ' + UserSignData.User_LogoFileName);

  CachPdfPath := GetBaseAppPath + '\' + DSHTTPWebDispatcher1.CacheContext;
  if CachPdfPath[length(CachPdfPath)] = '/' Then
    CachPdfPath[length(CachPdfPath)] := ' ';
  CachPdfPath := Trim(CachPdfPath);

  ForceDirectories(CachPdfPath);

  Try
    TmpPdfFile := CachPdfPath + '\Tmp_' + GetRandomStr + '.pdf';
    TmpTxtFile := ChangeFileExt(TmpPdfFile,'.txt');
    if UpperCase(DevApp) = UpperCase('C#') then
    begin
      JustWriteToLog('Before Replace PdfBase64 : ' + PdfBase64);
      SetLength(MyBytesArr,Length(PdfBase64));
      MyBytesArr := TEncoding.ASCII.GetBytes(PdfBase64);
      for CurChar := 1 to Length(PdfBase64) do
      begin
        if Ord(MyBytesArr[CurChar]) = 32 then
           MyBytesArr[CurChar] := byte(43);
      end;
      PdfBase64 := TEncoding.ASCII.GetString(MyBytesArr);
      JustWriteToLog('After Replace PdfBase64 : ' + PdfBase64);
      ConvertBase64ToPdfFile(PdfBase64,TmpPdfFile);
    end
    else
    if UpperCase(Base64Type) = UpperCase('Base64') then
    begin
      ConvertBase64ToPdfFile(PdfBase64,TmpPdfFile);
    end
    else
    begin
      ConvertBase64MimeToPdfFile(PdfBase64,TmpPdfFile);
    end;

    JustWriteToLog('PDF File : ' + TmpPdfFile + ' Saved');
  Except
    on e: exception do
    begin
      JustWriteToLog('Error on Save PDF File : ' + e.Message);
      Response.StatusCode := 390;
      Exit;
    end;
  End;

  DstPdfFileName := CachPdfPath + '\Rslt_' + GetRandomStr + '.pdf';
  CachFilesPath := GetBaseAppPath + '\Files';

 

 

so - it looks that when I read the value of the Base64 field - the + becomes space

 

  PdfBase64  := Request.ContentFields.Values['Base64'];

 

 

Edited October 29, 2023 by mazluta

[mazluta 0]

Hi Hi Remy Lebeau

 

if the Solution is to do that in C# : https://stackoverflow.com/questions/7842547/request-parameter-losing-plus-sign

 

descriptionsUrlAddition = descriptionsUrlAddition.replace("+", "%2B");

 

Then my Solution is the same

i tried :

  request.ContentType = "application/form-data";
  request.ContentType = "application/raw";

 

some give there solution like content = Uri.EscapeDataString(content);

and that work fine.

Edited October 30, 2023 by mazluta

[Remy Lebeau 1396]

The C# code is writing the base64 content as-is to the socket, it is not encoding the base64 to the "x-www-form-urlencoded" format.  Base64 can use '+' and '=' characters, which are reserved in "x-www-form-urlencoded" and must be encoded as "%2B" and "%3D" in application data, respectively.  The receiver must decode them back into '+' and '=' characters, respectively, before then decoding the base64.

 

Per the HTML standards, which define the "x-www-form-urlencoded" format:

 

HTML 4.01 Section 17.13.4 ("Form content types"):

Quote

application/x-www-form-urlencoded

 

Control names and values are escaped. Space characters are replaced by `+', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by `%HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A').

HTML 5 Section 4.10.16.4 ("URL-encoded form data"):

Quote

If the character isn't in the range U+0020, U+002A, U+002D, U+002E, U+0030 .. U+0039, U+0041 .. U+005A, U+005F, U+0061 .. U+007A then replace the character with a string formed as follows: Start with the empty string, and then, taking each byte of the character when expressed in the selected character encoding in turn, append to the string a U+0025 PERCENT SIGN character (%) followed by two characters in the ranges U+0030 DIGIT ZERO (0) to U+0039 DIGIT NINE (9) and U+0041 LATIN CAPITAL LETTER A to U+005A LATIN CAPITAL LETTER Z representing the hexadecimal value of the byte zero-padded if necessary).

 

If the character is a U+0020 SPACE character, replace it with a single U+002B PLUS SIGN character (+).

In other words, in a "x-www-form-urlencoded" submission, any non-syntax characters (ie, field separators '=' and '&') that are not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789*-._" should be encoded in %HH format, except for space characters which are encoded as '+' instead.

 

That means even your "DevApp" field should be transmitted as "C%23" instead of as "C#".

Edited October 30, 2023 by Remy Lebeau

[mazluta 0]

Hi Remy.

First: thanks for the knowledge.

 

Secondly: the "DevApp" field was sent as C# from the start.

 

Third: the line -

content = Uri.EscapeDataString(content);

fix it all put "%2B" as replaced for any "+" sign and the Delphi Rest got the '+' sign as it should be without doing anything in the service.

 

Fourth: 

when i sent the Base64 string back with '+' sign in the response the C# read it well and saved the PDF to wherever it should be saved.

 

[Remy Lebeau 1396]

7 minutes ago, mazluta said:

the "DevApp" field was sent as C# from the start.

I know, and that is technically wrong per the specs, even if it is being accepted as-is.

7 minutes ago, mazluta said:

the line -

content = Uri.EscapeDataString(content);

fix it all put "%2B" as replaced for any "+" sign and the Delphi Rest got the '+' sign as it should be without doing anything in the service.

Yes, that is the simplest way to go in this situation.

7 minutes ago, mazluta said:

when i sent the Base64 string back with '+' sign in the response the C# read it well and saved the PDF to wherever it should be saved.

I can't comment on that without seeing the code that was doing the sending/reading, and the raw data.

 

Personally I would not have used "x-www-form-urlencoded" for this kind of data.  "multipart/form-data" would have made more sense for posting a binary file with metadata. But JSON is also a common format to use in REST APIs. Either way would have avoided the url-encoding issue.

[mazluta 0]

Thanks Remy,

 

I Will try the binary data and the JSON way.