mazluta 0 Posted October 25, 2023 I write RESTserver with Delphi 10.3 The Service accepts 3 parameters. One of them is the string hold base64 string of PDF file. When I create the Base64 String with Delphi as : function ConvertPdfFileToBase64D(PdfFileName : String; var Base64Str : String) : Boolean; var success : Boolean; b64 : String; fBytes : TBytes; fSize : Integer; function FileToBytes(const AFileName: string; var Bytes: TBytes): Boolean; var Stream: TFileStream; begin if not FileExists(AFileName) then begin Result := False; Exit; end; Stream := TFileStream.Create(AFileName, fmOpenRead); try fSize := Stream.Size; SetLength(Bytes, fSize); Stream.ReadBuffer(Pointer(Bytes)^, fSize); finally Stream.Free; end; Result := True; end; begin Result := False; Base64Str := ''; if FileToBytes(PdfFileName,fBytes) then begin //Base64Str := TNetEncoding.Base64.EncodeBytesToString(fBytes, fSize); Base64Str := TNetEncoding.Base64.EncodeBytesToString(fBytes); Result := True; End; end; when i get this base64 in the Delphi REST Api I decode the string and successfully save a PDF file. when i send base64 string created in Visual Studio C# like : Byte[] fileBytes = File.ReadAllBytes(@textBox1.Text); var content = Convert.ToBase64String(fileBytes); I get different base64 strings. what is the right way to send base64 string as PDF file from C# to Delphi REST API Share this post Link to post
Remy Lebeau 1405 Posted October 26, 2023 Why are you using Base64 at all? REST runs over HTTP, and HTTP handles binary data without needing to encode it. Share this post Link to post
Dave Nottage 559 Posted October 26, 2023 3 minutes ago, Remy Lebeau said: Why are you using Base64 at all? REST runs over HTTP, and HTTP handles binary data without needing to encode it. Perhaps the file content is being passed as a property of the JSON payload? Share this post Link to post
Remy Lebeau 1405 Posted October 26, 2023 8 hours ago, mazluta said: when i send base64 string created in Visual Studio C# like : ... I get different base64 strings. Can you provide a concrete example of such a difference? Share this post Link to post
mazluta 0 Posted October 26, 2023 He remy. 1. I will prepare a text file that holds what c# sends and what Delphi Rest reads. 2. how do I send PDF as binary data from C# to Delphi Rest? 3. dose it matter if the C# is AnyCPU or X64 and the rest is x86? Thanks, Yossi Share this post Link to post
mvanrijnen 123 Posted October 26, 2023 (edited) Are the totally different, or: - Is it just the padding that differs - Is the .Net B64 a long string, and the Delphi B64 is multilne (a string with CR/LF's in it?) Can you dump here the 2 different strings?, use a testfile! Edited October 26, 2023 by mvanrijnen Share this post Link to post
Brian Evans 108 Posted October 26, 2023 A lot of file types. including PDFs, start with a known byte sequence which also mean they start with a known BASE64 sequence. So, to start with compare the first couple bytes to JVBERi0 . Format Bytes Chars BASE64 PDF 25 50 44 46 2D %PDF- JVBERi0 Share this post Link to post
mazluta 0 Posted October 28, 2023 (edited) Ok, it looks like I figured it out. but I am not sure it works for every c# version 🙂 (can't trust Windows developer). The Base64FormattingOptions_None.txt file contains the text of base64 created in C# before sending it to Delphi Rest. The ThisWhatDelphiRestGet.txt file contains the base64 text that Delphi rest moves to my var after Http Request envoke. The LogBase64Different.txt contains the differences Including Char No# and The Char ItSelf. It looks like DELPHI REST replaced the ASCII 43 to 32 or the HTTP request of the C# sent ASCII 32 instead of ASCII 43. So, If The Request comes from C#, I just convert all appearances of ASCII char 32 to 43, then save The PDF, and all work. SetLength(MyBytesArr,Length(PdfBase64)); MyBytesArr := TEncoding.ASCII.GetBytes(PdfBase64); for CurChar := 1 to Length(PdfBase64) do begin if Ord(MyBytesArr[CurChar]) = 32 then MyBytesArr[CurChar] := byte(43); end; PdfBase64 := TEncoding.ASCII.GetString(MyBytesArr); Can I Trust That Solution? Or I Missed Something? Thanks, Yossi Base64FormattingOptions_None.txt ThisWhatDelphiRestGet.txt LogBase64Different.txt Edited October 28, 2023 by mazluta Share this post Link to post
mazluta 0 Posted October 28, 2023 On 10/26/2023 at 9:21 AM, Remy Lebeau said: Why are you using Base64 at all? REST runs over HTTP, and HTTP handles binary data without needing to encode it. Hi Remy I Could not figure out how to Send binary data from C# to Delphi REST? Share this post Link to post
mvanrijnen 123 Posted October 28, 2023 (edited) THe ThisWhatDelphiRestGet.txt file, is that the immediate output of your Delphi method? Because i thought (ot ouf my headnot checked) that the TNetEncoding.Base64.EncodeBytesToString method returns CR/LF's in its result? see here: program TestBase64Encoding; {$APPTYPE CONSOLE} {$R *.res} uses System.SysUtils, System.IOUtils, System.NetEncoding; const cSourceURL = 'https://file-examples.com/storage/fe1134defc6538ed39b8efa/2017/10/file-sample_150kB.pdf'; cInputFilename = 'C:\SWDev\_TestData\file-sample_150kB.pdf'; cOutputFilename = 'C:\SWDev\_TestData\file-sample_150kB.b64.delphi.txt'; cOutputFilename2 = 'C:\SWDev\_TestData\file-sample_150kB.b64.delphi-CRLF.txt'; begin try TFile.WriteAllText(cOutputFilename, TNetEncoding.Base64.EncodeBytesToString(TFile.ReadAllBytes(cInputFilename))); TFile.WriteAllText(cOutputFilename2, TNetEncoding.Base64.EncodeBytesToString(TFile.ReadAllBytes(cInputFilename)).Replace(#13#10, '', [rfReplaceAll])); except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; end. .Net using System.IO; using System.Text; using System.Runtime.CompilerServices; namespace ConsoleApp { internal class Program { private const string cSourceURL = @"https://file-examples.com/storage/fe1134defc6538ed39b8efa/2017/10/file-sample_150kB.pdf"; private const string cInputFilename = @"C:\SWDev\_TestData\file-sample_150kB.pdf"; private const string cOutputFilename = @"C:\SWDev\_TestData\file-sample_150kB.b64.dotnet.txt"; static void Main(string[] args) { File.WriteAllText(cOutputFilename, Convert.ToBase64String(File.ReadAllBytes(cInputFilename))); } } } You can see that the CRLF stripped version is the same as the .Net version of the outputfile. Somethings mixes up the data for TS, Result: file-sample_150kB.b64.delphi.txt ile-sample_150kB.b64.delphi-CRLF.txt file-sample_150kB.b64.dotnet.txt Edited October 28, 2023 by mvanrijnen Share this post Link to post
mazluta 0 Posted October 28, 2023 Hi mvanrijnen i dont think it's the CRLF. The + sign is replace with the Space sign (43 --> 32) Share this post Link to post
Remy Lebeau 1405 Posted October 28, 2023 (edited) 11 hours ago, mazluta said: It looks like DELPHI REST replaced the ASCII 43 to 32 or the HTTP request of the C# sent ASCII 32 instead of ASCII 43. That should only happen if you are sending the data in the URL query string, or in the HTTP body in 'application/x-www-webform-urlencoded' format. You need to show your actual C# and Delphi codes for the REST request, you are clearly not setting up and/or processing the request correctly. DO NOT employ the workaround you have described, that is the wrong solution. You need to fix the underlying bug in your code that is messing up the data in the first place. Edited October 28, 2023 by Remy Lebeau Share this post Link to post
mvanrijnen 123 Posted October 29, 2023 13 hours ago, mazluta said: Hi mvanrijnen i dont think it's the CRLF. The + sign is replace with the Space sign (43 --> 32) Yes, but the question is where the data changes? As i demonstrated that Delphi & .Net creates the same b64 data. Share this post Link to post
mazluta 0 Posted October 29, 2023 (edited) Hi Remy Lebeau i think your answer is the right answer. this is the C# Code private void button1_Click(object sender, EventArgs e) { string requestUrl = "http://185.185.135.XXX:YYYY/MyTest"; HttpWebRequest request = HttpWebRequest.CreateHttp(requestUrl); request.Method = "POST"; // Optionally, set properties of the HttpWebRequest, such as: request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip; request.ContentType = "application/x-www-form-urlencoded"; // Could also set other HTTP headers such as Request.UserAgent, Request.Referer, // Request.Accept, or other headers via the Request.Headers collection. //Byte[] fileBytes = File.ReadAllBytes(@textBox1.Text, System.Text.Encoding.ASCII); Byte[] fileBytes = File.ReadAllBytes(@textBox1.Text); var content = Convert.ToBase64String(fileBytes,Base64FormattingOptions.None); // Set the POST request body data. In this example, the POST data is in // application/x-www-form-urlencoded format. string postData = "DevApp=C#&Base64Type=base64&username=mazluta&base64=" + content; using (var writer = new StreamWriter(request.GetRequestStream())) { writer.Write(postData); } // Submit the request, and get the response body from the remote server. string responseFromRemoteServer; using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { responseFromRemoteServer = reader.ReadToEnd(); fileBytes = Convert.FromBase64String(responseFromRemoteServer); File.WriteAllBytes(@"c:\a\mytest.pdf", fileBytes); MessageBox.Show("File c:\\a\\mytest.pdf signed and saved"); } } } This is The Delphi Rest Code : procedure TMyWebModule.MyWebModuleactSignAction(Sender: TObject; Request: TWebRequest; Response: TWebResponse; var Handled: Boolean); Var RequestObject: TJSONObject; PdfBase64 : string; RspBase64 : string; UserName : string; Base64Type : string; DevApp : string; CachPdfPath : String; CachFilesPath : String; TmpPdfFile : String; TmpTxtFile : String; DstPdfFileName : String; SignParam : TSignParam; PfxFileName : String; PfxPassword : String; UserDataRec : TUserDataRec; ParamArea : TParamArea; UserSignData : TUserSignData; //aPdfBase64 : AnsiString; aRspBase64 : PAnsiChar; aTmpPdfFile : AnsiString; aTmpTxtFile : AnsiString; SaveTextList : TStringList; MyBytesArr : TBytes; CurChar : Integer; begin ( JustWriteToLog(' '); JustWriteToLog('======'); JustWriteToLog('reqeust=base64 at : ' + formatdatetime('dd/mm/yyyy hh:nn:ss:zzz', now)); JustWriteToLog(' '); UserName := Request.ContentFields.Values['UserName']; Base64Type := Request.ContentFields.Values['Base64Type']; PdfBase64 := Request.ContentFields.Values['Base64']; DevApp := Request.ContentFields.Values['DevApp']; JustWriteToLog(' '); JustWriteToLog('UserName='+UserName); JustWriteToLog(' '); JustWriteToLog('Base64Type='+Base64Type); JustWriteToLog(' '); JustWriteToLog('DevApp='+DevApp); JustWriteToLog(' '); JustWriteToLog('Base64='+PdfBase64); JustWriteToLog(' '); UserDataRec := dm_DB.GetUserData(UserName); ParamArea := dm_DB.LoadAppParams; JustWriteToLog('User request = ' + UserName); JustWriteToLog('Start PDF 50 char = ' + Copy(PdfBase64, 1, 50)); if UserDataRec.Found then JustWriteToLog('User Found') else JustWriteToLog('User not found'); if ParamArea.Found then JustWriteToLog('ParamArea Record found'); If not ParamArea.Found Then begin Response.Content := 'Fail load Server params'; Response.StatusCode := 400; exit; end; UserSignData := dm_DB.GetUserSignRec(UserDataRec, ParamArea); If not UserSignData.Found Then begin Response.Content := 'Fail load UserSignData params'; Response.StatusCode := 400; exit; end; JustWriteToLog('Find User SignData, PfxFile = ' + UserSignData.User_PfxFileName); JustWriteToLog('Find User SignData, LogoFile = ' + UserSignData.User_LogoFileName); CachPdfPath := GetBaseAppPath + '\' + DSHTTPWebDispatcher1.CacheContext; if CachPdfPath[length(CachPdfPath)] = '/' Then CachPdfPath[length(CachPdfPath)] := ' '; CachPdfPath := Trim(CachPdfPath); ForceDirectories(CachPdfPath); Try TmpPdfFile := CachPdfPath + '\Tmp_' + GetRandomStr + '.pdf'; TmpTxtFile := ChangeFileExt(TmpPdfFile,'.txt'); if UpperCase(DevApp) = UpperCase('C#') then begin JustWriteToLog('Before Replace PdfBase64 : ' + PdfBase64); SetLength(MyBytesArr,Length(PdfBase64)); MyBytesArr := TEncoding.ASCII.GetBytes(PdfBase64); for CurChar := 1 to Length(PdfBase64) do begin if Ord(MyBytesArr[CurChar]) = 32 then MyBytesArr[CurChar] := byte(43); end; PdfBase64 := TEncoding.ASCII.GetString(MyBytesArr); JustWriteToLog('After Replace PdfBase64 : ' + PdfBase64); ConvertBase64ToPdfFile(PdfBase64,TmpPdfFile); end else if UpperCase(Base64Type) = UpperCase('Base64') then begin ConvertBase64ToPdfFile(PdfBase64,TmpPdfFile); end else begin ConvertBase64MimeToPdfFile(PdfBase64,TmpPdfFile); end; JustWriteToLog('PDF File : ' + TmpPdfFile + ' Saved'); Except on e: exception do begin JustWriteToLog('Error on Save PDF File : ' + e.Message); Response.StatusCode := 390; Exit; end; End; DstPdfFileName := CachPdfPath + '\Rslt_' + GetRandomStr + '.pdf'; CachFilesPath := GetBaseAppPath + '\Files'; so - it looks that when I read the value of the Base64 field - the + becomes space PdfBase64 := Request.ContentFields.Values['Base64']; Edited October 29, 2023 by mazluta Share this post Link to post
mazluta 0 Posted October 29, 2023 (edited) Hi Hi Remy Lebeau if the Solution is to do that in C# : https://stackoverflow.com/questions/7842547/request-parameter-losing-plus-sign descriptionsUrlAddition = descriptionsUrlAddition.replace("+", "%2B"); Then my Solution is the same i tried : request.ContentType = "application/form-data"; request.ContentType = "application/raw"; some give there solution like content = Uri.EscapeDataString(content); and that work fine. Edited October 30, 2023 by mazluta Share this post Link to post
Remy Lebeau 1405 Posted October 30, 2023 (edited) The C# code is writing the base64 content as-is to the socket, it is not encoding the base64 to the "x-www-form-urlencoded" format. Base64 can use '+' and '=' characters, which are reserved in "x-www-form-urlencoded" and must be encoded as "%2B" and "%3D" in application data, respectively. The receiver must decode them back into '+' and '=' characters, respectively, before then decoding the base64. Per the HTML standards, which define the "x-www-form-urlencoded" format: HTML 4.01 Section 17.13.4 ("Form content types"): Quote application/x-www-form-urlencoded Control names and values are escaped. Space characters are replaced by `+', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by `%HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A'). HTML 5 Section 4.10.16.4 ("URL-encoded form data"): Quote If the character isn't in the range U+0020, U+002A, U+002D, U+002E, U+0030 .. U+0039, U+0041 .. U+005A, U+005F, U+0061 .. U+007A then replace the character with a string formed as follows: Start with the empty string, and then, taking each byte of the character when expressed in the selected character encoding in turn, append to the string a U+0025 PERCENT SIGN character (%) followed by two characters in the ranges U+0030 DIGIT ZERO (0) to U+0039 DIGIT NINE (9) and U+0041 LATIN CAPITAL LETTER A to U+005A LATIN CAPITAL LETTER Z representing the hexadecimal value of the byte zero-padded if necessary). If the character is a U+0020 SPACE character, replace it with a single U+002B PLUS SIGN character (+). In other words, in a "x-www-form-urlencoded" submission, any non-syntax characters (ie, field separators '=' and '&') that are not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789*-._" should be encoded in %HH format, except for space characters which are encoded as '+' instead. That means even your "DevApp" field should be transmitted as "C%23" instead of as "C#". Edited October 30, 2023 by Remy Lebeau Share this post Link to post
mazluta 0 Posted October 30, 2023 Hi Remy. First: thanks for the knowledge. Secondly: the "DevApp" field was sent as C# from the start. Third: the line - content = Uri.EscapeDataString(content); fix it all put "%2B" as replaced for any "+" sign and the Delphi Rest got the '+' sign as it should be without doing anything in the service. Fourth: when i sent the Base64 string back with '+' sign in the response the C# read it well and saved the PDF to wherever it should be saved. Share this post Link to post
Remy Lebeau 1405 Posted October 30, 2023 7 minutes ago, mazluta said: the "DevApp" field was sent as C# from the start. I know, and that is technically wrong per the specs, even if it is being accepted as-is. 7 minutes ago, mazluta said: the line - content = Uri.EscapeDataString(content); fix it all put "%2B" as replaced for any "+" sign and the Delphi Rest got the '+' sign as it should be without doing anything in the service. Yes, that is the simplest way to go in this situation. 7 minutes ago, mazluta said: when i sent the Base64 string back with '+' sign in the response the C# read it well and saved the PDF to wherever it should be saved. I can't comment on that without seeing the code that was doing the sending/reading, and the raw data. Personally I would not have used "x-www-form-urlencoded" for this kind of data. "multipart/form-data" would have made more sense for posting a binary file with metadata. But JSON is also a common format to use in REST APIs. Either way would have avoided the url-encoding issue. Share this post Link to post
mazluta 0 Posted October 30, 2023 Thanks Remy, I Will try the binary data and the JSON way. Share this post Link to post