Remy Lebeau

Indy (more specifically, its OpenSSL IOHandler) supports TLS 1.2 just fine, so you should not have needed to switch to another SMTP library just to continue communicating with Microsoft.

In addition to everthing said about AddRef/Release(), just note that if your Form's window is ever recreated at runtime, you will loose your marking and have to re-apply it on the new window. Best way to avoid that is to have your Form class override the virtual CreateWnd() method to call MarkWindowAsUnpinnable(), don't call it after creating the Form object.

This is why you should configure your AntiVirus/AntiMalware to ignore compiler output folders as exceptions.

Yes, you can. You can see the RTL doing exactly that, for instance in the SysUtils.(Wide)FormatBuf() functions. Yes. Yes. Just know that if the AnsiString is empty, the pointer will be nil, so don't dereference it. I prefer to use this instead: Move(PAnsiChar(AnsiString(VAnsiString))^, If the AnsiString is empty (ie, a nil pointer), the PAnsiChar cast will return a non-nil pointer to a #0 AnsiChar in static memory. So the dereference is always valid. Also, this isn't affected by 0-based vs 1-based string indexing. Yes.

What is 'S' in this code? It appears to be a pointer into an array of AnsiChars. The code appears to be building up an inline array of length-prefixed ANSI strings. Maybe for transmission/storage somewhere? In any case... vtString indicates a ShortString. Since ShortString is a fixed-length string, the TVarRec.VString field is a pointer to an actual ShortString variable. The code is handling the VString field correctly, dereferencing that pointer to access the ShortString, and thus its length and characters. vtAnsiString indicates an AnsiString. An AnsiString is a dynamic length string, and an AnsiString variable is just a pointer to the AnsiString's payload. As such, the TVarRec.VAnsiString field is a pointer to an AnsiString's payload, NOT a pointer to an AnsiString variable. The code is not handling the VAnsiString field correctly, most importantly its use of sizeof() is wrong to determine the AnsiString's length. You need to typecast the VAnsiString pointer to an AnsiString and then query its Length() instead. Try this: var Len: Integer; ... for I := Low(Values) to High(Values) do with Values[I] do case VType of vtString: begin Len := Length(VString^); // will never exceed 255 S[P] := AnsiChar(Len); System.Move(VString^[1], S[P+1], Len); Inc(P, Len + 1); ls := I = High(Values); end; vtAnsiString: begin Len := Length(AnsiString(VAnsiString)); if Len > 255 raise ...; S[P] := AnsiChar(Len); System.Move(PAnsiChar(AnsiString(VAnsiString))^, S[P+1], Len); Inc(P, Len + 1); ls := I = High(Values); end; ... end; WideString and UnicodeString are also dynamic length string types implemented as pointers, and so vtWideString and vtUnicodeString are handled similarly to vtAnsiString, in that you simply type-cast the TVarRec.VWideString and TVarRec.VUnicodeString pointers as-is to WideString and UnicodeString, respectively. Just know that you will need an additional cast to AnsiString for those values to fit in with the rest of this code, eg: var Len: Integer; Tmp: AnsiString; ... for I := Low(Values) to High(Values) do with Values[I] do case VType of ... vtWideString: begin Tmp := AnsiString(WideString(VWideString)); Len := Length(Tmp); if Len > 255 raise ...; S[P] := AnsiChar(Len); System.Move(PAnsiChar(Tmp)^, S[P+1], Len); Inc(P, Len + 1); ls := I = High(Values); end; vtUnicodeString: begin Tmp := AnsiString(UnicodeString(VUnicodeString)); Len := Length(Tmp); if Len > 255 raise ...; S[P] := AnsiChar(Len); System.Move(PAnsiChar(Tmp)^, S[P+1], Len); Inc(P, Len + 1); ls := I = High(Values); end; ... end;

That is what the "Msg" in its name stands for - it can wait on the Message Queue AND on a list of signallable objects. SyncEvent is used to let the main thread know when TThread.Synchronize()/TThread.Queue() requests are pending. It is not used to let the main thread know when messages are pending. Simply call MsgWaitForMultipleObjects() with any of the flags that wait on the message queue (I usually just use QS_ALLINPUT), and then call Application.ProcessMessages() whenever MsgWaitForMultipleObjects() reports that messages are pending (when its return value is WAIT_OBJECT_0 + nCount).

Not if you pump the message queue whenever MsgWaitForMultipleObjects() tells you that messages are waiting.

Something that has been annoying me for awhile. When posting a code snippet and choosing the Pascal syntax highlighter, backslashes are treated as C/C++ escape sequences, which throws off the coloring. The actual Pascal language doesn't treat backslash as an escape character. Can this be fixed?

Can you be more specific? What is the ACTUAL problem? That is a LOT of code for a seemingly simple problem. Can you narrow it down to a much simpler test case?

Good point. I have removed that suggestion.

I've already addressed this on your StackOverflow question on this same topic: https://stackoverflow.com/questions/70977125/ There are several issues in the original code that you would need to fix, regardless of whether you use the code in Delphi or C++Builder. The code is not using module handles and Unicode buffers correctly, which clearly indicates that the code is old, predating Delphi's support for 64bit and Unicode environments. The code needs to be updated before you can then translate it correctly. That is because the code you are struggling with is translated incorrectly. Why are you translating this code AT ALL? You never answered that when I posted it on your StackOverflow question. You can use Delphi .pas files as-is in C++ Builder projects. The IDE will generate a .hpp file that you can then #include into your C++ code.

What exactly is not working? Please be more specific. Actually, I can't. I don't have to working IDE installed at the moment. Everything I wrote earlier was from memory only.

That is because you are not writing to the Memo while the reading loop is running. You are writing to the Memo only after the loop is finished. Change the code to write the current Buffer to the Memo after each successful read. And don't use the Memo.Text property to do that update, either. That will be very inefficient. A better way to append text to the end of a Memo is to use its SelText property instead, eg: procedure GetDosOutput(Output: TMemo; CommandLine: string; Work: string); var ... begin ... repeat WasOK := ReadFile(StdOutPipeRead, Buffer, 255, BytesRead, nil); if WasOK and (BytesRead > 0) then begin Buffer[BytesRead] := #0; Output.SelStart := Output.GetTextLen; Output.SelLength := 0; Output.SelText := Buffer; end; until (not WasOK) or (BytesRead = 0); ... end; procedure TForm7.Button1Click(Sender: TObject); begin GetDosOutput(Memo1, 'python mtk payload', ExtractFilePath(application.ExeName) + 'bin\'); end; If you don't want to pass in the TMemo directly, you could pass in a TStream instead, and then write a custom TStream descendant that overwrites the virtual Write() method to append to the Memo, eg: procedure GetDosOutput(Output: TStream; CommandLine: string; Work: string); var ... begin ... repeat WasOK := ReadFile(StdOutPipeRead, Buffer, 255, BytesRead, nil); if WasOK and (BytesRead > 0) then Output.WriteBuffer(Buffer, BytesRead); until (not WasOK) or (BytesRead = 0); ... end; type TMemoAppendStream = class(TStream) private FMemo: TMemo; public constructor Create(AMemo: TMemo); function Write(const Buffer; Count: Longint): Longint; override; end; constructor TMemoAppendStream.Create(AMemo: TMemo); begin inherited Create; FMemo := AMemo; end; function TMemoAppendStream.Write(const Buffer; Count: Longint): Longint; var BufferStr: AnsiString; begin Result := Count; SetString(BufferStr, PAnsiChar(@Buffer), Count); FMemo.SelStart := FMemo.GetTextLen; FMemo.SelLength := 0; FMemo.SelText := BufferStr; end; procedure TForm7.Button1Click(Sender: TObject); var Strm: TMemoAppendStream; begin Strm := TMemoAppendStream.Create(Memo1); try GetDosOutput(Strm, 'python mtk payload', ExtractFilePath(application.ExeName) + 'bin\'); finally Strm.Free; end; end;

Not really, no. Windows uses OLE for cross-process drag&drop. For backwards compatibility, if the receiving process does not implement IDropTarget, OLE synthesizes the legacy WM_DROPFILES window message for it. Yes, in-process drag&drop is the responsibility of the process to implement. There is no Win32 API support for it outside of OLE, so the process can basically use whatever it wants.

None of those leaks are related to IndyFormat(). However, you are clearly leaking a TIdHTTP object, which in turn leaks all of its internal objects, as well as the global GIdStack object in the IdStack.pas unit, because GIdStack is reference-counted and all Indy socket components increment its reference count when they are created and decrement it when they are destroyed. Fix the TIdHTTP leak, and the rest of the leaks in that screenshot will disappear. There are no known issues with memory leaks, no.

In Tokyo, IndyFormat() simply calls SysUtils.Format(), so any memory issue will have to be in the RTL itself.

I used to have a whole bunch of separate VMs for each IDE release going back many years, then I lost them all in a total system crash, and I didn't have backups of them at the time. But I did have backup copies of just the \Source and \Include folders for 5-XE3 (minus 7-2005), so at least I can refer to them when needed, I just can't compile for them anymore. Nice! Still, if you could get history working properly, that would make for a cool way to search for revisions between releases.

Neither. You keep saying "I get this" and show a call stack for a memory allocation. Why do you keep mentioning that, unless it is causing a problem for you? WHERE and WHEN are you getting that stack trace shown to you? I don't see any memory leak in this code, either. On the other hand, the stack trace you have shown is for a memory allocation made by the 1st call to IndyFormat() inside of TIdFormDataField.FormatHeader() while the HTTP post data is being prepared by TIdHTTP.Post(). But, FormatHeader() appends additional substrings (6, in your case) to the String returned by that 1st IndyFormat() call, so it doesn't make sense why you should be seeing a stack trace for only the 1st memory allocation, and not for other memory allocations (unless you just didn't show them?)

Fair enough. So, then I suggest Process Explorer to check open handles to the file at the time the problem occurs. Maybe something else is opening the file between the time you create it and the time you use it. Antivirus, perhaps? The alternative is to simply not use a file at all. As I demonstrated earlier, it is possible to give TIdMultipartFormStreamStream a TStream for posting. For instance, you could write your data to a TMemoryStream and then Post() that instead of a disk file. There is no point in performing the call to TIdHTTP.Post() in a loop that handles EFOpenError, because that exception will never happen. The file is not opened inside that loop, it is opened by the call to TIdMultipartFormDataStream.AddFile(). So, if the file can't be accessed by TIdMultipartFormDataStream, you are not catching that error. WHERE do you get that, exactly? That is a call stack for a memory allocation, so again I ask, are you trying to report a memory leak? That is a completely different issue than a file access error. Go up to the "Find" menu, choose "Find Handle or DLL...", and enter the name of your data file. You will get back a list of every open handle to the file, if any, including the name of the process that each handle belongs to. Correct.

Only if the app is written in Delphi or C++Builder to begin with...

First, you need to figure out where the file is actually open. Use a tool like SysInternals Process Explorer to see who has open handles to the file. If it turns out to be your app, then check your code to make sure you are closing all of your open handles to the file, and that if you need to open multiple handles then don't open them with conflicting permissions. You do know it is possible to get a file's size without actually opening a handle to the file, don't you? You can get the file's size from the filesystem's metadata for the file, such as with SysUtils.FindFirst(), rather than querying the size from the file itself. You shouldn't be using the file size as an indicator that the file is ready for use, though...

Then why mention it? Yes, it is. Yes. Though, behind the scenes, the OS may take some time to actually flush the data to the physical disk in the background, but its data cache will handle any requests for file data that hasn't been flushed yet. So, in that regard, if you KNOW the saving process has finished writing its data to the file before you call SaveTheSolvedSlice(), then you don't need the loop to wait to access the file. Just open the file and it will either success or fail, act accordingly: function TMain.SaveTheSolvedSlice(FileName: String): Boolean; var MimeType, Response, FullFilePath: String; jsonVal: TJSonValue; Stat, Size: String; SendParams: TIdMultiPartFormDataStream; Param: TIdFormDataField; //HTTPSendFile: TIdHTTP; begin Result := False; FullFilePath := 'tmp\out\'+FileName; if (not FileExists(FullFilePath)) then Exit; Response := 'NULL'; HTTPSendFile.Request.Clear; HTTPSendFile.ReadTimeout := 2000; HTTPSendFile.ConnectTimeout := 9000; SendParams := TIdMultiPartFormDataStream.Create; try MimeType := 'application/octet-stream'; //Params.AddFormField('_token', 'Myfb9OqYgDBwDws3zTL9QOs492XWfNtGLftUdNsH'); Param := SendParams.AddFile('slice', FullFilePath, MimeType); Param.ContentTransfer := 'binary'; StatusUpdate(Console, 'Attempting to post file ' + FullFilePath + ' (size: ' + Param.FieldStream.Size.ToString + ' bytes)'); Delay(10); Response := HTTPSendFile.Post(STORAGE_OUT_REPO, SendParams); Delay(5); if (Response <> 'NULL') then begin //{"Slice":{"Status":"ACCEPTED","FileName":"1_Some_Job_Filename.slc","Size":1812}} jsonVal := TJSonObject.ParseJSONValue(Response); try Stat := jsonVal.GetValue<String>('Slice.Status'); Size := jsonVal.GetValue<String>('Slice.Size'); finally jsonVal.Free; end; StatusUpdate(Console, 'Toasted Slice (size: ' + Size + ') ' + Stat.ToLower); Result := (Stat = 'ACCEPTED'); end; finally SendParams.Free; end; end; That being said, your ExportSliceToBinFile() is leaking MySliceFile, so it is leaving the file open. MyOPLToolPathFile.Free() should be MySliceFile.Free() instead. And it should be protected with a try..finally: function ExportSliceToBinFile(MyFileName: String): Boolean; var I: Integer; MySliceFile: TFileStream; begin try ForceDirectories('tmp\out'); MySliceFile := TFileStream.Create('tmp\out\'+MyFileName, fmCreate or fmOpenWrite); try for I := 1 to GlobalIndex do begin MySliceFile.WriteBuffer(GlobalSliceRecord[I], SizeOf(SLICE_RECORD)); end; finally MySliceFile.Free; end; Result := True; except Result := False; end; end;

Indicating what exactly? A memory leak? I would not do it that way. TIdMultipartFormDataStream.AddFile() opens the file for read-only access and denies further writing to the file. If you can't wait for the file to be fully written to disk before calling SaveTheSolvedSlice(), then I would suggest using a TFileStream in a loop to open the file for exclusive access until successful, and then pass the opened TFileStream to TIdMultipartFormDataStream.AddFormField(TStream), eg: function TMain.SaveTheSolvedSlice(FileName: String): Boolean; var MimeType, Response, FullFilePath: String; jsonVal: TJSonValue; Stat, Size: String; SendParams: TIdMultiPartFormDataStream; TOPLSliceFile: TStream; //HTTPSendFile: TIdHTTP; begin Result := False; FullFilePath := 'tmp\out\'+FileName; if (not FileExists(FullFilePath)) then Exit; Response := 'NULL'; HTTPSendFile.Request.Clear; HTTPSendFile.ReadTimeout := 2000; HTTPSendFile.ConnectTimeout := 9000; SendParams := TIdMultiPartFormDataStream.Create; try MimeType := 'application/octet-stream'; repeat try TOPLSliceFile := TFileStream.Create(FullFilePath, fmOpenRead or fmExclusive); except on E: EFOpenError do begin // I know, this is not the best option, but it is the only way to detect // a sharing violation error with TFileStream. The alternative is to call // CreateFile/FileOpen() and GetLastError() directly, and then create a // THandleStream from the result... if E.Message <> 'The process cannot access the file because it is being used by another process.' then raise; Delay(100); end; end; until False; try //Params.AddFormField('_token', 'Myfb9OqYgDBwDws3zTL9QOs492XWfNtGLftUdNsH'); SendParams.AddFormField('slice', MimeType, '', TOPLSliceFile, FileName).ContentTransfer := 'binary'; Delay(10); StatusUpdate(Console, 'Attempting to post file ' + FullFilePath+' (size: ' + TOPLSliceFile.Size.ToString + ' bytes)'); Delay(10); Response := HTTPSendFile.Post(STORAGE_OUT_REPO, SendParams); Delay(5); if (Response <> 'NULL') then begin //{"Slice":{"Status":"ACCEPTED","FileName":"1_Some_Job_Filename.slc","Size":1812}} jsonVal := TJSonObject.ParseJSONValue(Response); try Stat := jsonVal.GetValue<String>('Slice.Status'); Size := jsonVal.GetValue<String>('Slice.Size'); finally jsonVal.Free; end; StatusUpdate(Console, 'Toasted Slice (size: ' + Size + ') ' + Stat.ToLower); Result := (Stat = 'ACCEPTED'); end; finally TOPLSliceFile.Free; end; finally SendParams.Free; end; end;