Allen@Grijjy

Been experimenting with it myself and there are things I like.  I will give a full report on it once they address some things I reported.

3 hours ago, Stefan Glienke said:

If any third party leak analysis tool claims that FastMM has a memory leak it probably will tell you the call stack of where it comes from and you will be able to find it.

Also are you aware that there is RegisterExpectedMemoryLeak function that the third party tool might not be aware of and has a false positive?

The example I made is trivial.  It expands a queue, so the size of the leak is directly proportional to the number of records you en-queue.  Even if you destroy the queue, the leak persists.  I am aware of RegisterExpectedMemoryLeak function.

The call stack shows the problem comes from the Collections.TListHelper.InternalSetCapacity in Delphi's collections unit, that eventually calls the ReallocMem routine:

LeakExample.exe!@ReallocMem$qqrrpvi Line 4900 004070b8
LeakExample.exe!DynArraySetLength$qqrrpvpvipi Line 35920 + 0x5 bytes 0040c466
LeakExample.exe!Generics.Collections.TListHelper.InternalSetCapacity Line 4489 + 0x9 bytes 004eb6f1
 

In the example I provided the actual collection is a queue of records that is created and destroyed.  However when the queue grows, FastMM internally calls FastReallocMem to expand and internally determines it's a large block reallocation.  If I simply redirect the ReallocMem routine back to SysReallocMem inside of FastMM, everything deallocates properly.  Like you said, it could be a flaw in this tool, but the problem doesn't happen with Delphi's built in memory manager.  It also doesn't happen if I change FastMM's source to simply call SysReallocMem immediately upon calling FastReallocMem.  It also doesn't happen with numerous other data types, just collections, specifically queues of records.

Attached is the smallest example I could make of a leak (it leaks roughly 1MB) that only occurs in FastMM4 (latest edition) with FullDebugMode disabled but does not occur with the default memory manager of Delphi 10.3.  To test it, just compare it with FastMM4 commented/uncommented in the .dpr.  You will need some external leak analysis tool like this one, for example,

https://www.deleaker.com

I honestly would like to understand what is happening here because I have also used FastMM over the years in numerous projects, but I found a few issues like this one and a couple of others related to reference counted interface de-allocations.

Note: I am aware that FastMM4's leak checker doesn't show any leaks, but like I said previously, I don't trust it to analyze itself.

LeakExample.zip

20 minutes ago, Stefan Glienke said:

The problem is not any 3rd party MM but the fact that some pieces in the RTL are deallocated within System.pas finalization which takes place after detaching/finalizing any 3rd party MM 

Out of curiosity, what are you all using to determine you are leaking or not?  Let's assume we ignore System.pas related things.  Have you compared the results of a tool like the Deleaker using the base memory manager with a large project vs. Deleaker with FastMM4 enabled (without full debug mode) or any other memory manager?

In my experience and tests, the latest FastMM4 has various leak related issues, some of them are quite bad with the newer versions of Delphi that don't exist with the built-in memory manager which is also a subset of FastMM.  Most of the memory managers haven't been actively worked on in years and I doubt their ability to properly handle many scenarios.

Interesting article, thanks for sharing it

Some thoughts on OpenSsl 1.1.1....We recently finished porting the OpenSsl 1.1.1a headers to Delphi for all platforms (Windows, macOS, Android, iOS and Linux, 32 and 64-bit where appropriate) and may write an article for our grijjy blog on that sometime soon.  The challenge is building in a way that works for each Delphi platform, which we also did in the process.  Deploying OpenSsl with your app in a way that is uniform for all platforms but does not interfere with legacy OpenSsl that is sometimes part of the OS is also a challenge.  The libraries for LibSsl and LibCrypto often cause dynamic linking issues on POSIX platforms when they attempt to reference one another.  We solved this with some creative linking that is platform specific.  Also it isn't wired into Indy, because we don't use Indy internally, but it shouldn't be too hard (for someone else to do).  The other challenge is TLS 1.3 isn't really completely working in  OpenSsl 1.1.1 and has a few outstanding issues.  You probably don't want to use TLS 1.3 at this time.

Based upon your statement that it's the same binary but it only freezes on one machine would make me focus on a hardware issue first.  Perhaps bad RAM, overheating CPU, etc.   Back in the day there were hundreds of thousands of PCs with faulty capacitors from most major manufacturers and over time the capacitors failed.  The result for our Delphi app was it crashed and became unstable on those machines.  It took us quite some time to figure out this cause.

GiT - don't waste your time on anything else.  Lots of client choices, but most all it has such broad community support.

Sorry, the code is so tightly coupled with our private code, it would take some time to decouple the pieces.  We may do a blog article on this in the future on our site, blog.grijjy.com.  The approach we took is to use the example ITestRunner that are included with Delphi samples and just intercept the test results and send them over the wire.  You could use almost any communication library to send them, or even log them locally if you want to avoid that step.

The way we do it is we created a small class to inherit from ITestListener and ITestListenerX that acts like a simple remote TCP client on the mobile device to intercept the results of the tests as they happen and a simple TCP server with a GUI on Windows using Indy that the client connects to display the tests.

I was wondering if there was a Delphi source code parser that could emit YAML compatible syntax for documenting Rest/HTTP Apis? 

EMS server does this, http://docwiki.embarcadero.com/RADStudio/Berlin/en/Custom_API_Documentation but I want a solution that doesn't require EMS server for regular Delphi code so I can easily target Swagger.  Any ideas?

Is that completion handler actually a objective-c block?  If so, you need to implement it as a block.  We discuss how to do this more easily from Delphi in an article we wrote on using Apple's Grand Central Dispatcher that might help,

https://blog.grijjy.com/2018/05/20/using-the-apples-grand-central-dispatch-and-androids-scheduledthreadpoolexecutor-for-delphi-timers/

Our new article about how to package iOS and macOS Application Extensions with your Delphi developed iOS and macOS application and interact with the Application Extension from Delphi using the Application Groups API.

https://blog.grijjy.com/2018/11/15/ios-and-macos-app-extensions-with-delphi/

25 minutes ago, Geoffrey Smith said:

Thanks for that.  Problem solved! 🙂 I was expecting Delphi to have a reasonable default for my first ever linux build.  Obviously I was wrong about it being reasonable!

It's probably that your Delphi library path has environment variables that point to this folder, but those environment variables are not working for some other reason.

Actually it seems it cannot find librtlhelper_PIC.a.  Make sure that C:\Program Files (x86)\Embarcadero\Studio\19.0\lib\linux64\debug or release is in your path.

Glad to be here as well.  I like the forums, pretty slick overall.