Memory Management with many objects

[chkaufmann 17]

I have to solve some "out of memory" problems during an import in my application. My application is still 32bit and when I check the memory in the process explorer, it's not 100% clear for me, which values are critical. Here are my questions:

- What is an easy way to see the number of objects of each class during debugging? Do I have to build something on my own? Can I get this info for records as well?

- Is it worth to switch the application to 64bit?

- For 32bit, which values in the Process Explorer are critical and what are the maximum values during runtime that will be save?

 

Thanks for any hints and tips here.

 

Christian

[Guest]

1) Inherit all your objects from one extended e.g. TObjectWithCounter , use a compiler directive to enable/disable this behaviour 

2) If your application is reaching 2GB then definitely you should have 64bit version, although 32bit can handle up to 3GB when care is taken and you don't have some 3rd party library to mess with things.

3) Working Set, but if you see "Out of Memory" then your private bytes are hitting the virtual size limit. other counters important too and you should understand them as any other developer, so i will add a link to few articles that you really should read them to understand your OS

https://docs.microsoft.com/en-us/archive/blogs/markrussinovich/pushing-the-limits-of-windows-virtual-memory

 

and good luck

[David Heffernan 2345]

Often it's not how much memory you use, but how you use the address space. 

 

But if you can, just switch to 64 bit and move on. 

[Fr0sT.Brutal 900]

I'd recommend to check if such memory consumption is expected. Then you can use FastMM's debug facilities to check allocated items, their exact size, types and many other details.

[Lars Fosdal 1792]

That was my first reaction as well - do you have a memory leak - i.e. allocate objects which are never disposed of?

 

Private bytes is the one to watch.  Our services go belly up when it reaches about 1.6Gb - due to something leaking.  

I use Process Hacker 2 over Process Explorer as I can get at more statistics for each process with that one.

 

As suggested above, use FastMM to check for leaking objects.

Check the number of handles allocated to the application. 

 

Do you do allocations through Windows APIs?

Are those deallocated?

[Dalija Prasnikar 1396]

If you can switch to 64bit, this is the fastest way to solve intermediate memory issues that are not caused by memory leaks.

 

However, additional information about what exactly you are doing would be helpful. Memory issues can have number of causes and solutions depend on root cause.

 

For instance, what kind of file(s) you are importing? How are you processing them?

 

Importing 300 MB XML can easily kill 32-bit application. Even smaller than that can be problematic depending on particular code. Also, if you are parsing textual data, using UTF8String for processing generally reduces memory consumption by half (for Western character sets). Creating DOM is also wasteful, using SAX parsers for XML is faster and consumes less memory.... base64 encoding, decoding can also be a memory killer if you are doing it in memory, with strings... 

 

Like, I said... more information is needed for better diagnostic...

[Guest]

50 minutes ago, Lars Fosdal said:

Our services go belly up when it reaches about 1.6Gb - due to something leaking.  

First thought here is bizarre memory fragmentation, this can happen if the application have many thread that been created and destroyed, because the reserved stack for each thread by default is 1 mb , and OS will try to find 1mb as whole to reserve, in case of failure to find such one piece the symptoms and messages shown will be red herring.

 

31 minutes ago, Dalija Prasnikar said:

Like, I said... more information is needed for better diagnostic...

Exactly, it is easier to start teaching someone how to fish,... at least he will have better questions to ask, e.g. such question will be less frequent "is this a fish or a duck ?" !

 

 

In all cases i recommend for all to get familiar with VMMap from Sysinternals https://docs.microsoft.com/en-us/sysinternals/downloads/vmmap

This great tool is the best to get the whole picture what is going under the hood.

 

SPOILER : not many knows that when a 32bit app runs on 64bit OS, then the system will reserve 2 stacks for each thread in such application ! one for the 32bit that the app is factually is using and one for WOW64 (default with 256kb) to work with it ( shadowing , emulating ...)

[Fr0sT.Brutal 900]

Btw, keep in mind that trivial TStrings.LoadFromStream for ASCII-encoded files eats up to 5x of file size at some moment. And 2x piece of that 5x must be allocated in a whole (so with fragmented memory and insufficient swap file it could fail even with ~500 MB)

[vfbb 285]

Out of memory

I really like to use Grijjy Cloud Logger. You can monitor all objects in your program in real time, and each time you update this list it will indicate the growth (in number of instances and memory) of each class. Best of all, this can be done in both debug mode and release mode. Use this only in the development environment as there will be a small loss of performance.

 

Move to 64 bits

I advise you to start writing code with 64-bit support as soon as possible, this is the future, because although microsoft is not forcing this migration yet, several others have already done it, such as Android, iOS and OSX which today only allow 64 bits. It is a trend even from Embarcadero itself since it added support for Linux only 64 bits. But, of course, keep support for 32-bit too, as there may be users with 32-bit windows still and also because the Win32 debugger is better than Win64. 😉

[Stefan Glienke 2002]

I just scribbled something together that should be good enough to start with and get some basic information about how many objects are created at a given time - needs to be first or directly after the memory manager unit in the dpr:

 

Replace the dictionary with a Spring4D one and you have some easy sorting and filtering on top 🙂

 

unit ObjectCounter;

interface

uses
  Generics.Collections;

function GetObjectCounts: TDictionary<TClass,Integer>;

implementation

uses
  madCodeHook;

type
  TObjectCounter = record
    class var ObjectCounts: TDictionary<TClass,Integer>;
    class var OldInitInstance: function (Self: TClass; Instance: Pointer): TObject;
    class var OldCleanupInstance: procedure (Self: TObject);

    class function InitInstance(Self: TClass; Instance: Pointer): TObject; static;
    class procedure CleanupInstance(Self: TObject); static;

    class procedure Init; static;
    class procedure Finalize; static;
  end;

function GetObjectCounts: TDictionary<TClass,Integer>;
begin
  Result := TObjectCounter.ObjectCounts;
end;

{ TObjectCounter }

class function TObjectCounter.InitInstance(Self: TClass;
  Instance: Pointer): TObject;
var
  count: Integer;
begin
  if ObjectCounts.TryGetValue(Self, count) then
    ObjectCounts[Self] := count + 1
  else
    ObjectCounts.Add(Self, 1);
  Result := OldInitInstance(Self, Instance);
end;

class procedure TObjectCounter.CleanupInstance(Self: TObject);
var
  count: Integer;
begin
  if ObjectCounts.TryGetValue(Self.ClassType, count) then
    if count = 1 then
      ObjectCounts.Remove(Self.ClassType)
    else
      ObjectCounts[Self.ClassType] := count - 1;
  OldCleanupInstance(Self);
end;

class procedure TObjectCounter.Init;
begin
  ObjectCounts := TDictionary<TClass,Integer>.Create;

  HookCode(@TObject.InitInstance, @InitInstance, @OldInitInstance);
  HookCode(@TObject.CleanupInstance, @CleanupInstance, @OldCleanupInstance);
end;

class procedure TObjectCounter.Finalize;
begin
  UnhookCode(@OldInitInstance);
  UnhookCode(@OldCleanupInstance);

  ObjectCounts.Free;
end;

initialization
  TObjectCounter.Init;

finalization
  TObjectCounter.Finalize;

end.

 

Edited January 30, 2020 by Stefan Glienke

[dummzeuch 1505]

If switching to 64 bits is not feasible, you could try to enable IMAGE_FILE_LARGE_ADDRESS_AWARE which might solve the issue for now.

 

(no idea where this formatting comes from, apparently I can't turn it off)

[chkaufmann 17]

Thanks for all hints. I already use FastMM full version, so I know, there are no memory leaks. I run a huge import from one database to another. I have to pass the data through my business classes.

 

Since I use interfaced objects a lot, destruction is sometimes late. Combined with caching of some complex structures it's not always obvious, where there are still references. I will try to add logging with the code of Stefan.

 

Christian

[aehimself 396]

22 hours ago, Kas Ob. said:

First thought here is bizarre memory fragmentation, this can happen if the application have many thread that been created and destroyed, because the reserved stack for each thread by default is 1 mb , and OS will try to find 1mb as whole to reserve, in case of failure to find such one piece the symptoms and messages shown will be red herring.

I had this issue once on a 32 bit application running on a Windows 2000 test machine with 64-128 MB of RAM (strangely enough, Win2k3 with 128 MB ran the same code perfectly). Solution was easy though - instead of launching and destroying a thread when the work is done and recreating it when needed again, I just left the threads in an empty loop, signaling when I need them to do their job.

Took me a couple of days to figure it out, though 🙂

[Guest]

OK !
I want to share some information and explanation that might help someone to get better understanding of memory management specially if this someone is trying to build memory hungry application, even though some might think they already know everything no you don not, i myself don't know everything just enough about memory management, reading this is not complete is waste of time, i really hope i am not wasting anyone time, Delphi/Pascal after all is one of the best computer languages out there which let you go low level as much as you want, and Garbage Collector is garbage need to be collected in container when not done right, sorry i am not trying to offend anyone, but it is personal opinion and i respect all opinions.

 

Here i gonna use VMMap to review the memory usage of the Delphi IDE ( all has the same behaviour but here i am using Seattle), understanding and utilizing the best tools to test and debug is crucial to build better and solid software, and here lets try to understand why the performance degrade that much after opening and compiling few projects, same result with one project opened and compiled few times, how the developers of the IDE lack the understanding the low level fundamentals of memory management, few years back i had a conversation about most of those following facts over emails with someone from Embarcadero, he could change things but he didn't understand or didn't want to admit the wrong doing and the abuse of memory allocation, at the end thankfully he didn't call me stupid, (no naming here).

 

Things to watch and observe with facts:
1) if you open the IDE without opening a project, use VMMap and you will see hundreds of those 4k private data blocks allocation, now keep in mind one important thing : 4K is the page granularity allocation, but it will allocated according to space allocation granularity which is 64k , in other words your application just allocated 4k but in process killed/reserved a 64k that can't be allocated but only can be used by extending the 4k, for more details refer to this 2003 article https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223

2) VMMap reports 3GB as free space, fine , but it doesn't and will not report the rounded up of those 4k to 64k, it also reports the largest piece as 1.5GB, the largest is not much important yet but will have it huge impact later which will lead to the IDE crash, on side note: to know for fact the reserved/used memory for an application you have to walk through all those blocks ( shown in VMMap) and round up each size to n*64 and you will have the used memory.

3) open a project and compile it and see most of those 4k now are 64k, that is OK, wait it is great ,but not quite yet !, most of those 64k are sequenced and can be allocated differently in one piece, like they grow from 4k then why not to grow one big piece like 1mb from 4k, why is this important ? because if you miss the fact the OS VMM ( virtual memory manager is software) and it will have lookup tables to manage those allocating in thread safe manner ( even when they are sequenced ) then you missed the big picture, and this is critical and the longer/bigger this list/table the slower the OS MM, the longer in nano/micro seconds the memory operations will take, hence slower process, and this will have its impact on whole system ( all process's), example my chrome start to look like the IDE in flickering, closing the IDE restore the chrome fast response, Google Chrome doesn't have this ugly fragmentation.

4) use Cheat Engine to take a peek inside these 64k and the data between them, they contain a continuation of the data, so definitely can be allocated from bigger blocks.

5) before opening a project (3) there was something around 700 of Private Data allocated, that is understandable due the many BPL's and DLL's, and huge amount of data for Code Insight and internal search's..etc, after one project compilation we have 1300 private data block (this is very very small application), with each close project and open and compile those numbers are getting bigger rapidly, so is it memory leak ?, YES and NO, dangling stuff everywhere, sometimes they freed some times not !

6) Couple hours of using the IDE, now the allocating block reaching thousands and thousands, and the idea : if LSP or whatsoever it will be called will fix this, think again the problem in memory management itself, - have Embarcadeo developers heard about FastMM ?! it perform way better than what the ide is doing now, and moving chunk of that memory allocating to another process, for some reason might cover or lets call fix the memory bad management? ,may be or may be not, will it make faster ? may be for short time, IMO it will slow the performance degradation speed, and buy some time, nothing more, as for IPC between such abomination (again IMHO) will kill the performance for Code Insight, or the ide will use cache locally and guess what, it is the same result from, with the same thread synchronizing, i mean what can be better with the data been sent from different process that can't be internally generated ? , here i doubt utilizing named pipes or tcp connections or even shared memory will have less delay.

7) Comparing with Visual Studio 2017, running the IDE will report 176 allocated Private Data block, that is it only 176 !!!!, opening a medium/small project (mbedtls-2.16.3) and compile it successfully, that number raise to 331, close the project and reopen and recompile and repeat the same number, with BDS the numbers are .. you know, it is in hundreds increase.

😎 BDS IDE is using many runtime libraries and those will cause high memory usage, this is unavoidable, but are we sure it is the libraries wrong doing ?!, i mean there is many signals pointing in different direction here, take an example when the IDE become sluggish cleaning the project will inject little life in it for short period of time but watching the memory usage and the allocated blocks not been decreased or freed, then the question what cleaning is really doing, sorting pointers ?!... closing file handles ?!.. this is beyond my small corrupted brain.

9) Here i want to explain one of many hidden impact of fragmentation, so lets go little lower level, for each 4k page allocated there is 64k (aka 16 pages) reserved , and those pages snapped out of reach the non pages pool, can you see the point here ?, by fragmenting the memory with hundreds of blocks you have side effect on the non pages pool, and that is critical as those pages are the memory used and reserved for the kernel and device drivers, so the bigger your tables ( more fragmentation and allocation pieces) the slower non paged pool table will perform, example will had hard time to received 1mb from the afd driver ( socket driver ) to fit them in MTU chunks separated in page size across many 64k sizes, this example shows the effect which is very visible with heavy threaded sockets server, also this is very visible when you running two or three Delphi IDE's for long period of time and you see WireShark logging slower and slower TCP operations, even with relatively small traffic usage, same goes for file drivers ! but this is very known with the IDE and many think that memory is depleted, no it is not , it is just fragmented beyond help.

 

Now we return to the question, have they heard of FastMM ? do they understand it ? why the applications built with Delphi ( FastMM  by default ) is behaving way better than the IDE itself ?

Many thanks to Pierre le Riche for his FastMM, because without it those application will be slower than Java like 10 times.

 

And the moral from all of this, Embarcadero is not doing memory right, don't be like Embarcadeo, you see problem with memory then do your diligence and investigate, just don't go and add a skin your application and call it a day, failed on your own then ask who have seen things, this forum have very decent and professional people and most important they are helping for free.

My bad is English  !
Sorry for my English, i honestly tried almost quarter my best, i wasn't trying to offend anyone or any group of people or a product that i love.

 

I feel much better now, thank you for reading, and i really hope somehow i helped someone, not wasted times of many. 

 

ps: pasting brain thought as end of the topic for this, : from history of business model of Embarcadero it has tend to divide and sell separately, (like command line compiler ...etc) so if an abomination will be given birth it will have very decent chance to be online service with monthly subscription, and it will be only working with active subscription, so no high hopes, and please no discussion about it, lets wait and see !

ps2: pretty please don't steer this conversation beyond this line in discussing LSP ( or whatever) , lets keep it about memory helpful thoughts.

[chkaufmann 17]

I wanted to try your code, but I just realized that madCodeHook unit is not part of madExcept.

 

So is there another way or code sample to hook these functions globaly?

 

Christian

[Stefan Glienke 2002]

Probably with https://github.com/MahdiSafsafi/DDetours

[chkaufmann 17]

Thanks, this works.

 

But for objects only. Is there a general way to count the number of a "record" instance as well?

 

Christian

[shineworld 73]

I still have a lot of 32-bit applications and often found myself with memory problems because of the 2GB maximum limitation per project.

By keeping the application 32-bit you can add another GB to the available ram via FASTMM4 using {$SetPEFlags $20}.

Example:
 

program myprogram;

uses
    {$IFDEF CPUX86}
      {$SetPEFlags $20}
    {$ENDIF}
    FastMM4 in 'sources\memory-managers\FastMM4-4.992\FastMM4.pas',
    FastMM4Messages in 'sources\memory-managers\FastMM4-4.992\FastMM4Messages.pas',
    ...projects files

I'm using wonderful FastMM4 from pleriche:
https://github.com/pleriche/FastMM4

 

Intervention in the code is as simple as checking whether there are actual benefits to the application.
In my case SynEdit without {$SetPEFlags $20} and FastMM can load 1.5 million lines but with the patch can reach 4 million lines. 

That application gain per-processes extra memory when executed on a 64-bit windows OS.

https://stackoverflow.com/questions/1849344/how-can-i-enable-my-32-bit-delphi-application-to-use-4gb-of-memory-on-64-bit-win
 

Edited August 31, 2022 by shineworld