Jump to content
Dave Novo

64 bit compiler running out of memory

Recommended Posts

Hello,

We are using Delphi Seattle.

 

We have a ~ 2M LOC program with hundreds of units. The 64 bit compiler is running out of memory when compiling. Even when we set to "use MSBuild" to compile in a separate process. The separate process dies with an out of memory around 2GB of RAM usage.  The VM I am using has 8 GB of RAM and plenty of spare RAM when the process dies.

 

Aside from breaking the application into different packages/projects, does anyone have any bright ideas.

 

We are in the process of upgrading to Delphi 10.4.1.  Is the 64 bit compiler still a 32 bit app?

 

Share this post


Link to post
2 minutes ago, Dave Novo said:

The 64 bit compiler

There is no Delphi 64 bit compiler. There is only a 32 bit compiler able to generate 64 bit executable 🙂

 

Share this post


Link to post

The easiest way to kill the compiler is to have huge cyclic unit dependencies. Try to reduce that. Also if the project was created before unit scope names were introduced you should add the scope names to the units in the uses lists and remove from project options in Unit Scope Names. 

  • Like 3

Share this post


Link to post
1 hour ago, Uwe Raabe said:

Perhaps this may  help a bit: 

UsesCleanerSource.zip

I can definitely recommend this tool. I ran it over my 4M line project group and it did a fantastic job. The only things I had to manually fix where where I was referencing a type or function with the unit name, e.g SysUtils.StringReplace(....) had to change to System.SysUtils.StringReplace(....).

 

This will also speed up the compile time as the compiler spends less time doing namespace lookups. 

 

Share this post


Link to post

@Uwe Raabe - thank you for the tool. After I made it compile for Delphi I ran it, and all it seemed to do was rearrange my uses clause. An example

image.thumb.png.2fc5a63a68cace39fb57d181895a93b5.png

 

If the unit did have the fully qualified name, it moved them to the top of the uses clause, but otherwise left things alone.  i.e.

image.thumb.png.abd6b6d68b54e4ff3e161a0290737fc8.png

 

Am I missing something to make it prefix the fully qualified namespaces?

Share this post


Link to post
9 hours ago, Dave Novo said:

 

Am I missing something to make it prefix the fully qualified namespaces?

Perhaps you are missing a proper config file (default is UsesCleaner.cfg next to the exe). If there is none, the program will use the default settings which are - well, somewhat special:

 

procedure TSourceFileUsesClauseFormatter.InitSettings;
begin
  UsesHelper.UnitAliases := 'WinTypes=Winapi.Windows;WinProcs=Winapi.Windows;DbiTypes=BDE;DbiProcs=BDE;DbiErrs=BDE;';
  UsesHelper.UnitScopeNames := 'Winapi;System.Win;Data.Win;Datasnap.Win;Web.Win;Soap.Win;Xml.Win;Bde;System;Xml;Data;Datasnap;Web;Soap;Vcl;Vcl.Imaging;Vcl.Touch;Vcl.Samples;Vcl.Shell';
  UsesHelper.SearchPath := 'c:\Program Files (x86)\Embarcadero\Studio\19.0\lib\win32\release';
  UsesHelper.GroupNames := '<UnitScopeNames>';
end;

In your case, the SearchPath may be the culprit, which may cause the standard units not to be found.

 

A decent config file can look like this (with an adjusted SearchPath entry of course):

 

[Settings]
Indentation=2
Compressed=0
MaxLineLength=130
SearchPath=c:\program files (x86)\embarcadero\studio\21.0\lib\Win32\release;
UnitAliases=WinTypes=Winapi.Windows;WinProcs=Winapi.Windows;DbiTypes=BDE;DbiProcs=BDE;DbiErrs=BDE;
UnitScopeNames=Winapi;System.Win;Data.Win;Datasnap.Win;Web.Win;Soap.Win;Xml.Win;System;Xml;Data;Datasnap;Bde;Web;Soap;Vcl;Vcl.Imaging;Vcl.Touch;Vcl.Samples;Vcl.Shell
GroupNames=@DelphiOTA;Winapi;System.Win;System;Data;FireDAC;Vcl

[Groups]
DelphiOTA=ToolsAPI;DesignIntf;DesignEditors

 

  • Thanks 2

Share this post


Link to post
34 minutes ago, Uwe Raabe said:

Perhaps you are missing a proper config file

The config file helped for me :classic_smile:

 

One thing though: The conversion doesn't preserve the file encoding; The new file is written as ANSI even if the source file was UTF-8 encoded.

Share this post


Link to post
17 minutes ago, Anders Melander said:

The conversion doesn't preserve the file encoding; The new file is written as ANSI even if the source file was UTF-8 encoded.

Good point!

Share this post


Link to post
16 hours ago, Dave Novo said:

 

Aside from breaking the application into different packages/projects, does anyone have any bright ideas.

This is the correct solution and the Delphi ide/compiler 64 bits is far. 
 

I faced this problem on a daily basis, the project became too big, and a solution adopted by us was to divide the project into packages, so as not to overload with a single compilation. You can for example divide main project, FMX package and RTL package.

  • Like 1

Share this post


Link to post
10 hours ago, vfbb said:

Delphi ide/compiler 64 bits is far

Kids these days - making them 64bit would be the lazy solution - being more reasonable with allocating memory would be the good one.

Especially with confirmed performance issues in the compiler that are caused by unnecessary heap allocations.

  • Like 2
  • Haha 2

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×