Fighting the bloat: Delphi links unnecessary files (COM library projects)

[A.M. Hoornweg 144]

Hello all,

 

in my quest to port a very large project -containing dozens of COM DLL's- from Delphi XE to Sydney, the main obstacle for me was the exploding executable size.  Each and every DLL in my project group had grown between 1-4 megabytes and that really added up painfully.  Most of that growth, of course,  is due to the new extended RTTI (which I do not use).   The project was already huge before, and a doubling or tripling in size would cause deployment problems (the software is used on oil wells on remote locations and bandwidth is often costly and slow).

 

Trying to strip off some MB's wherever I could, I started studying the MAP files to make sure no unnecessary stuff was linked in.

 

I noticed that all of my non-visual COM DLL's nevertheless pull in huge chunks of the visual VCL, including units VCL.Forms, VCL.Themes, VCL.Imagelist etc.  I am absolutely sure that I don't use these routines anywhere and still they make up over 60% of the executable's code! The underlying cause is that all of the xxxx_TLB.PAS files, which the IDE auto-generates,  use a unit called OleServer which has an initialization section. No code in the unit itself is called, but the initialization section manages to pull in most of the VCL for whatever reason.

 

As a test, I made a dummy unit "oleserver.pas" and referenced it as the first file in my COM DLL applications. The project compiled fine, and this change alone instantly reduced executable size with a whopping 1 MB.  And the best thing, the project still worked as expected. So it seems to me that this whole OleServer unit is an unnecessary thing in my projects.

 

Another thing. When I link my projects against a "stripped" version of the RTL/VCL (recompiled without extended RTTI), the size difference becomes astronomic.   My DLL's have now typically lost 80% of their size, going from 2MB+ to only 400kb.   With these executable size reductions I am finally able to port this project to Delphi Sydney.

 

 

 

I am in the process of writing a tool to automate the "rtti stripping" of the RTL/VCL which I plan to release as open source.  For example, projects like "inno setup" might benefit from it. 

 

 

 

 

 

 

 

Edited March 2, 2021 by A.M. Hoornweg
fixed a typo

[Kryvich 165]

I would like to have a compiler option to disable modern RTTI throughout the application, including the linked standard library. Glad to hear that you managed to disable this information in your project.

[Lajos Juhász 293]

26 minutes ago, A.M. Hoornweg said:

I noticed that all of my non-visual COM DLL's nevertheless pull in huge chunks of the visual VCL, including units VCL.Forms, VCL.Themes, VCL.Imagelist etc.  I am absolutely sure that I don't use these routines anywhere and still they make up over 60% of the executable's code! The underlying cause is that all of the xxxx_TLB.PAS files, which the IDE auto-generates,  use a unit called OleServer which has an initialization section. No code in the unit itself is called, but the initialization section manages to pull in most of the VCL for whatever reason.

I think is a poorly engineered code for the IDE. As I scanned the code most probably only the initialization section requires the VCL.Controls and for this one:

 

GroupDescendentsWith(TOleServer, Vcl.Controls.TControl);

 

I've no idea where is this used as find in files returns that RegGroups is only used in System.Classes so my guess is that it is for the IDE. Why they polluted the code only to be used in the IDE? Most probably it could be moved in some package used only by the IDE and not pollute user executables. I wonder what @Marco Cantu, @David Millington or @Dmitry Arefiev can publicly say about this case.

[Stefan Glienke 2002]

1 hour ago, A.M. Hoornweg said:

Most of that growth, of course,  is due to the new extended RTTI

This cannot be because XE already had extended RTTI. It's more likely that internal refactorings of the RTL and VCL such as using generic lists instead of good old Classes.TList and Contnrs.TObjectList contributes to the bloat. Of course with RTTI being enabled on those lists it leaves all the typically inlined method calls in the binary.

I wonder what difference {$WEAKLINKRTTI ON} would have.

Edited March 2, 2021 by Stefan Glienke

[A.M. Hoornweg 144]

45 minutes ago, Stefan Glienke said:

This cannot be because XE already had extended RTTI.

I use a RTTI-stripped version of the XE libraries since ages, because when I migrated from Delphi 2009 to XE my executable sizes also grew a lot.   That RTTI stripping is much easier to achieve with XE because the {$RTTI....} directive was still a global option then, so you don't need to patch any source files.  Stripping RTTI is now an integral part in my Finalbuilder build scripts.

 

 

Edit: I have attached an image with the executable sizes. 

 

All executables were compiled in "release mode" without any debugging info. All executables were compiled with  {$WEAKLINKRTTI ON}  and {$RTTI EXPLICIT METHODS([]) PROPERTIES([]) FIELDS([])}  in the DPR.

 

(1) Compiled with XE10.4.2 Sydney.

(2) As (1), replaced unit "Oleserver" with a dummy unit so no unnecessary VCL code is pulled in. 

(3) As (2), using a RTTI-Stripped RTL/VCL.

(4) Compiled with Delphi XE, replaced unit "Oleserver" with a dummy unit so no unnecessary VCL code is pulled in. 

(5) As (4), using a RTTI-Stripped RTL/VCL.

 

 

 

 

Edited March 2, 2021 by A.M. Hoornweg
Added image

[Stefan Glienke 2002]

2 hours ago, A.M. Hoornweg said:

{$RTTI EXPLICIT METHODS([]) PROPERTIES([]) FIELDS([])}  in the DPR.

Which does absolutely nothing since like XE7 or so when they fixed the $RTTI switch to be local to the current unit.

[A.M. Hoornweg 144]

Just now, Stefan Glienke said:

Which does absolutely nothing since like XE7 or so when they fixed the $RTTI switch to be local to the current unit.

I haven't ported the project group from XE to Sydney yet, for now the directive is needed for XE where it applies to all units that the compiler rebuilds.  I just wish Embarcadero had optimized RTTI for size from the very beginning and given us the choice wether to use it at all.  Having RTTI in your code offers possible attack vectors for reverse engineering etc.

 

 

[Attila Kovacs 629]

Well, no pain, no gain. Used to I also stripped every RTTI because I was annoyed of the binary sizes, but once you get hooked with RTTI and its benefits, there is no back.

I can remember watching those code snippets and conversations back to the days on G+ and always thought, man, for WTF do they need that, they are crazy.

[KodeZwerg 54]

1 hour ago, Stefan Glienke said:

Which does absolutely nothing since like XE7 or so when they fixed the $RTTI switch to be local to the current unit.

Since it is like you said, I do {$I includeme.inc} with above mentioned content straight after every 'unit xyz;' line of code (mostly just for libraries, executables only when it matters).

Saves here and there some bytes, depending on used units.

[A.M. Hoornweg 144]

1 hour ago, Attila Kovacs said:

Used to I also stripped every RTTI because I was annoyed of the binary sizes, but once you get hooked with RTTI and its benefits, there is no back.

To each his own.  For single-exe projects it is a non-issue and I wouldn't bother either if my project consisted of only a handful of modules.  

[Kryvich 165]

If a single-exe project is some command line utility, which is called very often, the size matters. I'd prefer if RTTI will be disabled by default, and it could be enabled for types where it really needed. 

[David Millington 79]

On 3/2/2021 at 11:16 AM, Lajos Juhász said:

my guess is that it is for the IDE. Why they polluted the code only to be used in the IDE? ... I wonder what @Marco Cantu, @David Millington or @Dmitry Arefiev can publicly say about this case.

Looks like something unintentional, we'd certainly want to fix. Can you file a QP with a sample project please (to be sure we see exactly what you see) and email me the QP number? david.millington@embarcadero.com. Thankyou!

[A.M. Hoornweg 144]

1 hour ago, David Millington said:

Looks like something unintentional, we'd certainly want to fix.

I'll do that (create the QP), thank you.

(edit) https://quality.embarcadero.com/browse/RSP-33241

Edited March 8, 2021 by A.M. Hoornweg