Reduce exe weight : Link with runtime package

[gioma 20]

Hi,
I have a project in Delphi 12.2 of an exe that extracts other EXEs always made with delphi. I would like to reduce the weight of these executables, I thought of using the "Link with runtime package" option and linking the programs with the two largest BPLs VCLxxx.BPL and RTLxxx.BPL.
The problem is that adding RTL to the list of "Runtime packages" does not produce any reduction in the weight of the EXE.
Why I get this result?
Thanks

[Remy Lebeau 1467]

Does your project use multiple build configurations (debug/release, 32bit/64bit, etc)? Do you have the runtime packages applied to every configuration? 

[gioma 20]

I only set these options for the release build.

 

A strange thing happens: if I add only "vcl" when I start the EXE, it requires both VCL and RTL.
Why is this?

Edited Tuesday at 08:50 AM by gioma

[Remy Lebeau 1467]

12 minutes ago, gioma said:

if I add only "vcl" when I start the EXE, it requires both VCL and RTL.
Why is this?

Because the VCL depends on the RTL. If you use the Runtime Package for the VCL then you need the Runtime Package of the RTL as well. 

[gioma 20]

But I don't get it, compared to two BPLs weighing a total of about 22 MB, I only have a saving of 6 MB... mah!

[Uwe Raabe 2083]

That is most likely because you only use a small part of these packages. When building a monolithic exe the compiler will only take the parts actually used, but the packages contain a lot more.

[A.M. Hoornweg 147]

I have made an application that copies the Delphi RTL/VCL source files into a temporary folder,  inserts the text '{$RTTI EXPLICIT METHODS([]) PROPERTIES([]) FIELDS([])}' in the right places and re-builds the whole lot.  The compiled *.dcu files contain a lot less RTTI than before. 

 

By linking against this compacted RTL I achieve a size reduction in a 64-bit VCL application from 14 MB to 10.3 MB using Delphi 12.

 

For just a single executable that isn't worthwhile, but I use this in some multi-module projects that consist of a few dozen ActiveX EXE's and DLL'S. There the cumulative size reduction is substantial.   Proper testing is hugely important, one has to be sure that the missing RTTI doesn't cause the program to malfunction. 

[Ian Branch 131]

If it simply the size of the .exe you are looking to reduce, I use UPX, https://github.com/upx/upx/releases,  it handles 32 & 64 bit Apps.

I use a compression value of 7 and get a roughly 30% reduction in .exe size.  It will also handle wild card file names, i.e. upx -7 *.exe.

[Brandon Staggs 318]

24 minutes ago, Ian Branch said:

If it simply the size of the .exe you are looking to reduce, I use UPX, https://github.com/upx/upx/releases,  it handles 32 & 64 bit Apps.

I use a compression value of 7 and get a roughly 30% reduction in .exe size.  It will also handle wild card file names, i.e. upx -7 *.exe.

Never do this. It's pointless, defeats stuff the OS does to make accessing data in the file efficient, triggers more false-positives, and provides no real benefits, unless you are dealing with a severe lack of storage on the deployment target, which should be fixed in other ways. You can zip the file for transmission bandwidth savings.

[aehimself 402]

1 hour ago, Ian Branch said:

If it simply the size of the .exe you are looking to reduce, I use UPX, https://github.com/upx/upx/releases,  it handles 32 & 64 bit Apps.

I use a compression value of 7 and get a roughly 30% reduction in .exe size.  It will also handle wild card file names, i.e. upx -7 *.exe.

I'd strongly advise against UPX especially in production executables. I had my fair share of after-the-last-moment struggle because of it

 

[Ian Branch 131]

Hi Guys,

I acknowledge the previous two commentors and their valid inputs, however it is horses for courses.  I was simply offering a suggestion.

I have used UPX for many years without any issues.  Note:  I use Eurekalog not MadExcept.

It provides not only compression, but also therefore a small level of obfuscation to casual file browsers.

That is all I need.

4 hours ago, Brandon Staggs said:

defeats stuff the OS does to make accessing data in the file efficient

Like?

4 hours ago, Brandon Staggs said:

You can zip the file for transmission bandwidth savings.

Yes I could, but, the other end is so computer illiterate that they would screw up the unzip and save process.  Pacific island users.  It has to be simple for them to just copy into their directory.  Even then they screw it up sometimes...:-(

 

[shineworld 79]

I use UPX by decade without any issue also with Virustotal and installed BitDefender. I guess it expand program in RAM.
My 38.681.088 byte exe are compressed with LZMA to 10.189.312 byte with "upx-4.2.2.exe --best --lzma normal-version.exe upx-version.exe"

Edited yesterday at 07:04 AM by shineworld

[A.M. Hoornweg 147]

The problem with using UPX is the following.

 

Normally Windows would open an executable file using a "memory mapped file" mechanism instead of bluntly allocating memory and loading the contents. This allows the Windows virtual memory manager to efficiently "page in" sections of the executable as needed (on demand paging), improving performance and minimizing memory use.

 

If the file is compressed using UPX, execution starts in the UPX loader stub which immediately allocates a big block of memory and extracts the executable into that before proceeding with the "real" execution. This simply wastes precious resources.

[Ian Branch 131]

Just now, A.M. Hoornweg said:

Normally Windows would open an executable file using a "memory mapped file" mechanism instead of bluntly allocating memory and loading the contents. This allows the Windows virtual memory manager to efficiently "page in" sections of the executable as needed (on demand paging), improving performance and minimizing memory use.

 

If the file is compressed using UPX, execution starts in the UPX loader stub which immediately allocates a big block of memory and extracts the executable into that before proceeding with the "real" execution. This simply wastes precious resources.

Interesting.  I was not aware of that.  Food for thought.

On what basis does UPX allocate the space/block of memory?

I guess this wouldn't really be an issue with modern Windows/PCs, but my Customer's Users are using older Win 7 SP1 PCs with 4GB of memory.

[David Heffernan 2364]

36 minutes ago, A.M. Hoornweg said:

This simply wastes precious resources

I mean, the memory can get paged out again so it's not that terrible. 

 

Another issue is that the paging system can't share the memory between processes if it's in memory rather than loaded off disk. 

 

I still don't understand what actual problem people who want to make their executables a few mb smaller are trying to solve. 

Edited yesterday at 09:13 AM by David Heffernan

[A.M. Hoornweg 147]

38 minutes ago, David Heffernan said:

I still don't understand what actual problem people who want to make their executables a few mb smaller are trying to solve. 

It adds up in some projects consisting of dozens of dlls and executables.  I had to cope with deployment over slow connections paid by data volume until quite recently. 

[Anders Melander 1848]

3 hours ago, David Heffernan said:

I mean, the memory can get paged out again so it's not that terrible. 

Yes it is because it trashes the page cache.

If memory is low then something else has to be paged out before the exe can be extracted to memory - and then paged out.

[Anders Melander 1848]

2 hours ago, A.M. Hoornweg said:

I had to cope with deployment over slow connections paid by data volume until quite recently. 

So use an installer or zip the files.

I'm not buying the argument that users can't figure out how to unzip a file or run an installer. IME, they can if they have to.

[A.M. Hoornweg 147]

5 minutes ago, Anders Melander said:

So use an installer or zip the files.

I have *always* used installers. 

 

Nowadays executable size is not much of an issue anymore, thanks to better networks.  But in a not so distant past I had to regularly push software updates to locations that were using T&T Inmarsat Mini-M satellite terminals. Imagine the slowest and most expensive connection imaginable, 2400 bps, costing like 5 euros per minute.  I'm glad those days are over.

 

 

[Brandon Staggs 318]

5 hours ago, A.M. Hoornweg said:

It adds up in some projects consisting of dozens of dlls and executables.  I had to cope with deployment over slow connections paid by data volume until quite recently. 

Then fix the deployment process instead. Compressing the EXE instead of using a compressed data package for deployment is at best lazy. Even a basic apache web server can be configured to compress your binaries for downloading of the client and you wouldn't have to do anything special to support it.

[A.M. Hoornweg 147]

16 hours ago, Brandon Staggs said:

Then fix the deployment process instead. Compressing the EXE instead of using a compressed data package for deployment is at best lazy. Even a basic apache web server can be configured to compress your binaries for downloading of the client and you wouldn't have to do anything special to support it.

What are you talking about??? I'm not compressing any EXEs.  I'm just preventing that 30-40% of unused stuff gets linked into them (it would be nice if Delphi's linker had such an option). And of course I use highly compressed setups.  

[Brandon Staggs 318]

4 hours ago, A.M. Hoornweg said:

What are you talking about??? I'm not compressing any EXEs.  I'm just preventing that 30-40% of unused stuff gets linked into them (it would be nice if Delphi's linker had such an option). And of course I use highly compressed setups.  

I'm speaking in general terms about people who are obsessed with compressing EXEs, as if it is a legitimate solution to create un-pageable files just to avoid proper deployment methods.

 

As to  that "unused stuff," Delphi's compiler does exclude a lot of unused code if you don't build your EXEs with shared libraries. Could it be better? Probably, but, when RTTI gets involved, things aren't so straightforward.