Main thread for getting 9.5 to work in C++ Builder 13.0

[w0wbagger 6]

Meant to start a new thread for this, sorry.

 

I've started running the build groups in C++ Builder 13.0. Going to use this thread to capture any learning, so folks who only use Delphi can skip it.

 

Okay, first thing, I dropped Win64 and Win64x from the build group to just try to get Win32 working first.

Trying to build, I got this error:

[DCC Fatal Error] OverbyteIcsWinnls.pas(112): F2039 Could not create output file '..\Source\Include\37.0\Win32\OverbyteIcsWinnls.hpp'

 

When C++ Builder attempts to create the .hpp files from the delphi files, it can't output them here, because the directory doesn't exist. Maybe add this to the folder structure in the .zip file? 

 

When I added the 37.0\Win32 folder, it finished and linked everything (I just need to install the design packages manually, although this could be automated as well).

HOWEVER, this was the directory structure after compilation:
 

There *were* two files in the Win32/37.0 Folder:
IcsCommonCBNewDesign.bpi and IcsCommonCBNewDesign.lib

So it seems as if there is inconsistent "pathing" in the C++ project files. I believe all of these files should be under ($BDSCOMMONDIR)\37.0\BPL\Win32, and ($BDSCOMMONDIR)\37.0\DCU\Win32, or more correctly (to preserve integrity as the version increases):

 

($BDSCOMMONDIR)\($ProductVersion)\BPL\($Platform) and
($BDSCOMMONDIR)\($ProductVersion)\DCU\($Platform)

There's an argument to be made to move HPP files to:
($BDSCOMMONDIR)\($ProductVersion)\HPP\($Platform)

as well, which I will, with your permission, Angus - I don't want to break too much, but these file paths have long been an issue for me compiling the C++ Versions.

 

I'm going to see if I can fix the project files so everything is where we expect it to be in C++ Builder, using the Embarcadero Macros ($Platform), etc. - will send you updated ones when I have this working.

 

I think you'll need to add the folder structure to the zip for the .hpp file output yourself, though - I don't think there's an option for "Create folder if it doesn't exist".

 

[w0wbagger 6]

FYI, I have the Win32 libraries compiling and installing correctly (to the correct paths), including using build groups, and have recompiled our app successfully with them. Need to test everything (including OpenSSL, etc), But will attempt Win64 next. If anyone needs these before I've finished the 64-bit testing, let me know. 

 

[Angus Robertson 683]

I was looking at C++ 13 yesterday, and have just uploaded new C++ packages to SVN, will be zipped overnight.  

 

I believe I've corrected all the Win64x paths and a few Win64 paths. 

 

Win32 built OK once I specified a Windows SDK to use, which strangely is version 10 and not 11, 

 

Win64 still gives undefined symbol errors, four in Common, dozens in Run, I've tried to fix these before but unsuccessfully. 

 

Win64x gives different errors, but I don't believe the linker bug is fixed yet the C++ compiler does not create the necessary files to allow other C++ packages to reference them, so you can build CommonRun but not CommonDesign or VCLRun which both need CommonRun.  An Embarcadero engineer investigated this last year for 12.3, but RSB-503 is still open.  

 

Perhaps you could check and update these latest packages as necessary, email them, and I'll put them in SVN quickly for others,  

 

Angus

 

[w0wbagger 6]

32-bit ICS v9.5 compiles and installs correctly and my app *appears* to run fine, although we haven't tested it comprehensively. I've not tested the FTPServer component's functionality, which was giving us problems in v9.3, so jury's still out on that.

 

Interestingly, when I try to explicitly compile the first 64-bit (NOT modern, just classic toolchain) project, it's asking me to remove the 32-bit components before I do this, even though I'm only trying to compile the common run-time project. This behaviour...shouldn't happen? It should let me recompile the common run-time in 64-bit without touching the installed 32-bit components (installed with the design-time projects, which I shouldn't have to compile at *all* for Win64, since the IDE only requires the 32-bit design-time components to be installed. Do I understand this correctly? We don't need the 64-bit classic design-time projects compiled as long as we're just using the 32-bit IDE?

 

[w0wbagger 6]

I resolved the prior problem by just not installing the components. When I compile the 64-bit, I am able to compile 64-bit IcsCommonCBNewRun, but can confirm I get only 4 unresolved externals when attempting to compile IcsVCLCBNewRun and icsFmxCBNewRun.  Maybe some issues with name-mangling?
These are the 4 unresolved external errors I'm getting. we don't get these with the Win32, so maybe some conditional compilation is at play here? Will investigate further.

[ilink64 Error] Error: Unresolved external 'Overbyteicscharsetutils::IcsSystemCodePage' referenced from C:\USR\SRC\3RDPARTYVCL\ICS V9.5\LIB\DEBUG\WIN64\37.0\ICS.FMX.OVERBYTEICSCHARSETCOMBOBOX.O
[ilink64 Error] Error: Unresolved external 'Overbyteicstypes::GWSockCritSect' referenced from C:\USR\SRC\3RDPARTYVCL\ICS V9.5\LIB\DEBUG\WIN64\37.0\ICS.FMX.OVERBYTEICSWSOCKET.O
[ilink64 Error] Error: Unresolved external 'Overbyteicszlibhigh::zlibProblemString' referenced from C:\USR\SRC\3RDPARTYVCL\ICS V9.5\LIB\DEBUG\WIN64\37.0\ICS.FMX.OVERBYTEICSFTPCLI.O
[ilink64 Error] Error: Unresolved external 'Overbyteicstypes::in6addr_any' referenced from C:\USR\SRC\3RDPARTYVCL\ICS V9.5\LIB\DEBUG\WIN64\37.0\ICS.FMX.OVERBYTEICSSOCKETUTILS.O

[Angus Robertson 683]

I was getting warnings building C++ IcsCommonCBNewRun that units were found in IcsCommonNewRun which is the Delphi package, and is not referred anywhere in C++, but they went away when I specified the Windows SDK, which I don't recall every doing before in RAD Studio.  

 

If you are not using Win64, just ignore the packages.  I rarely test with Win32 now, all my servers and some GUIs are Win64, a couple use old components and need Win32. 

 

Are you using the old or new packages?  Are changes still needed per your root message. 

 

Angus

 

[w0wbagger 6]

Until I can get a Win64 version of ICS working, I can't migrate our app from 32-bit, so I'd first like to get Win64 compiling with the classic toolchain so I can at least migrate to that, and *then* work on the Modern Toolchain migration for C++. Just need to figure out what's keeping these 4 things from being resolved. My hypothesis is either name-mangling or conditional compilation, but that's just speculation at this point. 

[w0wbagger 6]

Okay, I think I've found the issue:
 

In v9.4, several new global variables (e.g. IcsSystemCodePage, GWSockCritSect, zlibProblemString, in6addr_any) were marked with {$EXTERNALSYM} so they would be visible to C++ Builder. That works fine in 32-bit builds because Delphi always emits storage for uninitialized globals. But in 64-bit builds, the compiler/linker optimizes away unreferenced globals that have no explicit initializer. The C++ headers still declare them (because of {$EXTERNALSYM}), but there is no corresponding symbol in the Win64 object file, which causes unresolved externals when linking from C++ Builder.

The fix is simple and Delphi-safe: add a trivial initializer under WIN64 (e.g. = 0, = (), or = '') to force the compiler to emit the variable. This doesn’t change behavior for Delphi users, since global vars are already default-initialized; it just makes the initialization explicit so the symbols are guaranteed to exist in Win64 builds. With these small changes, both Delphi and C++ Builder projects build cleanly, and no C++-only shims are needed.

 

The four problematic globals were spread across three different ICS units. 

1. OverbyteIcsCharsetUtils.pas

   • Variable: IcsSystemCodePage : LongWord;

2. OverbyteIcsTypes.pas

   • Variable: GWSockCritSect : TRTLCriticalSection;

   • Variable: in6addr_any : TIn6Addr;

3. OverbyteIcsZlibHigh.pas

   • Variable: zlibProblemString : AnsiString;

Here's the code I changed:
var
{$IFDEF WIN64}
  IcsSystemCodePage: LongWord = 0;
{$ELSE}
  IcsSystemCodePage: LongWord;
{$ENDIF}
  {$EXTERNALSYM IcsSystemCodePage}             { V9.4 }

var
{$IFDEF WIN64}
  GWSockCritSect: TRTLCriticalSection = ();
{$ELSE}
  GWSockCritSect: TRTLCriticalSection;
{$ENDIF}
  {$EXTERNALSYM GWSockCritSect}                { V9.4 }

var
{$IFDEF WIN64}
  in6addr_any: TIn6Addr = ();
{$ELSE}
  in6addr_any: TIn6Addr;
{$ENDIF}
  {$EXTERNALSYM in6addr_any}

and 

var
{$IFDEF WIN64}
  zlibProblemString: AnsiString = '';
{$ELSE}
  zlibProblemString: AnsiString;
{$ENDIF}
  {$EXTERNALSYM zlibProblemString}       { V9.4 }

All this proves is that I can now compile all the code in the classic Win64 toolchain. Doesn't necessarily mean it works!


[Angus Robertson 683]

Excellent debugging, I'll make those changes and try Win64 again. 

 

On reflection, I see a pattern with the 'units were found in required package' error.  It seems C++ can not build packages if any similar packages are installed in the IDE.  I guess most other developers build their C++ packages using command line tools, not in the IDE, so don't see the problem.  

 

I saw the error with the Delphi package because it had been installed earlier and removed, but was still in memory until an IDE restart.  C++ Win32 then works, but not Win64 if Win32 has been installed into the IDE.  With Delphi, you don't get a Win64 error if Win32 is installed.

 

 Angus

 

[Angus Robertson 683]

I've updated SVN with the changes needed so that ICS can be built with C++ for Win32 and Win64, will be zipped overnight. 

 

One package builds OK for Win64x, but the other two packages dependent upon it fail with undefined symbol errors, which I believe is a long term Win64x linker problem, no quick fix.  

 

I'll look at a second V9.,5 refresh in a few days, once I've finished documentation that missed this release.  OpenSSL has security fixes next week as well.  

 

Angus

[w0wbagger 6]

Yeah, so long as you don't *install* the components before compiling everything with build groups, you won't have any issues, it seems. Once all the targets are built, then you can install the 32-bit components to the toolbar (I'm still only using the 32-bit IDE). As soon as I have my app compiling in Win64 classic, I'll turn to Win64x to see if there's anything i can work out there with (first) the 32-bit IDE and then the 64-bit IDE.

Will send over my updated .cbproj files once I have everything working. I think they're already better than the current ones, but they only do the Win32 and Win64 builds.

[w0wbagger 6]

Having an issue getting my app to compile in 64-bit. Specifically it's looking to link with crypt32.lib and cryptui.lib (from ICS), but 64-bit versions of these libs don't ship with RAD Studio. Do I have to import them from the windows SDK? Modern toolchain should be able to use the COFF format of the windows sdk, but I'm trying to get this to to work with classic win64 first. Do I have to use Dynamic RTL? Up untilnow, we've always statically linked all our libraries just so we could ship a (mostly) single .exe.

NVM. I found them (also, just learned that lib files are .a files in 64-bit classic)

 

Edited yesterday at 09:05 PM by w0wbagger
Spelling, figured it out.

[Angus Robertson 683]

Provided you undefine MSCRYPT, which should happen automatically in defs.inc for BCB, ICS will ignore a lot of Windows crypto stuff that caused C++ problems in the past.

 

It means components to access the Windows Certificate Store are not available for C++.  Would be nice to fix those errors at some point.  

 

Angus

 

[w0wbagger 6]

Okay, I'm deep in the weeds here, and recognize that trying to solve this for the classic Win64 toolchain is probably stupid, but it may help us in getting the Win64x toolchain working. I'm working with an LLM because I don't understand Delphi well enough, but even though I managed to get the Win64 build of ICS v9.5 working (i.e. compiling), I was unable to link it with my 64-bit build of my app, because it kept looking for a 64-bit version of Crypt32.lib (it should be looking for crypt32.a, which *does* ship with RAD Studio.
After a lot of back and forth, this is what the LLM and I came up with:

• Delphi assumes direct DLL imports.

• C++Builder assumes it must have import libraries (.lib/.a).

• The Delphi compiler bakes link directives for .lib into the object files, even for Win64 where .a is required.

• Result: every C++ user who pulls in such units gets ilink64 errors unless they patch around it.

Does this seem plausible?
There are two solutions I can see:
1) add a shim to every C++ project that's included somewhere:
#if defined(_WIN64)
  #pragma link "crypt32.a"
  #pragma link "cryptui.a"
#else
  #pragma link "crypt32.lib"
  #pragma link "cryptui.lib"
#endif

or, try changing it in the source directly, i.e.:
{$IFDEF WIN64}
  {$HPPEMIT END '#pragma link "crypt32.a"'}
  {$HPPEMIT END '#pragma link "cryptui.a"'}
{$ELSE}
  {$HPPEMIT END '#pragma link "crypt32.lib"'}
  {$HPPEMIT END '#pragma link "cryptui.lib"'}
{$ENDIF}

However, this can *not* be hidden within an {$IFDEF CPLUSPLUS}, because CPLUSPLUS is not (I think?) defined when generating the .hpp files.
Can anyone with better understanding of Delphi than I confirm that this is at least plausible? 
The shim version would require every C++ programmer to add this to their project somewhere. The second solution would be Delphi neutral, but might solve the issue with trying to pull in the wrong library. 

 

[Angus Robertson 683]

I'm confused here, why does your application need Crypt32.lib?  What APIs is it using. 

 

As I said earlier, ICS does not use it and the Win64 packages now build OK on C++.  

 

Angus

 

[w0wbagger 6]

My app *doesn't* need it, but it's being attempted to be pulled in by Ilink64 when I compile my app, even though *I'm* not using it anywhere, nor referring to it in my code anywhere. It's not referred to in my cbproj file, and yet It's being attempted to be linked when linking to the Win64 ICS library, and ilink64 says it can't find it (which is correct, because Crypt32.lib doesn't exist for Win64 (it's called Crypt32.a). I think it's an artifact of the different way that import libraries are handled in C++ vs. Delphi (see above).  I see tons of references to crypt32.lib in the .o files of the Win64 ICS Build. 

I was stoked that we got ICS Win64 compiling and linking, but it's only "working" if I can use it with my app, and when I tested this, this was the ilink64 error I got. (couldn't find crypt32.lib) 

[Angus Robertson 683]

And yet the Delphi Win64 compiler manages to open crypt32.lib using the usual Windows magic where Win64 DLL are in the System32 directory...

 

Perhaps there is a crypt32.lib reference I've missed or a HPPEMIT that ignores other defines.  I'll have a look tomorrow.  

 

Angus