C++ / windows.h and data alignment

[alank2 5]

This isn't an Embarcadero question per se, but a generic #include <windows.h> and data alignment question.

 

Do the structures defined in windows.h require a specific alignment in the operating system?

 

I have a legacy program that uses 1-byte alignment and the result is a VOLUME_DISK_EXTENTS structure of 24 bytes.  If I call DeviceIoControl(Volume, IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS, ...) with this 24 bytes for the size, it fails, but when packing by 8 bytes resulting in a structure size of 32, it works fine.

 

I solved this by wrapping the windows.h header with pshpack8/poppack, but if the structures require 8 byte alignment, why didn't they do this inside windows.h?

 

#include <pshpack8.h>
  #include <windows.h>
  #include <shlobj.h>
  #include <poppack.h>

 

edit : I found this:

https://stackoverflow.com/questions/29518184/64-bit-windows-api-struct-alignment-caused-access-denied-error-on-named-pipe

 

Does the VCL expect a specific alignment (#include <vcl.h>) ?

Edited October 17, 2023 by alank2

[Remy Lebeau 1394]

47 minutes ago, alank2 said:

Do the structures defined in windows.h require a specific alignment in the operating system?

Yes - 8 bytes

47 minutes ago, alank2 said:

I solved this by wrapping the windows.h header with pshpack8/poppack, but if the structures require 8 byte alignment, why didn't they do this inside windows.h?

Because the Windows SDK is old and predates those headers? I don't know.

 

Not sure if the original SDK headers do this, but In Embarcadero's copy of windows.h and other SDK headers, there are actually #pragma statements to setup 8-byte alignment, eg:

#pragma option push -b -a8 -pc -A- /*P_O_Push*/
...
#pragma option pop /*P_O_Pop*/

The -a8 parameter is the alignment.

47 minutes ago, alank2 said:

Does the VCL expect a specific alignment (#include <vcl.h>) ?

Yes, and most VCL headers have #pragma statements for that purpose, eg:

#pragma pack(push,8)
...
#pragma pack(pop)

 

Edited October 17, 2023 by Remy Lebeau

[DelphiUdIT 176]

Windows API calls are generally 8-byte aligned on a 64-bit system.

Compared to APIs, VCLs are simply wrappers and do not modify the structures.

The result (outgoing) and the data (incoming) are and must conform to what is documented by Microsoft, the VCLs do not test, change or modify anything with respect to this, much less the required alignment.
Where necessary, the includes files define the correct alignments if a different alignment is necessary (this in both C++ and Delphi) as required by the API.
An example for everyone is the very old (almost archaic) definition of the _ImageDosHeader which is aligned to 2 bytes.
Bye

Edited October 17, 2023 by DelphiUdIT

[alank2 5]

2 hours ago, Remy Lebeau said:

Not sure if the original SDK headers do this, but In Embarcadero's copy of windows.h and other SDK headers, there are actually #pragma statements to setup 8-byte alignment, eg: 

Oddly, they don't.  If a project's alignment is changed from quad word (8) to something else, there will be issues including windows.h because its structures may not be 8 aligned as the OS expects them to be.

 

I'm glad Embarcadero's copy does force 8 byte alignment, I would have thought that Microsoft's windows.h would have done the same thing, but it doesn't.  I only ran into this with some legacy code that needs to be compiled with 1 byte alignment.  I'll just be careful to wrap windows.h in a pack 8.

 

Thanks everyone!

[David Heffernan 2345]

Obviously Microsoft's header files handle struct alignment correctly

[Remy Lebeau 1394]

6 hours ago, DelphiUdIT said:

Windows API calls are generally 8-byte aligned on a 64-bit system.

On 32bit systems, as well.

 

Using the Windows Headers

Quote

Projects should be compiled to use the default structure packing, which is currently 8 bytes because the largest integral type is 8 bytes. Doing so ensures that all structure types within the header files are compiled into the application with the same alignment the Windows API expects. It also ensures that structures with 8-byte values are properly aligned and will not cause alignment faults on processors that enforce data alignment.

For more information, see /Zp (struct member alignment) or pack.

 

What structure packing do the Windows SDK header files expect?

Quote

/Zp8

 

Edited October 17, 2023 by Remy Lebeau

[alank2 5]

14 hours ago, David Heffernan said:

Obviously Microsoft's header files handle struct alignment correctly

That is the whole problem I am describing here.  They do not.  If you change the data alignment from its default of 8, then the structures defined in windows.h will not be correct for what the OS expects.

 

Remy - I usually use #include <pshpack8.h>, but I thoght I'd try your #pragma pack(push,8) and #pragma pack(pop) like this:

 

#pragma pack(push,8)
#include <windows.h>
#pragma pack(pop)

 

The result is a warning from cppb10.3:

[bcc32 Warning] program.cpp(34): W8083 Pragma pack pop with no matching pack push

 

If I remark out the #include, then no warning.  Does this mean that MS's header pops more than it pushes leaving my final pop to not have its matching push?

 

[Remy Lebeau 1394]

3 hours ago, alank2 said:

Remy - I usually use #include <pshpack8.h>, but I thoght I'd try your #pragma pack(push,8) and #pragma pack(pop) like this:

 

#pragma pack(push,8)
#include <windows.h>
#pragma pack(pop)

I would just stick with using the <pshpack#.h>/<poppack.h> headers, since they support multiple compilers, and will use #pragma pack(push) if the compiler supports it.

3 hours ago, alank2 said:

The result is a warning from cppb10.3:

[bcc32 Warning] program.cpp(34): W8083 Pragma pack pop with no matching pack push

I have never seen that warning before when using #pragma pack directly.  But then, I don't ever wrap the standard Win32 headers with it, either.

3 hours ago, alank2 said:

Does this mean that MS's header pops more than it pushes leaving my final pop to not have its matching push?

I doubt it, but anything is possible, I guess.  You will have to review the headers for yourself.

[DelphiUdIT 176]

3 hours ago, alank2 said:

#pragma pack(push,8)

#include <windows.h>
#pragma pack(pop)

The result is a warning from cppb10.3:

[bcc32 Warning] program.cpp(34): W8083 Pragma pack pop with no matching pack push

If I remark out the #include, then no warning.  Does this mean that MS's header pops more than it pushes leaving my final pop to not have its matching push?

I added the "#pragma pack" in one of my projects,  but don't have this kind of Warning:

[David Heffernan 2345]

3 hours ago, alank2 said:

That is the whole problem I am describing here.  They do not. 

Sounds a little implausible 

[alank2 5]

1 hour ago, DelphiUdIT said:

I added the "#pragma pack" in one of my projects,  but don't have this kind of Warning: 

Try moving it before the vcl.h include and see if that causes it.

[alank2 5]

48 minutes ago, David Heffernan said:

Sounds a little implausible 

I thought so too, but see the Remy's link above for:

 

What structure packing do the Windows SDK header files expect?

 

Expect as in it should be set to 8 byte packing BEFORE including the windows.h header.

 

Unlike the vcl.h and presumably fmx.h headers which Remy mentioned above will changing the packing to 8 if it is something different.

[David Heffernan 2345]

That blows my mind!!

[alank2 5]

52 minutes ago, David Heffernan said:

That blows my mind!!

Mine too; I couldn't believe it until I saw the sizeof() some structures were different based on the data packing setting.

[DelphiUdIT 176]

3 hours ago, alank2 said:

Try moving it before the vcl.h include and see if that causes it.

I commented "vcl.h" (i don't use it) and put "windows.h" at the top. Same result, no warnings (Rad Studio 11.3).

 

 

[DelphiUdIT 176]

This is "mine" windows.h first lines ... and there is an alignment forced at 8 bytes.

 

[Remy Lebeau 1394]

2 hours ago, DelphiUdIT said:

This is "mine" windows.h first lines ... and there is an alignment forced at 8 bytes.

That header is modified by Embarcadero, as evident by the check for __CODEGEARC__.  Microsoft doesn't care about checking for CodeGear/Embarcadero compilers.

 

The question is, what does Microsoft's native SDK version of the header look like?

[DelphiUdIT 176]

6 hours ago, Remy Lebeau said:

The question is, what does Microsoft's native SDK version of the header look like? 

My fault, sorry ... since the first post talked about "vcl.h" and in others the references to the compilation of "bcc32" were reported, I considered the discussion linked to the Embarcadero C++ environment.

What is the reason for using Microsoft SDK headers instead of Embarcadero ones?

 

I don't use C++ very often and when I need to I use Rad Studio on Windows or gcc on Linux.
And I have never had any problems using the APIs (with the original headers of the respective environments).

If I have to use "packed" structures, I septate and restore alignment. In Delphi, where there is no possibility to "push/pop" the settings, I have a small structure that I test to find out the original alignment and then restore it at the end of my needs.

 

The example I gave in the previous post on the "imagedosheader" was the first alignment mistake (happened many decades ago) and at the time it drove me crazy...

[Remy Lebeau 1394]

18 hours ago, DelphiUdIT said:

What is the reason for using Microsoft SDK headers instead of Embarcadero ones?

Didn't say there is one. From the get go, the discussion has been about compiling with <windows.h> in general, and how Microsoft doesn't force the alignment but Embarcadero does.

18 hours ago, DelphiUdIT said:

In Delphi, ... there is no possibility to "push/pop" the settings

Yes, a shame (FreePascal does). Even Marco agreed it would be a useful feature to add, but they still haven't added it yet. 

[DelphiUdIT 176]

10 hours ago, Remy Lebeau said:

Didn't say there is one. From the get go, the discussion has been about compiling with <windows.h> in general, and how Microsoft doesn't force the alignment but Embarcadero does. 

@alank2 said that, he compiles the windows sdk headers (if that is what he means) with BCC32 !!! and I asked why ... but never mind may be I understood wrong.

 

On 10/18/2023 at 2:49 PM, alank2 said:

The result is a warning from cppb10.3:

[bcc32 Warning] program.cpp(34): W8083 Pragma pack pop with no matching pack push

  

If I remark out the #include, then no warning.  Does this mean that MS's header pops more than it pushes leaving my final pop to not have its matching push?

 

Edited October 20, 2023 by DelphiUdIT

[alank2 5]

Also, just to add to this, the latest headers in VS2019 do warn of this problem - if the structure packing is changed to something other than 8 and you include <windows.h>, it will give this error.

 

Severity    Code    Description    Project    File    Line    Suppression State
Error    C2338    Windows headers require the default packing option. Changing this can lead to memory corruption.
This diagnostic can be disabled by building with WINDOWS_IGNORE_PACKING_MISMATCH defined.

 

Which is easily fixed by wrapping the include like this:

#include <pshpack8.h>
#include <windows.h>
#include <poppack.h>

 

[Remy Lebeau 1394]

10 minutes ago, alank2 said:

Also, just to add to this, the latest headers in VS2019 do warn of this problem - if the structure packing is changed to something other than 8 and you include <windows.h>, it will give this error.

Still makes one wonder why Microsoft doesn't just force the alignment in its own headers where needed.

[PeterBelow 238]

On 11/7/2023 at 10:39 PM, Remy Lebeau said:

Still makes one wonder why Microsoft doesn't just force the alignment in its own headers where needed.

In  Delphi an alignment directive only affects the unit it is in, from the location of the directive to the end of the unit. If I understand the workings of C/C++ compilers re headers correctly (which I may not) a directive in a header also affects all other headers included after the one with the directive, so enforcing alignment in a frequently used header may have unintended effects in customer projects.

[Remy Lebeau 1394]

1 hour ago, PeterBelow said:

If I understand the workings of C/C++ compilers re headers correctly (which I may not) a directive in a header also affects all other headers included after the one with the directive, so enforcing alignment in a frequently used header may have unintended effects in customer projects.

Yes, your understanding is correct, but that wouldn't actually be a problem if each header handled its own alignment directives locally and made sure any alignment changes didn't bleed out into the caller's code.