TColor breaks memory layout

[dormky 2]

TBloodyTColor1 = record
  aVar: smallint;
  aColor: TColor;
 end;
 
 TBloodyTColor2 = record
  aVar: smallint;
  aColor: TColor;
  aVar2: smallint;
 end;
 
 TBloodyTColor3 = record
  aVar: smallint;
  aVar2: smallint;
  aVar3: smallint;
 end;

The results of "sizeof" for these 3 records are :

TBloodyTColor1 : 8

TBloodyTColor2 : 12

TBloodyTColor3 : 6

 

The size of a TColor is 4 bytes, a smallint is 2 bytes. TBloodyTColor1 & 3 should take the same amount of memory, yet when using TColor 2 bytes are added.

I believed this was for memory alignment purposes, but we can see that's not the case by looking at TBloodyTColor2, where 4 bytes are added (if I had to assume, they're probably 2 bytes before & after the TColor).

 

This is incredibly annoying. It's basically impossible to predict how much space a record is going to take !

Looking at the definition of TColor, TColor = -$7FFFFFFF-1..$7FFFFFFF; indicates that the 7 here might be at fault ?

 

Just looking for some insight on the subject, because this makes static analysis incredibly difficult for really no good reason...

[Roger Cigol 103]

This looks like standard compiler memory alignment to me and is entirely expected.

 

I am pretty sure that the compiler writers would take (gentle) issue with your "really no good reason" statement.

 

I am not sure why it makes static analysis difficult - memory alignment is a common feature of many (most?) compilers. What static analysis tool are you using and where do you find the problem?

[FPiette 383]

Maybe you could consider using "packed record"?

[Brandon Staggs 277]

57 minutes ago, dormky said:

This is incredibly annoying. It's basically impossible to predict how much space a record is going to take !

When you need to know, use SizeOf. If you must know before you compile, use a packed record. If you care about the memory size of these records it is probably due to concerns about size constraints, in which case a packed record will guarantee the smallest possible size based on the field types. If that is not what you're concerned about, it's hard to know why you want to "predict" the record size, and so SizeOf is reliable and will suffice for any needs in code (pointer arithmetic, etc.).

[Brian Evans 105]

You can also re-arrange fields to minimize the gaps added for alignment. A re-arranged second version of the record is also 8 bytes. 

  TBloodyTColor2 = record
   aColor: TColor;
   aVar: smallint;
   aVar2: smallint;
  end;

So why is it 12 bytes in your second version?

1. aVar takes 2 bytes.

2. Padding of 2 bytes to a 4-byte boundary since next member is 4 bytes wide

3. aColor takes 4 bytes

4. aVar2 takes 2 bytes.

5. padding of 2 bytes to make total record size a multiple of 4 (the largest member size).

 

As described in the documentation: Internal Data Formats - Record Types (Delphi) - RAD Studio (embarcadero.com)

Edited December 15, 2023 by Brian Evans

[David Heffernan 2345]

2 hours ago, dormky said:

This is incredibly annoying. It's basically impossible to predict how much space a record is going to take !

It's actually very easy to predict this, once you know the rules. 

 

2 hours ago, dormky said:

Just looking for some insight on the subject, because this makes static analysis incredibly difficult for really no good reason...

There is a good reason. 

 

Instead of ranting, maybe you should assume that the people that designed this knew what they were doing and that you just don't understand it. 

 

The question I have for you, is why are you so interested in size and layout of the records. What is motivating your interest. That might be helpful for us to understand. 

[dormky 2]

On 12/15/2023 at 6:25 PM, Brian Evans said:

You can also re-arrange fields to minimize the gaps added for alignment. A re-arranged second version of the record is also 8 bytes. 

  TBloodyTColor2 = record
   aColor: TColor;
   aVar: smallint;
   aVar2: smallint;
  end;

So why is it 12 bytes in your second version?

1. aVar takes 2 bytes.

2. Padding of 2 bytes to a 4-byte boundary since next member is 4 bytes wide

3. aColor takes 4 bytes

4. aVar2 takes 2 bytes.

5. padding of 2 bytes to make total record size a multiple of 4 (the largest member size).

 

As described in the documentation: Internal Data Formats - Record Types (Delphi) - RAD Studio (embarcadero.com)

Ah, thank you for the explanation and link, this is the info I was looking for !

 

@David Heffernan I'm looking into this because we have config files that are just memory dumps of records. I need to be able to meaningfully compare and manipulate those files. There is A LOT of them and doing so manually would be hopeless.

[David Heffernan 2345]

1 hour ago, dormky said:

I'm looking into this because we have config files that are just memory dumps of records. I need to be able to meaningfully compare and manipulate those files. There is A LOT of them and doing so manually would be hopeless.

This sounds like a pretty unfortunate design choice. But since your compiler knows the layout of those records you can just work with the record types and the compiler takes care of the layout. 

[dormky 2]

7 minutes ago, David Heffernan said:

This sounds like a pretty unfortunate design choice. But since your compiler knows the layout of those records you can just work with the record types and the compiler takes care of the layout. 

Unfortunately, no. Most of said records are just wrapping underlying anonymous arrays and records, and there's no RTTI generated for those (I need to get the actual name of the field so a human can read the comparison).

A field like "array [0..1] of record X, Y: smallint; end;" doesn't have RTTI, and there's a lot of those.

 

And for your entertainment, things are actually worse than this because anytime one of these arrays needs to be grown for a change in the program, we need to keep the old version of the record so we can read the old file, and a new one with the change. And all the fields need to be manually copied over, there's tens of thousands of lines of "current.field := V22.field" :')

So yeah. I don't know who went with this model of configuration storage, but if I get my hands on them they're in trouble, to say the least.

 

Edit : I could take out the anonymous stuff and declare it properly, but that would make an even bigger mess of things, add a lot more versioning like shown above, etc... All to be able to compare the god damned configs. So I want to evade doing that if possible, such a big number of changes would have no chance of me not making a mistake somewhere in my copy-pasting, and of course we have 0 tests.

Edited December 18, 2023 by dormky

[David Heffernan 2345]

Hmm, sucks to be you rn!! 

[DelphiUdIT 176]

May be you must write a "little" interface between disk data and you record data. You can maintain your record data in memory without any changes, and when you read and write the data you can adjust in memory the reading.

If you works with records, you can works writing and advances records, put the variables in private section (renaming them) and writing functions with the old names:

 

current.field := V22.field

TV22 = record
  private
     fV22_field: cardinal;
     function getV22_field: smallint;
     procedure setV22_field(Value: smallint);
  public
    property field: smallint read getV22_field write setV22_field;
end;

function TMyRecord.getV22_field: smallint;
begin
  //You can do here what you want to adjust the operation
  Result := fV22_field;
end;

procedure TMyRecord.setV22_field(Value: smallint);
begin
  //You can do here what you want to adjust the operation
  fV22_field := Value;
end;

Of course you must do this for every field you have declared that needed to be adjust.