Record Alignement and Delphi 10.4.1

[Arnaud Bouchez 407]

From a Github issue description for our SynPDF Open Source project: Generating a PDF via VLCCanvas and TPdfDocumentGDI causes access violation when compiled with Delphi 10.4.1 with record field alignment compiler option set to "byte" or "off". When this option is set to either of "word", "double word" or "quad word", the PDF gets created without errors. The same exact code works fine when compiled with Delphi 10.4 (patch 3), regardless of the field alignment flag.

 

We added {$A+} and it seemed to fix the problem.
https://blog.synopse.info/?post/2020/09/09/Record-Alignement-and-Delphi-10.4.1

 

Sadly, I don't have access to Delphi 10.4.1 since I don't have any commercial licence, and I am waiting for the Community Edition - which is still 10.3 IIRC. So I couldn't debug the root cause and fill a JIRA ticket to EMB.
Perhaps some people from Delphi-Praxis may have encountered this issue, and found the root cause...

 

Maybe it is was a real fix introduced in 10.4.1, and the previous behavior was incorrect: perhaps an explicit {$A+} is required when working with records... but at least, it breaks existing code, so spreading the info may help...

Edited September 9, 2020 by Arnaud Bouchez

[FPiette 382]

Using the test project attache to Git report, the AV occurs in the irst line of function TPdfWrite.Add. The value of Self is inaccessible.

Call stack is

SynPdf.TPdfWrite.Add(0)
SynPdf.TPdfObjectStream.InternalWriteTo($ADF2DD8)
SynPdf.TPdfObject.WriteValueTo($ADF2DD8)
SynPdf.TPdfDocument.SaveToStreamDirectEnd
MainForm.TFMainForm.testButtonClick(???)

In the test project, it is the first call of TPdfObjectStream.InternalWriteTo, in the first iteration of the for-loop:

procedure TPdfObjectStream.InternalWriteTo(W: TPdfWrite);
var i: integer;
begin
  Attributes.AddItem('N',fObjectCount);
  for i := 0 to fObjectCount-1 do
    with fObject[i] do
      Writer.Add(Number).Add(' ').Add(Position).Add(' ');
  Attributes.AddItem('First',Writer.Position);
  Writer.Add(fAddingStream.ToPDFString);
  inherited;
end;

When compiled with record alignment set to Byte, the FObject array has a random length, in the 3000+ range.  All value are zero.

When compiled with record alignment set to Word, the FObject array contains 6 elements which have good looking values.

 

 

[FPiette 382]

constructor TPdfObjectStream.Create(aDoc: TPdfDocument);
begin
  inherited Create(aDoc,false);
  Attributes.AddItem('Type','ObjStm');
  fAddingStream := TPdfWrite.Create(ADoc,THeapMemoryStream.Create);
end;

fObjectCount gets his random value from the inherited Create call.

And in the inherite Create, fObjectCount gets his random value from the call to TPdfWrite.Create:

constructor TPdfStream.Create(ADoc: TPdfDocument; DontAddToFXref: boolean=false);
var FXref: TPdfXRef;
begin
  inherited Create;
  if DontAddToFXref then
    FXRef := nil else begin
    FXRef := ADoc.FXref;
    FXRef.AddObject(self);
  end;
  FAttributes := TPdfDictionary.Create(FXref);
  FAttributes.AddItem('Length', TPdfNumber.Create(0));
  if ADoc.CompressionMethod=cmFlateDecode then
    FFilter := 'FlateDecode';
  FWriter := TPdfWrite.Create(ADoc,THeapMemoryStream.Create);
end;

I single stepped TPdfWrite.Create to find out where fObjectCount in the caller class is corrupted. It is not corrupted before reaching the end! It is corrupted after return. I then single stepped in the assembly language. Evrything is OK up to the return point in TPdfStream.Create. The corrupting line is the assignation to FWriter

SynPdf.pas.4146: FWriter := TPdfWrite.Create(ADoc,THeapMemoryStream.Create);
 

006D8B57 B201             mov dl,$01
006D8B59 A188B66900       mov eax,[$0069b688]
006D8B5E E8AD06D3FF       call TObject.Create
006D8B63 50               push eax
006D8B64 8BCF             mov ecx,edi
006D8B66 B201             mov dl,$01
006D8B68 A180D66C00       mov eax,[$006cd680]
006D8B6D E83A1B0000       call TPdfWrite.Create
006D8B72 89431D           mov [ebx+$1d],eax          <=== This instruction corrupt fObjectCount

The address in [EBX+$1D] is the same as the address given by the debugger for FWriter ($AC7CFED) BUT the address of fObjectCount if only one byte away ($AC7CFEF). It should be 4 bytes since FWriter is  apointer and I compiled to code in 32bits.

 

IMO it is the compiler which generate bad code.

 

Edited September 9, 2020 by FPiette
Added more info.

[Arnaud Bouchez 407]

Thanks for the detailed feedback... from the asm sounds like a compiler issue.

 

It may be worth a ticket, since it may affect not only our code, but a lot of it!

Edited September 9, 2020 by Arnaud Bouchez

[FPiette 382]

Before writing a report at Embarcadero Quality Portal, I suggest that you debug the code by yourself. You probably know it much than me. Since you don't have D10.4.1, I can arrange a TeamViewer session for you on my computer so that you debug it. Let me know if you are interested.

[Stefan Glienke 2002]

The offset in the code is correct but the binary layout of the classes are not - here is the dump of the memory layout from the defect:

 

--- TPdfObject ---
offset:   4 size:  1 FObjectType: TPdfObjectType
offset:   5 size:  4 FObjectNumber: Integer
offset:   9 size:  4 FGenerationNumber: Integer
offset:  13 size:  1 FSaveAtTheEnd: Boolean
--- TPdfStream ---
offset:  14 size:  2 ---PADDING---
offset:  16 size:  4 FAttributes: TPdfDictionary
offset:  20 size:  4 FSecondaryAttributes: TPdfDictionary
offset:  24 size:  1 FDoNotEncrypt: Boolean
offset:  25 size:  4 FFilter: AnsiString
offset:  29 size:  4 FWriter: TPdfWrite
--- TPdfObjectStream ---
offset:  31 size:  4 fObjectCount: Integer
offset:  35 size:  4 fAddingStream: TPdfWrite
offset:  39 size:  4 fObject: :TPdfObjectStream.:2
offset:  43 size:  1 ---PADDING---
size:    48

So the [ebx+$1d] is correct ($1d = 29) - but the starting offset in TPdfObjectStream is wrong.

 

With align Word it looks like this:

 

--- TPdfObject ---
offset:   4 size:  1 FObjectType: TPdfObjectType
offset:   5 size:  1 ---PADDING---
offset:   6 size:  4 FObjectNumber: Integer
offset:  10 size:  4 FGenerationNumber: Integer
offset:  14 size:  1 FSaveAtTheEnd: Boolean
--- TPdfStream ---
offset:  15 size:  1 ---PADDING---
offset:  16 size:  4 FAttributes: TPdfDictionary
offset:  20 size:  4 FSecondaryAttributes: TPdfDictionary
offset:  24 size:  1 FDoNotEncrypt: Boolean
offset:  25 size:  1 ---PADDING---
offset:  26 size:  4 FFilter: AnsiString
offset:  30 size:  4 FWriter: TPdfWrite
--- TPdfObjectStream ---
offset:  34 size:  4 fObjectCount: Integer
offset:  38 size:  4 fAddingStream: TPdfWrite
offset:  42 size:  4 fObject: :TPdfObjectStream.:2
offset:  46 size:  2 ---PADDING---
size:    52

 

Edited September 9, 2020 by Stefan Glienke

[FPiette 382]

5 minutes ago, Stefan Glienke said:

The offset in the code is correct but the binary layout of the classes are not

I understand that you confirm my analysis: it is a compiler bug.

 

[Stefan Glienke 2002]

Reported: https://quality.embarcadero.com/browse/RSP-30890

[Der schöne Günther 316]

That sounds quite nasty. I think I'll stick with Delphi 10.4.0...