Range Check Error - what actually happens

[david_navigator 12]

I just compiled a colleagues code with Range checking turned on and got an immediate error.

 

It's fairly obvious why and an easy fix (aLen had a value of 9 in this circumstance and the bytes array is simply the wrong size), but I was wondering what actually happens when the faulty code is executing with range checking turned OFF ?

What actually happens when the line 

abyte := bytes[B]; 

is executed with a value of 8 for B ?
Is there a potential for memory corruption or does the code internally fail gracefully ?

function GetSetCardinality(const aSet; aLen: Word): Word;
var
  B, i: Integer;
  abyte: Byte;
  bytes: array [0 .. 7] of Byte absolute aSet; 
begin
  result := 0;
  for B := 0 to aLen - 1 do
  begin
    abyte := bytes[B];

...

...

Many thanks

 

Cheers

 

David

 

[Der schöne Günther 316]

As far as I understand the documentation Range checking - RAD Studio (embarcadero.com), enabling range checks adds additional code around every array access. If that is not the case, then a write beyond the arrays end can silently succeed, but corrupt memory.

 

People Wizards who understand assembly code can probably identify the additional code added by the compiler.

Edited December 31, 2020 by Der schöne Günther

[David Heffernan 2345]

48 minutes ago, david_navigator said:

Is there a potential for memory corruption or does the code internally fail gracefully ?

Behaviour is undefined. You might get an AV, or a write might corrupt data. Definitely not graceful failure. 

[FPiette 383]

16 hours ago, david_navigator said:

I was wondering what actually happens when the faulty code is executing with range checking turned OFF ?

What actually happens when the line 

abyte := bytes[B]; 

is executed with a value of 8 for B ?
Is there a potential for memory corruption or does the code internally fail gracefully ?

The code line read a byte from memory. When the array index if out of range, memory outside of the array is read.

 

16 hours ago, david_navigator said:

abyte : Byte;

bytes: array [0 .. 7] of Byte absolute aSet;

As declared, the array "bytes" is stored by the compiler at the same address as the variable aSet which is an argument which is passed by address (untyped const). The net effect is that accessing the array is accessing the bytes of the variable passed as argument directly.

 

In the code you show, the code intentionally read variable bytes. Given the name of the function (GetSetCardinality), that argument is probably a set (set of something).

 

Reading outside of the array bounds will probably have no adverse effect. It depends on what the read values are used for. Accessing the byte right after the last item array will just return an apparently random value which is in memory at that place. Depending where the original variable passed as argument, that memory may be unreadable. Very unlikely unreadable: it would be the last variable in a virtual memory page and the next page not being allocated. We can assume that the alen argument give the size of the passed variable and the code (not shown) accessing the array make sure to not go beyond that length. So there will be no access violation.

 

Edited January 1, 2021 by FPiette
I wrongly stated that the array was stored at same address as aByte. Fixed that and the consequences.

[David Heffernan 2345]

2 hours ago, FPiette said:

As declared, the array "bytes" is stored by the compiler at the same address as the variable abyte which is a local variable which is stored on the stack.

That's wrong. The code reads absolute aSet rather than absolute aByte.

 

2 hours ago, FPiette said:

May be you have not noticed that the array is specifically used to read the stack

Consequently this statement is also incorrect. aSet could be on the stack, or the heap, or a global. 

 

In any case, I took the question a bit more generally than you. Accessing arrays out of bounds can lead to AVs or corruption of other memory in the case of a write operation. Yes, I know that the example here is a read. I was generalising. 

Edited January 1, 2021 by David Heffernan

[david_navigator 12]

Thanks for the replies.

 

The programmer's comment above the code indicates that the function is supposed to return the number of elements in a set. If I'm not mistaken I believe delphi allows up to 256 elements which are represented as 256 bits (rather than encoded in to 8 bytes), which I assume means that 

bytes: array [0 .. 7] of Byte absolute aSet;

should actually be

bytes: array [0 .. 31] of Byte absolute aSet;

The rest of the code simply counts the number of bits set in each abyte which it returns as the result.

 

[David Heffernan 2345]

5 hours ago, David Heffernan said:

I know that the example here is a read. I was generalising. 

Thinking again, the specifics are important here, the use of an untyped parameter that is overload with an absolute variable. This basically renders range checking close to useless, because of the unsafe typecast. 

[dummzeuch 1505]

19 minutes ago, David Heffernan said:

Thinking again, the specifics are important here, the use of an untyped parameter that is overload with an absolute variable. This basically renders range checking close to useless, because of the unsafe typecast. 

Even  worse: It provides a false sense of security and on top of that results in a runtime error that isn't really an error in this case.

 

On the other hand, there is the aLength parameter which supposedly contains the length of the aSet parameter in bytes and there is nothing to prevent it from being larger than Length(bytes) (Assuming that aSet is a set as the name implies it has a maximum size of 32 bytes = 256 elements, each represented by one bit).

 

Using a PByte would probably be better here and some comment would also be in order.

[david_navigator 12]

Quote

Using a PByte would probably be better here 

 I think I might need some help here then.

The colleague that wrote this function is on furlough for at least the next four months due to Covid, so I have to address the problematic code myself and I'm not sure I understand why it's done like this to be able to refactor it correctly.

 

The function is passed a set along with the sizeof(the set)

 

This is the original function (with the array of bytes set at the original wrong value of 8 bytes, rather than I believe the correct value of 32 bytes)

function GetSetCardinality(const aSet; aLen: Word): Word;
{ Sample call would be:

  count := GetSetCardinality(SomeSet, sizeof(SomeSet)); }

var
  B, i: Integer;
  abyte: Byte;
  bytes: array [0 .. 7] of Byte absolute aSet; 
begin
  result := 0;
  for B := 0 to aLen - 1 do
  begin
    abyte := bytes[B];
    for i := 0 to 7 do
      if (abyte and (1 shl i)) <> 0 then
        Inc(result);
  end;
end;

 

[dummzeuch 1505]

26 minutes ago, david_navigator said:

I think I might need some help here then.

This should do the trick:

function GetSetCardinality(const aSet; aLen: Word): Word;
var
  B, i: Integer;
  abyte: PByte;
begin
  Result := 0;
  abyte := @aSet;
  for B := 0 to aLen - 1 do begin
    for i := 0 to 7 do
      if (abyte^ and (1 shl i)) <> 0 then
        Inc(Result);
    Inc(abyte);
  end;
end;

test code:

var
  SomeSet: set of 0..255;
  Count: Word;
begin
  SomeSet := [1, 5, 6, 100, 200];
  Count := GetSetCardinality(SomeSet, SizeOf(SomeSet));

Count should be set to 5, the number if elements in the set. ALen should be 32.

var
  SomeSet: set of 0..10;
  Count: Word;
begin
  SomeSet := [1, 5, 6];
  Count := GetSetCardinality(SomeSet, SizeOf(SomeSet));

Count should be set to 3, ALen should be 2 (one byte has 8 bits, the set can have 11 elements (0..11) which is > 8 and <= 16 -> 2 bytes.

Edited January 1, 2021 by dummzeuch

[david_navigator 12]

Quote

This should do the trick:

Brilliant. Thank you very much. Saved me hours of googling & head scratching 🙂

Happy New Year.

[David Heffernan 2345]

There is another issue here, which is that you check bits that are unused, it the base type has number of members not divisible by 8. Probably you get away with it because it's probably hard to get 1 in any of the unused bits in a set. 

[Guest]

What type of set ?

How many many different of them ?

 

I am asking because may be the following is shorter and more concrete apprach

  Count: Word;
  i:Integer;
begin
  Count:=0;
  for I in SomeSet do
    Inc(Count);

With known and defined types like integers it will work just right

 

[David Heffernan 2345]

34 minutes ago, Kas Ob. said:

What type of set ?

How many many different of them ?

 

I am asking because may be the following is shorter and more concrete apprach

  Count: Word;
  i:Integer;
begin
  Count:=0;
  for I in SomeSet do
    Inc(Count);

With known and defined types like integers it will work just right

 

The issue is that you need to write this code for every single set type that you use. 

[Guest]

8 minutes ago, David Heffernan said:

The issue is that you need to write this code for every single set type that you use. 

For that i asked how many, i think repeating a word show that i had a glitch in the head 🙂

Also if the numbers of these types is small a helper per type will be clean.

[David Heffernan 2345]

19 minutes ago, Kas Ob. said:

For that i asked how many, i think repeating a word show that i had a glitch in the head 🙂

Also if the numbers of these types is small a helper per type will be clean.

Unfortunately Emba never implemented proper generics support for sets which would make this trivial and type safe. 

[david_navigator 12]

Quote

What type of set ?

How many many different of them ?

Quickly Grepping the source, it looks like there are 8 independent sets that use the GetSetCardinality function.

[Guest]

44 minutes ago, David Heffernan said:

Unfortunately Emba never implemented proper generics support for sets which would make this trivial and type safe. 

It is so sad, that the language itself on hold, one can call it in stale state, while tens of features can be implemented or added without breaking backward compatibility to give Delphi and Pascal power, the power that C++ dream of with such readability.

 

45 minutes ago, david_navigator said:

Quickly Grepping the source, it looks like there are 8 independent sets that use the GetSetCardinality function.

Although, i don't not quite understand this 8, is it 8 different types or used 8 times, with different named vars declared sets with the same type like set of integers...

This does work and it is elegant

type
  TIntegerSet = set of Byte;

function GetSetCardinality2(const aSet: TIntegerSet): Word;
var
  i: Integer;
begin
  Result := 0;
  for i in aSet do
    Inc(Result);
end;

var
  SomeSet: set of 0..255;
  SomeSmallSet: set of 0..10;
  Count: Word;
begin
  SomeSet := [1, 5, 6, 100, 200];
  SomeSmallSet := [1, 5, 6];

  Count:=GetSetCardinality2(SomeSmallSet);  // return 3
  Count:=GetSetCardinality2(SomeSet);       // return 5

 

[Guest]

This also will work with typecasting

var
  Digits: set of '0'..'9';
  Count: Word;
begin

  Digits := ['6', '8'];
  Count := GetSetCardinality2(TIntegerSet(Digits));  // return 2

 

[Guest]

Never used set of char, trying it shows this

 

The compiler doesn't complain about that, for me it looks like a bug, the question is : does this behaviour exist in newer versions ?

[Mike Torrettinni 198]

43 minutes ago, Kas Ob. said:

Never used set of char

@Kas Ob. it's not that much different than other Sets, like Set of integer or Set of byte, same construct.

 

If I may recommend read these resources, I'm sure you will find some value in them:

http://docwiki.embarcadero.com/RADStudio/Sydney/en/Structured_Types_(Delphi)

 

Also perhaps of interest:

 

Marco Cantù's Essential Pascal Chapter 4 User-Defined Data Types: https://www.marcocantu.com/epascal/English/ch04udt.htm

CharInSet is much slower than IN, should I fix W1050 warning hint?: https://stackoverflow.com/questions/35942270/charinset-is-much-slower-than-in-should-i-fix-w1050-warning-hint

 

[Attila Kovacs 629]

3 hours ago, David Heffernan said:

Unfortunately Emba never implemented proper generics support for sets which would make this trivial and type safe. 

No, but obviously you already made a nice helper https://stackoverflow.com/questions/34442102/how-can-i-get-the-number-of-elements-of-any-variable-of-type-set

AD? 😉

[Guest]

9 hours ago, Mike Torrettinni said:

should I fix W1050 warning hint?:

I think i managed to mess the default settings on my almost all IDE's !

 

Thank you to remind me of that.

[David Heffernan 2345]

12 hours ago, Mike Torrettinni said:

it's not that much different than other Sets, like Set of integer or Set of byte, same construct.

Not so. In fact set of Char is actually implemented as set of AnsiChar. It has very special treatment introduced to "help" porting from ANSI Delphi to Unicode Delphi. 

[david_navigator 12]

Quote

Although, i don't not quite understand this 8, is it 8 different types or used 8 times, with different named vars declared sets with the same type like set of integers...

 

8 different types.