FreeAndNil 10.4 vs 10.3.1 and Pointers

[Sherlock 663]

So I've been digging into some component codes just for the heck of it, and stumbled across a whole lot of code like the following snippet:

procedure Foo;
var
  Bar: PArrayOfByte;
begin
  // Do some more or less elaborate thing with Bar
  // and in the end
  FreeAndNil(Bar);
end;

Now, first off, that is not how I had understood FreeAndNil. I always thought it to be usable for object instances. Of course one could argue, that whatever is behind a pointer could be an object itself, but I mean some thingy that was created via a TObject.Create. And sure enough, Delphi 10.4 will no longer compile stunts like these. It will give an E2010 and bluntly state that TObject and PArrayOfByte are not compatible. And rightly so.

 

So...What to do? How to fix? Given that (in the above crude example) Bar must have been allocated memory via System.GetMem or System.New, System.FreeMem or Sysem.Dispose will do the trick, right?

Why was this not done in the first place, though? Why mix paradigms at the risk of creating some unholy entity mucking up the entire system?

Perhaps looking into FreeAndNil will enlighten us?

This is 10.3.1s take on that method:

procedure FreeAndNil(var Obj);
{$IF not Defined(AUTOREFCOUNT)}
var
  Temp: TObject;
begin
  Temp := TObject(Obj);
  Pointer(Obj) := nil;
  Temp.Free;
end;
{$ELSE}
begin
  TObject(Obj) := nil;
end;
{$ENDIF}

And here is what 10.4 considers to be correct:

procedure FreeAndNil(const [ref] Obj: TObject);
{$IF not Defined(AUTOREFCOUNT)}
var
  Temp: TObject;
begin
  Temp := Obj;
  TObject(Pointer(@Obj)^) := nil;
  Temp.Free;
end;
{$ELSE}
begin
  Obj := nil;
end;
{$ENDIF}

Subtle, innit? Now FreeAndNil requires the thing it's supposed to free and nil to be of TObject, which is cool. But it also considers it to be const...why is that? I learned const renders the parameter unmodifiable, see here: http://docwiki.embarcadero.com/RADStudio/Rio/en/Parameters_(Delphi)#Constant_Parameters. I also took a wrong turn somewhere and ended up in this C++ explantion of const http://docwiki.embarcadero.com/RADStudio/Rio/en/Const in detail this sentence: "A const pointer cannot be modified, though the object to which it points can be changed." OK, so we can change the object, we are freeing it after all. But then we also change where the pointer points to, by niling it. Now that should violate the const boundary, but it does not. Is it because of the "fancy shmancy" use of @s and ^s here?

TObject(Pointer(@Obj)^) := nil;

Why? Why go to the lenghts of casting, adressing and dereferencing?

What are your thoughts and comments on this?

 

I think this almost cured my urge to look into the sources of other people 🤣

[Dalija Prasnikar 1396]

Maybe you will find some answers here https://dalijap.blogspot.com/2020/06/magic-behind-freeandnil.html

[Sherlock 663]

@Dalija Prasnikar Thank you! I thought that might be the reason. Which brings me to my earlier question: Why use FreeAndNil on Pointers in the first place? Do you have any idea?

[Uwe Raabe 2057]

Just now, Sherlock said:

Why use FreeAndNil on Pointers in the first place? Do you have any idea?

Either someone didn't understand FreeAndNil or the code used TObject in the first place and later changed to a typed pointer, but forgot to adjust the FreeAndNil call.

[Guest]

Just now, Uwe Raabe said:

Either someone didn't understand FreeAndNil or the code used TObject in the first place and later changed to a typed pointer, but forgot to adjust the FreeAndNil call.

Or the idea initially was to have FreeAndNil for Objects and Memory Pointers, but the overloaded versions were dropped because the confusion they caused.

[Sherlock 663]

8 minutes ago, Uwe Raabe said:

Either someone didn't understand FreeAndNil or the code used TObject in the first place and later changed to a typed pointer, but forgot to adjust the FreeAndNil call.

You could think that to be the reason, but I'm looking at code with tons of PArrayOfByte like what I posted. This seems like some very, very old code and an array of byte is rarely implemented as a class is it? Much less back in the olden days.

[Stefan Glienke 2002]

Refactoring away from objects to something else and forgetting FreeAndNil is probably the number one source of defects of this category which is why I am happy for this change in 10.4 even though it also has its quirks most notably an API that tells a lie.

[sakura 45]

2 minutes ago, Stefan Glienke said:

... most notably an API that tells a lie.

Yeah, that was my first thought... not really better, though the name of the method itself tells the story. Yet, for someone, who does not quite yet understand all the quirks of Delphi, this is a little frustrating...

[Dalija Prasnikar 1396]

Everyone please vote for 

 

Global Generics

https://quality.embarcadero.com/browse/RSP-13724

 

So we can have type safe FreeAndNil without quirks and lies.

[dummzeuch 1505]

36 minutes ago, Kas Ob. said:

Or the idea initially was to have FreeAndNil for Objects and Memory Pointers, but the overloaded versions were dropped because the confusion they caused.

At least the Borland/Codegear/Embarcadero version of FreeAndNil never supported anything but instances of TObject descendants. And I doubt that there is a good way to implement this with overloading for other data types in any safe way.

[dummzeuch 1505]

33 minutes ago, Sherlock said:

You could think that to be the reason, but I'm looking at code with tons of PArrayOfByte like what I posted. This seems like some very, very old code and an array of byte is rarely implemented as a class is it? Much less back in the olden days.

This code must crash horribly on the first call to FreeAndNil with a non-nil pointer. So this code was either never used or the memory is already freed and the pointer set to nil before that call.

[Guest]

4 minutes ago, dummzeuch said:

And I doubt that there is a good way to implement this with overloading for other data types in any safe way.

May be the case, on other hand, changing ( or forcing it to be inlined) with little smart on compiler side, it can be used even with strings !, like for strings set its length to 0, with memory pointers call FreeMemory on it and so on..

But as you said it is doubtful to see such thing.

[Guest]

Such creature can be called FreeNullator or FreeNilator 

[Rollo62 536]

20 minutes ago, Kas Ob. said:

Such creature can be called FreeNullator or FreeNilator 

Would choose FreeNullminator, ... hasta la vista .... Object

[Remy Lebeau 1393]

8 hours ago, Sherlock said:

I also took a wrong turn somewhere and ended up in this C++ explantion of const http://docwiki.embarcadero.com/RADStudio/Rio/en/Const in detail this sentence: "A const pointer cannot be modified, though the object to which it points can be changed."

in C++, const is much more flexible than in Delphi, more so than that documentation explains.  The semantics of const depend on WHERE the const is placed.  In general, const applies to the thing on its left, unless there is nothing there then it applies to the thing on its right instead.

 

So, for instance, if we have these declarations in C++:

void FreeAndNil(const TObject* &Obj);
or
void FreeAndNil(TObject const * &Obj);

Then the const applies only to the TObject instance that is being pointed at, so its data members cannot be changed by the function.  The const does not apply to the TObject* pointer itself, which is being passed by reference, so that pointer can be freely assigned a new value by the function to point at a different TObject instance or null, and that change is reflected to the caller's variable. 

 

Whereas if we have this declaration instead in C++:

void FreeAndNil(TObject* const &Obj);

Then the const applies only to the TObject* pointer itself, thus it can't be assigned a new value by the function.  The const does not apply to the TObject instance, so its data members can be freely modified by the function.

 

The two examples above can be combined to have a const pointer to a const TObject instance:

void FreeAndNil(const TObject* const &Obj);
or
void FreeAndNil(TObject const * const &Obj);

It does not make sense to put const after Obj, since by definition a reference cannot be reseated once it has been initialized:

void FreeAndNil(... &Obj const);
or
void FreeAndNil(... &Obj const);

Now, given this declaration in Delphi:

procedure FreeAndNil(const [ref] Obj: TObject);

The semantics of this are roughly equivalent to the above C++ code - a reference to a const pointer to a const TObject instance (I don't have 10.4 installed to look at how FreeAndNil() is actually declared in C++Builder).  That is fine and good to get a pointer to a TObject-based instance passed into the function by reference, but normally Delphi would then not allow that pointer to be assigned a new value.  So FreeAndNil() employs a workaround for that:

TObject(Pointer(@Obj)^) := nil;

Applying the @ operator to a reference parameter returns the address of the passed variable.  Then all const-ness on that variable is being casted away, thus allowing the function to assign a new value to that variable.

 

That would be something along the lines of the following in C++:

((TObject*&)(*(void**)&ref)) = nullptr;
or
reinterpret_cast<TObject*&>(const_cast<void*&>(*reinterpret_cast<const void * const *>(&ref))) = nullptr;
or
const_cast<TObject*&>(const_cast<const TObject * const &>(ref)) = nullptr;

Except that if a C++ compiler sees a 'const TObject* const &' function parameter, and is passed anything other than a 'const TObject*' variable, an implicit conversion is performed and an rvalue gets passed to the reference parameter, so the function modifies the rvalue, not the caller's original variable. For example: https://ideone.com/CO4Dac So, there is likely some additional compiler magic in C++Builder to avoid this problem with FreeAndNil() specifically.  Unless I'm missing something.

[David Heffernan 2345]

2 hours ago, Remy Lebeau said:

In general, const applies to the thing on its left, unless there is nothing there then it applies to the thing on its right instead.

Is that is a little bit loose? Consider

2 hours ago, Remy Lebeau said:

void FreeAndNil(const TObject* const &Obj);

The second const binds to what exactly? Not "the thing on its left".

[Remy Lebeau 1393]

29 minutes ago, David Heffernan said:

The second const binds to what exactly? Not "the thing on its left".

Yes, actually it does.  The thing on its left is the * token.  It might help to visualize this if you add whitespace between the TObject and * tokens - the compiler will simply ignore that whitespace, but the tokens are still treated separate:

void FreeAndNil(const TObject * const &Obj);

So, the 1st const binds to the TObject type, and the 2nd const binds to the * pointer, so you end up with the Obj parameter being a reference to type "const pointer to const TObject".

Edited July 1, 2020 by Remy Lebeau

[Arnaud Bouchez 407]

At first place, I don't understand how FreeAndNil() on an array pointer could work properly.
It would try to call a Destroy method in the VMT, which doesn't exist... as @dummzeuch reported above.

 

What you should do ASAP:

 

1. Get rid of all those FreeAndNil() on something else than class instances.

 

2. If you can , try to replace those pointer arrays with dynamic arrays, so you would have reference-counting and automatic free of the content when the variable comes out of scope.
You can transtype a dynamic array into an array pointer just by using `pointer(aByteDynArray)`.

Edited July 2, 2020 by Arnaud Bouchez

[Remy Lebeau 1393]

10 hours ago, Arnaud Bouchez said:

At first place, I don't understand how FreeAndNil() on an array pointer could work properly.

It can't, that is a bug in the original code.  But the pre-10.4 version of FreeAndNil() would allow it to compile (but it would fail to operate properly at runtime).  The new 10.4 version of FreeAndNil() will not compile it.

Quote

1. Get rid of all those FreeAndNil() on something else than class instances.

 

2. If you can , try to replace those pointer arrays with dynamic arrays

Absolutely, and Delphi even has a TBytes type for this exact purpose (unless the PArrayOfByte pointer is coming from another library, in which case it needs to be passed back to that library for proper freeing).

Edited July 2, 2020 by Remy Lebeau

[Sherlock 663]

I have gone through the code and replaced those pesky FreeAndNils with FreeMem or Dispose, whatever fit. It compiles fine now, but next step is to redesign the code to get rid of those ancient pointer thingies altogether.  Quite annoying.

 

Thanks everyone for your help and insight!

[Rollo62 536]

38 minutes ago, Sherlock said:

I have gone through the code and replaced those pesky FreeAndNils with FreeMem or Dispose, whatever fit. It compiles fine now, but next step is to redesign the code to get rid of those ancient pointer thingies altogether.  Quite annoying.

That shows the core differences in the different languages:

 

C++    = pointer love
Delphi = pointer hate

 

But why: pointers are always, and will be always out there, till the end of days (at least when it comes to the bare metal).

Edited July 3, 2020 by Rollo62

[dummzeuch 1505]

42 minutes ago, Sherlock said:

I have gone through the code and replaced those pesky FreeAndNils with FreeMem or Dispose, whatever fit.

Have you checked whether this code is ever called? Since it must have resulted in runtime errors, I doubt that. So maybe there is a design flaw there.

[Sherlock 663]

12 minutes ago, Rollo62 said:

That shows the core differences in the different languages:

 

C++    = pointer love
Delphi = pointer hate

 

But why: pointers are always, and will be always out there, till the end of days (at least when it comes to the bare metal).

I don't hate them, I just don't need them. 😉

[maXcomX 3]

Why is CPPFreeAndNil declared in sysutils delphi 10.4 ?

Edited February 23, 2023 by maXcomX
Clarifying

[programmerdelphi2k 237]

OMG, sorry... wrong thread! sorry!

Edited February 23, 2023 by programmerdelphi2k