Pointers are dangerous

[dummzeuch 1517]

Pointers can be really helpful, especially they can improve performance and readability, but they are also dangerous. I spent nearly a day tracking down the reason why the code a former colleague wrote about 5 years ago all of a sudden led to access violations. The code used to work fine, the problem only surfaced when I changed the size of a record. Consider this code ...

https://blog.dummzeuch.de/2018/12/11/pointers-are-dangerous/

[Ugochukwu Mmaduekwe 42]

Aeroplanes are dangerous, does that mean we don't take flights? 

[Attila Kovacs 631]

32 minutes ago, dummzeuch said:

The code used to work fine

 

 *Read: The code used to work by accident.

Edited December 11, 2018 by Attila Kovacs

[Sherlock 663]

Aside from performance I have never considered pointers to be an improvement for readability, it's something that's messed up in my brain I guess. And the way it was taught back in the day: Calling it "pointer arithmetic" really did not help understanding the simple things that are going on there. Luckily there is no syntactical need for pointers in Delphi anymore. Been living free of ^ and @ for years now. Of course sometimes some API smuggles in PBytes or PChars, but its nicely encapsulated.

[Kryvich 165]

A good case for a static code analyzer.

[dummzeuch 1517]

21 minutes ago, Sherlock said:

Aside from performance I have never considered pointers to be an improvement for readability,

I consider this

Ptr := @SomeArray[SomeStructure.SomeIndex];
Ptr.bla := 5;
Ptr.blub := 10;
// and 10 more lines like this

as more readable than

SomeArray[SomeStructure.SomeIndex].bla := 5;
SomeArray[SomeStructure.SomeIndex].blub := 10;
// and 10 more lines like this

Of course we could always use WITH instead. 😉

 

Or we could use a list with objects where the pointer is implicit, but then we'd need a lot of boilerplate code.

 

I would have written it differerently, but this is legacy code which I'd rather touch as little as possible (because it will hopefully become redundant shortly).

Edited December 11, 2018 by dummzeuch

[Rollo62 539]

So you're looking for a pointer as alias 🙂

 

Maybe this will be the next language extension

[Kryvich 165]

I would prefer it like this: 

with const Ptr = @SomeArray[SomeStructure.SomeIndex] do begin // Inline constant declaration with type inference
  Ptr.bla := 5;
  Ptr.blub := 10;
end;

@Rollo62 Not only. Using a temporary variable can optimize the binary code.

[dummzeuch 1517]

6 minutes ago, Kryvich said:

A good case for a static code analyzer.

Not sure what kind of code analyzer would have spotted that problem. One that flags all pointer types as violation of "good coding practice"? Yes, that would work, but it would also prevent any usage of pointers. Or is there really one that would have found this problem?

[Stefan Glienke 2019]

7 minutes ago, dummzeuch said:

Not sure what kind of code analyzer would have spotted that problem.

Simple, one that knows that you are pointing to an array element and sees that you are resizing the array thus invalidating this pointer.

[dummzeuch 1517]

Just now, Stefan Glienke said:

Simple, one that knows that you are pointing to an array element and sees that you are resizing the array thus invalidating this pointer.

That would proabbly have worked for the simplified example, but I doubt that it would have caught the problem in the real code. Do you know an analizer that finds this kind of problems? I would like to try it.

[Kryvich 165]

@dummzeuch There could be a warning: "The pointer ptr1 to an element of the array arr is invalid after its reallocation by the function SetLength". There is a plenty of static code analyzer for Delphi - https://stackoverflow.com/questions/532986/are-there-any-static-code-analysis-tools-for-delphi-pascal. But I do not know one that catches this error. It would be great to have a utility for Delphi similar to PVS Studio.

Edited December 11, 2018 by Kryvich

[Sherlock 663]

41 minutes ago, dummzeuch said:

I consider this

Ptr := @SomeArray[SomeStructure.SomeIndex];
Ptr.bla := 5;
Ptr.blub := 10;
// and 10 more lines like this

as more readable than

SomeArray[SomeStructure.SomeIndex].bla := 5;
SomeArray[SomeStructure.SomeIndex].blub := 10;
// and 10 more lines like this

 

And I would identify the type behind @SomeArray[SomeStructure.SomeIndex] which you would have to do for that typed pointer you're using anyway and gotten a variable of that type. Making  this just as readable without explicit use of pointers:

Element := SomeArray[SomeStructure.SomeIndex];
Element.bla := 5;
Element.blub := 10;
// and 10 more lines like this

I am very well aware, that there are pointers floating around behind the scenes all the time. But I prefer developing with a high level language also because I wont have to deal with them.

[Uwe Raabe 2064]

6 minutes ago, Sherlock said:

And I would identify the type behind @SomeArray[SomeStructure.SomeIndex] which you would have to do for that typed pointer you're using anyway and gotten a variable of that type.

That would work for classes, but fails for records. Introducing a record variable will give you a copy of the array element for your changes. The original array element will stay unchanged.

 

A valid alternative would be a record method like SetBlaBlub(5, 10) which directly manipulates the underlying record. This might even benefit the readability.

[Sherlock 663]

Correct! Luckily I have very few records and if I need a list of them I use generic lists and the readability is just fine.

 

Edith also feels the need to mention I have no legacy code to deal with aside from Delphi (hahaha, small joke). So, of course Thomas' method is a very good and simple way to increase readability in such problematic code segments.

[Stefan Glienke 2019]

46 minutes ago, dummzeuch said:

Do you know an analizer that finds this kind of problems?

Probably not for Delphi because there is virtually no market but you would be surprised what static code analysis can do.

[Arnaud Bouchez 407]

FastMM4, in FullDebugMode is able to track such pointer problems.

I always use it, and also the FPC's Heaptrc unit (compile with -gl) which can find some other problems.

 

I use pointers only when the code needs it, mainly for performance reasons.
Otherwise, high-level structures like classes or dynamic arrays are just enough.

Edited December 11, 2018 by Arnaud Bouchez

[Clément 148]

Pointers are part of the package. I use them for performance and low memory footprint especially when dealing with low level code (bytes, records or low level TCP communication).
The gain pointers bring to the table is worth the risk.

[Sherlock 663]

While I believe performance benefits to be true, I wonder how often this is checked and verified. The compiler team may indeed have sped up things in the more recent releases.

[Attila Kovacs 631]

@Uwe Raabe Or one could just assign the local variable back to its origin. Should work from D1.

Edited December 11, 2018 by Attila Kovacs

[Uwe Raabe 2064]

4 minutes ago, Attila Kovacs said:

Should work from D1.

In that (probably unrealistic) case I prefer a method taking the record as the first var parameter (which is pretty much what the record method does internally, but I like the elegance of record methods and helpers).

[Attila Kovacs 631]

@Uwe Raabe Yes, but no, it does not fit to the shown example.

[Clément 148]

4 hours ago, Sherlock said:

While I believe performance benefits to be true, I wonder how often this is checked and verified. The compiler team may indeed have sped up things in the more recent releases.

Let's say you receive a string with null chars in several places separating information. For instance:
delphipraxis#0pointers are dangerous#0but worth the risk#0000#000#000000#0

 

How can you quickly process the information, check if they are valid and answer to that request.  You have to return a modified string to the client.
In that example you have to answer:

delphipraxis#0pointers are dangerous#0but worth the risk#0101#022#014995#0

Of course the trick here is to avoid copying string over and over. Allocating and deallocating classes or complex structures to process the input information.

With a simple array of pointers I can use this very same memory spot pointing to the beginning of each string without allocating or copying information.

The alternative would be to create a class (TStringList for example), set some properties, use "DelimitedText", have a copy of each string , allocate another "Answer string" and at the end deallocating everything ->defragmentation nightmare. (The input string length varies!)
 

Just picture this example in a network with hundred (or thousands) of devices sending requests of 10k bytes ( strings just like the above one but with 10k null separated values), and you get your performance example. Fragmentation is kept to a minimum and a service running for months without any problem.

 

Is it even possible to improve the compiler as much?

Edited December 11, 2018 by Clément

[Stefan Glienke 2019]

3 hours ago, Clément said:

Is it even possible to improve the compiler as much?

Yes, you can improve the compiler (the language) as much as that it is partially safe to access raw memory without copying it - google for: .NET Span<T>

I did some experiments with it in Delphi and it improved some string parsing code that did not have to allocate new strings but also did not have to work with raw and rather unsafe PChar.

See https://bitbucket.org/snippets/sglienke/7e6xoe/span - this however cannot do what the C# compiler can do with ref (ensure that it only lives on the stack to not lives longer than what it refers to and these things) plus C# has readonly ref - basically a readonly pointer.

Edited December 11, 2018 by Stefan Glienke