Delphi and "Use only memory safe languages"

[Dalija Prasnikar 1396]

23 minutes ago, David Schwartz said:

That's what I'd like to hear more about. 

If you really need to have multiple references to an object, then use interfaces and ARC to manage lifetime. If not, then you will have to invent similar mechanism for managing the lifetime.

[Lars Fosdal 1792]

Ref := ItemProvider.Grab(ItemId); 
// or Ref := ItemProvider.GrabForChange(ItemId);
try
  // do stuff
finally
  ItemProvider.Drop(Ref);
end;

ItemProvider can do the allocation and loading, as well as the disposal.
If there is parallell use, it can secure against readers/other writers, have a keep-alive in cache period, etc.

In theory, with the "smart pointer" trick, you could even do away with the try/finally.

[Cristian Peța 103]

43 minutes ago, David Schwartz said:

So what DO you do in a case where, say, you might use an object to collect an accumulation of data that is provided by multiple sources? 

 

It's often done for contextual state management over time...

 

The sources must register to the collector. And unregister when the source does not need the collector anymore.

The collector must not be destroyed if there is an active source.

 

This is a little like ARC for interfaces work.

[Stefan Glienke 2002]

This is why many modern and widely used languages have settled on using garbage collection (an automatic memory reclamation system) - because then the runtime deals with that and not the developer.

Edited April 9 by Stefan Glienke

[JonRobertson 72]

3 hours ago, mitch.terpak said:

I once tested GPT4.0 for some assembly code. Its actually quite good at explaining and improving Assembly code, but absolutely horrendous at writing it from scratch.

GPT and other AI services aren't capable of thinking. They only "know" what has been trained or other resources to use as a reference (ie code online that a human originally wrote). If I wrote some assembly code that I posted online somewhere and GPT used my code as a reference point, any code that GPT generated would blow big time. 

[PeterBelow 238]

2 hours ago, David Schwartz said:

 

That's what I'd like to hear more about. 

Consequent use of interface references (with rerf counting for lifetime control)  instead of object references avoids that problem completely.

[Brandon Staggs 277]

10 hours ago, David Schwartz said:

However, I seriously doubt they're using Delphi to do so. (queue up those here who want to tell me otherwise...)

I'll avoid direct pointer math when I can, but when you need to drive video, text, and geometry effects dynamically, live at 60fps, it helps to know theses things and use them where optimization is needed.

[FreeDelphiPascal 19]

On 3/1/2024 at 11:31 AM, David Heffernan said:

obj := TObject.Create;
obj.Free;
obj.Free;

Take that, NSA!

obj := TObject.Create;
FreeAndNil(obj);
FreeAndNil(obj);

------------------------------

 

On SO somebody wrote an example like this to prove how unsafe Delphi is:

  s: String;
  i: Integer;
begin
  s:= 'four';  
  for i:= 1 to 1138 do begin 
    write( s[i] );  // What will it access after i=4?
  end;
end;

 

At one point, we should stop talking about how safe the programming language is and start talking about how "unsafe" is the programmer

🙂

🙂

Edited April 11 by FreeDelphiPascal

[FreeDelphiPascal 19]

On 3/7/2024 at 12:39 PM, Kas Ob. said:

I mentioned or to be exact dreamed about more evolved Pascal enhancement, mentioned it here at once as i recall, but really can't remember any comments or any discussions.

So will write it here as it is relevant for memory safety.

My suggestion is to introduce new modifier(s) to managed (and non managed) variables, this include classes/objects, the new main modifier is "Auto", example

procedure Test(aFileName: string);
var
  SList: TStringList; auto;           // the compiler will ensure calling the default constructor and destructor
begin
  //SList := TStringList.Create;      //  should stop the compiler with specific Error, you are not allowed to create an auto
  SList.LoadFromFile(aFileName);
  //SList.Free                        //  should stop the compiler with specific Error, you are not allowed to destreuct/free an auto
end

..........

Too much useless talk about this topic.
But this guy (Kas Ob) came with real ideas/solutions. Great job Kas!

[Attila Kovacs 629]

TStringList has 4 other constructors with parameters.

This would just pollute the code and cause traffic jams everywhere.

[FreeDelphiPascal 19]

1 hour ago, Attila Kovacs said:

TStringList has 4 other constructors with parameters.

This would just pollute the code and cause traffic jams everywhere.

When have you ever called one of those constructors?

[Attila Kovacs 629]

5 minutes ago, FreeDelphiPascal said:

When have you ever called one of those constructors?

Doesn't matter. It was an example.

[FreeDelphiPascal 19]

On 3/8/2024 at 8:28 AM, David Schwartz said:

....

So I really don't want to hear about all of the exceptions and reasons why it's important to shave 50 ms off of the execution time of some loop you may be working on. The last place I worked printed bills and invoices, stuffed them into envelopes, and sent them out to the US Post Office -- 7 or 8 full 40' semi-trailers EVERY DAY, 7 days a week. They got pissed off if part of a line on an invoice was cut off on a print run of 50,000 invoices; not whether it took an extra 15 minutes to run a given print job, as long as the people who got the materials could read them and they looked "professional".

 

If customers didn't complain, that meant nothing was wrong. Nobody cared what we programmers might find.

 

One thing I've found over the past 15 years is that Management is far more scared of allowing programmers to refactor code than just about anything else. Because trying to shave a few seconds of processing time often leads to bugs that did not exist previously. And yes, that's because they don't have comprehensive test suites (which are notoriously hard to write for form-based Delphi apps).

 

 

Nice post David S!
I had recently to fix a bug in a program that was sending wrong financial reports to a governmental financial institution (equivalent to RSI in America) for 18 years. After 18 years, somebody observed the problem. It ended up that there was not only one bug, like 15. Nobody ever observed the other 14 instances where data was wrong.

I was told to shut up and move on. Also, nobody cared that my new program was running 100x faster than the previous one.

 

I told them that probably there were another 100 bugs in there (pascal-like code, no objects, thousands of warnings and hints - at least, there were no pointers!). They wouldn't let me rewrite the program. They said, it now it works, let it as it is.

 

So people, don't blame Delphi for not being totally memory-safe. It is mostly the programmer that dictates the safety of a language.
I guess the programmer that wrote that program in Delphi would be able to sneak in some calamities like this also in a language like Rust.

__________

 

Also, many people complained about buffer overflow here. Turn on the damn Range and Overflow checking. It will catch a good bunch of those overflows - unless the devil thought you to use pointers all over the place.

Edited April 11 by FreeDelphiPascal

[FreeDelphiPascal 19]

On 3/13/2024 at 5:44 PM, Rollo62 said:

To get back to try-except: The last time I learned about Pascal-vulnerabilities, it shook me and I understood many of the sporadic issues I saw in mobile development.

Thanks to Dalija's blog posts.

I don't think you read Dalija's article. It clearly states that it is hardware related, not language related.

[Rollo62 536]

1 hour ago, FreeDelphiPascal said:

I don't think you read Dalija's article. It clearly states that it is hardware related, not language related.

Yes, underlying hardware issues which could causes by various reasons, from permissions to misbehaving sensors and many more.
Since then, I enclose all these features inside a double-protection, to be hoepfully able to catch more of such execeptions.

 

[Dalija Prasnikar 1396]

55 minutes ago, FreeDelphiPascal said:

I don't think you read Dalija's article. It clearly states that it is hardware related, not language related.

It is language related, more precisely compiler related (LLVM backend). The hardware part is related to exceptions that happen in hardware which cannot be caught by LLVM backend. Old Delphi compiler (Windows one which has custom frontend and backend) can catch those because it was designed that way. LLVM origins are in C++ which on language level does expect catching such exceptions, so there was a "design mistake" in there that is now very hard to change, to accommodate slightly different languages.

 

Most common hardware exceptions are access violations (accessing nil or invalid memory) and floating point exceptions. See: https://dalijap.blogspot.com/2023/12/catch-me-if-you-can-part-ii.html

 

Plenty of languages are designed around notion that if they encounter certain types of issues they should just crash the whole application instead of trying to recover. Now, such behavior reduces the chances of certain vulnerabilities, but on the other hand makes very hard to deal with certain scenarios, especially gracefully detecting and handling bad input data, and they put a lot of more pressure to the developer to imagine all kinds of potential failures and prevent them explicitly in code. 

 

So in Delphi (Windows platform) accessing nil pointer will result with exception which you can catch and handle, so you can write code that will not check for nil instance and just try to access it and if it fails you can have exception handler that will handle such situation without anything bad happening. On the other hand, if you have such code in Swift (which also uses LLVM) your application will crash and burn and you cannot recover from such error at all. You need to pre-emptively check whether instance is nil when you expect it might be to avoid accessing nil instance. 

 

In Delphi compilers with LLVM backend, you can still catch nil instance access and recover, you just cannot catch it in the immediate code block, but it must be in separate procedure or method and only outside exception handler can catch and gracefully handle such exception. 

 

 

[FreeDelphiPascal 19]

23 minutes ago, Dalija Prasnikar said:

It is language related, more precisely compiler related (LLVM backend).

LLVM is not quite a programming language. So, you are right in a way: you could recover at the hardware level, but LLVM fails to do so. But then, let's not blame Delphi language itself for this.

 

So, I would correct myself and say that the issue is between the hardware and the language, in the LLVM. Do you agree with this "technicality"? 
🙂 

Edited April 11 by FreeDelphiPascal

[Dalija Prasnikar 1396]

7 minutes ago, FreeDelphiPascal said:

So, I would correct myself and say that the error is between the hardware and the language, in the LLVM. Do you agree with this "technicality"? 

Well, the problem is technically in the LLVM, so blaming the LLVM is fine 😉

 

But again, also it is not technically a bug in LLVM, but merely lack of certain feature, which is not so easy to add at this time.

[FreeDelphiPascal 19]

4 minutes ago, Dalija Prasnikar said:

But again, also it is not technically a bug in LLVM, but merely lack of certain feature, which is not so easy to add at this time.

Yes. I totally understand and agree with that.

Because of this, I went in my post above, and I changed the word "error" with "issue", to eliminate any interpretation. I like to be pedantic 🙂

Edited April 11 by FreeDelphiPascal

[David Schwartz 426]

On 4/9/2024 at 12:18 PM, Brandon Staggs said:

I'll avoid direct pointer math when I can, but when you need to drive video, text, and geometry effects dynamically, live at 60fps, it helps to know theses things and use them where optimization is needed.

Pretty much any programming language that lets you do bitwhacking can do this. Just like anybody can walk on their hands or ride a unicycle with some practice. The question is, "Why?" Just because you can? Most people would argue that C is far better for this sort of thing. 

 

Sure, Delphi already does this, but I don't see it being used as a language of choice by devs who do a lot of bitwhacking, like gamers.

 

But at some point, this becomes a circular argument: Why doesn't Dephi support language features that most contemporary languages now include? Because nobody uses Delphi for that purpose. Why is that? Because it's too inefficient without the built-in language support. As if that's going to encourage the Powers That Be to take an "Add it and they will come" attitude...

I want to see enhancements that make the language safer, and make it easier to use with less verbiage for things that other languages now routinely support. Constantly offering-up verbose and unsafe workarounds is not encouraging to anybody.

 

Edited April 14 by David Schwartz

[David Schwartz 426]

On 4/11/2024 at 4:08 AM, FreeDelphiPascal said:

Nice post David S!
I had recently to fix a bug in a program that was sending wrong financial reports to a governmental financial institution (equivalent to RSI in America) for 18 years. After 18 years, somebody observed the problem. It ended up that there was not only one bug, like 15. Nobody ever observed the other 14 instances where data was wrong.

I was told to shut up and move on. Also, nobody cared that my new program was running 100x faster than the previous one.

 

I told them that probably there were another 100 bugs in there (pascal-like code, no objects, thousands of warnings and hints - at least, there were no pointers!). They wouldn't let me rewrite the program. They said, it now it works, let it as it is.

 

So people, don't blame Delphi for not being totally memory-safe. It is mostly the programmer that dictates the safety of a language.
I guess the programmer that wrote that program in Delphi would be able to sneak in some calamities like this also in a language like Rust.

 

 

I can totally relate to this!

 

At one gig I was on, I discovered some coding patterns in a quite large Delphi system that compiled without any warnings or hints, but were wrong. It was by sheer luck that the code worked most of the time, and when it didn't, it produced strange down-stream errors unrelated to the actual coding mistakes. 

 

I tracked down over 100 instances of these patterns and tried mightily to get them fixed. At one point, I pushed out a version with about half of the issues corrected. The testing group asked what happened because my new update was running about 25% faster. They were very happy. At the end of the day I asked if they had encountered any of the random irreproducible errors they seemed to get regularly, and was told "not one!" The next day, I pushed out an update with the rest of the issues corrected and speed increased again. Also, none of the irreproducible errors showed up. But the third day I was told to revert everything because the tickets I put into the backlog weren't done properly; they wanted me to enter a separate ticket for each change. So I reverted the code and the next day the testing team was unhappy because it was running slowly again, and those weird errors were back.

 

I kept trying to get these issues resolved, but after 9 months of this nonsense, I believe I was terminated for refusing to stop.

 

At one point, someone suggested that, for this particular project, the company had guaranteed to the client that it was "bug-free", although any errors they found and could give us code that reproduced the error, then it would be fixed.  For all my efforts, all that happened was that more stringent rules were published about who can submit bugs, and ultimately the Dev Team was explicitly prohibitied from introducing either bug or refactoring tickets into the backlog (even though we supposedly "owned" the backlog). A few days before I was let go, word came down that every single line of code that was changed had to have a comment tying it back to a specific ticket in the backlog, and an audit of said changes was added to our sprint completion meetings. 

 

To make matters worse, there had been dozens of reports of a certain type of error that was not reproducible, and part of what I discovered explained this -- it was a data-sensitive error, and feeding the same data in repeatedly via the UI did not produce the exact same result in terms of what the erroneous code was looking at, so it was not reproducible. 

 

--------------------------------------------------

 

To this day, nothing has been done to the Delphi language that would help these issues be detected and reported by the compiler. I have no idea if this code ever went into production or not, but those 100+ coding errors were bugs waiting to be triggered. I only found them by accident, stumbling into some code that looked odd but actually compiled.

 

After a while, it suddenly made sense when I looked at from another perspective -- one that was specific to the main programmer's previous experience and the language he used for several years prior to working with Dephi. Reading bits of that code as if it was written in that other language was exactly what he expected the code to be doing. Delphi considered it syntactically ok, but it was nonsense from a semantic standpoint, although Delphi didn't care about that part. I even ran samples of this code through some Pascal code analyzers and they didn't catch these errors either.

 

Unfortunately, backtracking from the points where the faults were raised was fruitless. 

Edited April 14 by David Schwartz

[FreeDelphiPascal 19]

On 4/14/2024 at 2:41 PM, David Schwartz said:

I want to see enhancements that make the language safer

I have seen nasty sh*t in my life. So, I am totally for this!

[Brandon Staggs 277]

On 4/14/2024 at 7:41 AM, David Schwartz said:

Pretty much any programming language that lets you do bitwhacking can do this. Just like anybody can walk on their hands or ride a unicycle with some practice. The question is, "Why?" Just because you can? Most people would argue that C is far better for this sort of thing. 

So what? Lots of people use Delphi to do it and... why is that a problem for you? You seem irrationally fixated on the fact that some people use this tool for tasks you would not personally use it to accomplish. 

 

An no, not "just because you can." "Just because you can" doesn't pay the bills. Anyway I hardly see the point in trying to explain it to someone who seems absolutely determined not to get it.

Edited April 15 by Brandon Staggs

[dummzeuch 1505]

1 hour ago, Brandon Staggs said:

And no, not "just because you can." "Just because you can" doesn't pay the bills.

Actually, in my case it sometimes is "just because I can". I use Delphi not only to pay my bills, sometimes I just want to have some programming fun, and I imagine I'm not alone.

 

But yes, there are actual reasons to use features in Delphi that are frowned upon by some people, even the discouraged goto.

[JonRobertson 72]

3 minutes ago, dummzeuch said:

there are actual reasons to use features in Delphi that are frowned upon by some people, even the discouraged goto

Hopefully not with 

 

Shift-Alt-W (MMX Convert with statement) is a good friend of mine when "migrating" legacy Delphi code.

Edited April 15 by JonRobertson