Dalija Prasnikar

16 hours ago, balabuev said:

I meant in Delphi's documentation. You yourself suspected me that I'm speaking about anything, but not about Delphi, right?

Programming languages share some common concepts. But it is important to distinguish things that are comparable and things that are not comparable. Definition of value and reference type does not change with language. There are various implementation details around both, but basic classification is the same. Value types are stored directly in memory associated by variable. And when you are accessing that memory you will always access valid memory as long as you can access that variable. Reference types have two parts, and variable only holds pointer to the actual data that is dynamically allocated on heap. That associated memory holding actual data, may or may not be allocated and your variable can point at invalid memory location that does not hold valid data.

That crucial difference between value types and reference types also has huge impact on their behavior in any language. But the actual behavior will differ because every language has different memory management. So what happens and how memory of associated reference is handled cannot be easily compared - this is where you cannot say, Java does this... so Delphi should be able to do this too.  

And I said "Java has garbage collection, so you absolutely cannot compare it to what Delphi does."

The fact that Delphi documentation does not specifically mentions value types and reference types that does not mean that this classification does not exist and that it does not have impact on behavior of different data types. Many things are not explicitly written in documentation, that does not mean they are not real.

From http://docwiki.embarcadero.com/RADStudio/Sydney/en/Internal_Data_Formats_(Delphi) description of string type it is pretty clear that string variable holds pointer to actual data, and that is clear definition of reference type.

Quote

A string variable of type UnicodeString or AnsiString occupies 4 bytes of memory on 32-bit platforms (and 8 bytes on 64-bit) that contain a pointer to a dynamically allocated string. When a string variable is empty (contains a zero-length string), the string pointer is nil and no dynamic memory is associated with the string variable. For a nonempty string value, the string pointer points to a dynamically allocated block of memory that contains the string value in addition to information describing the string.

16 hours ago, balabuev said:

But, even this definition from Wikipedia, which is by the way quite abstract (as it should be!), does not contradict with my point of view. Value of the string type is an ordered sequence of characters. Logically, nothing wrong

You are confusing value of the string with value type. Value of the string is sequence of characters, but they are not directly stored in string variable, but in some other location. Value stored in the string variable itself is just pointer to that location.

1 hour ago, balabuev said:

You invented this definition for youself. Btw, in fact, the definitions of "value type" and "reference type" are nowhere documented. And I do not agree to equalize reference type notion with just something allocated on heap.

Sure... yep... this is my imagination running wild... https://en.wikipedia.org/wiki/Value_type_and_reference_type

1 hour ago, balabuev said:

In common sense reference type is something, which allow to hold references to shared data (emphasis, as you say, is on SHARED word). This "shared" property should be visible to the user. So, the user should be able to determine, whether two references are pointing to single data or not.

There is no common sense there is clear definition what reference type is and what value type is. 

You can always see address stored in any reference type, including string. It just requires typecast.

5 minutes ago, balabuev said:

The "desired behaviors" of the string type is to behave like a value type.

Emphasis is on BEHAVE LIKE. It is not string IS value type. And simulating something is not equal to having that exact thing.

5 minutes ago, balabuev said:

Anyway, "s1" and "Value" in the test code are two different variables. Both of them store values, so we speak about two different (but equal) string values. And when you change one of them, the other should not be changed. Because:

 

- [My viewpoint] string is a value type.

- [Your viewpoint] internally string implements COW, and so, this is a desired behavior.

string in Delphi is reference type. It is not value type. This is not a viewpoint it is a fact. If you cannot acknowledge facts, this discussion is pointless.

s1 and Value are two different references to ONE memory location. When data in that memory location is destroyed because you change s1 variable that was only strong reference to that memory location, Value will point to invalid memory. 

5 minutes ago, balabuev said:

When something is leaking through - it's called a bug.

I am talking about leaking abstractions (as implementation details cannot be completely hidden), not bugs. You cannot completely separate idea (concept) of a string and implementation of a string. 

26 minutes ago, balabuev said:

String type conceptually is a value type. Come on! All thouse tricks, like ARC and copy-on-write (as well as copy on taking address of char, etc) are exist to allow string type to be a value type.

Copy on write is not equal to being value type. You may not like "implementation details" and "tricks" but they are more than implementation details, they achieve desired behaviors. You cannot have full value type behavior with something that is not value type. You can only simulate some parts, but that also means that implementation details of actually being a reference type and having ARC to handle its lifetime and COW leak through.

30 minutes ago, balabuev said:

I want to say that you again messed up concepts and implementation details. Pointers, heap data, ref-counts - are all implementation details terms. And just for note, I undestand how things works internally very precisely. But, I intentionaly want to keep the logic on the conceptual level.

Compilers (languages) are always leaky abstractions. At some level you have to define what specific concept means and then that leads to its implementation. There is no universal concept of const parameters. There are many ways how parameters are passed and different languages greatly differ.

Quote

[1] Well, yes and no. Being formal - you are not correct, because the value of reference type is the reference itself. Yes it points to some associated heap data, but the value is just a reference itself.

 

[2] Conceptually, string type in Delphi is a value type (I don't care about implementation details).

 

[3] As described in documentation, "const" parameters are variables, just like value parameters. And this is quite important, because we should recall simple thing: variable is a slot which holds a value.

 

From [1], [2] and [3] the following very important idea can be formulated:

 

Because of [3] each variable holds its own value. Values (conceptually) are not shared between two or more variables. Because of [1] this is also true for reference types. And, finally repeating [2] - string type in Delphi is a value type, NOT a reference type.

 

* Disclaimer: As I noted in this topic previously, practically, I agree that "const" modifiers cannot be implemented better than currently. And so, this is a kind of unfixable inconsistency.

 

String type is reference type, not a value type. Only short strings are value types. There is no conceptual here, at least not when Delphi is concerned. Your conceptual discussion would be discussing some completely different language.

16 hours ago, Kas Ob. said:

in C we have string has refcount at +3 from initially value at begin of A, why ? ... what could possibly have changed in lets say B to increase the refcount, if the value to be shared then lets C do it, in C we didn't change anything then remove the refcount, in WriteLn do the refcount only if needed,

In other words do we have thread safe strings ? No, then why the compiler is sticking and overdoing an unproductive and useless pattern with no benefit at all, when the execution path that the compiler is generated machine code for is clear as rain.

Your posts clearly show that you have no idea how reference counting actually works, nor how compiler works. I don't mean this as insult, but merely as observation.  Not knowing something is not a problem on its own, but it is hard to have meaningful discussion when other party does not have basic idea about discussed topics. 

I could try to explain it in more details, but I don't know from where should I start. Especially, because you seem to have wrong perception about how compiler works. I am worried that we would be running in circles. Also both topics have enough content around that covering everything from the start would require writing a book.

16 hours ago, balabuev said:

We should try to split concepts from implementation details.

In theory concepts are one thing, compiler specifications and implementation details another. Unfortunately Delphi does not have formal compiler specification, and compiles implementation is often the only specification. Of course, there are always corner cases where it is hard to say whether some behavior is intended or it is a bug, but in this case - const parameters behaviors are pretty well defined.

16 hours ago, balabuev said:

Conceptually "const" means that we are not allowed to change the value of the parameter inside the function. Conceptually "const" does not necessary imply passing by reference. Here is a definition from docs:

 

Passing by reference can be considered only as an optimization, which compiler should do silently and on its own risk. Optimizations should not violate formal language concepts and should not introduce side effects in correct code.

I don't think you fully grasp meaning of the above sentence. Yes, const means that you are not allowed to change parameter inside function and it does not necessarily imply passing by reference. But, you are forgetting that reference types consist of two parts - reference (that is the only part passed as parameter and cannot be changed) and associated content (data) that is allocated on a heap. 

Besides reference types, Delphi also has value types. And the optimization part of passing by reference is primarily for passing value types larger than register size.  Since references (pointers) are not larger than register size, they will not be passed by reference, but by value. 

16 hours ago, balabuev said:

Original Marco's code is fully correct, and should work, simply because Value parameter is not assigned inside the function body, and thus "const" constraint is not violated by the programmer.

procedure TForm1.Button1Click(Sender: TObject);
var
  s1: string;

  procedure Test(const Value: string);
  begin
    s1 := '456';
    ShowMessage(Value);
  end;

begin
  s1 := Copy('123', 1);
  Test(s1);
end;

* All this is my theoretical viewpoint.

 

Code is not correct and it will fail, not because Value is somehow changed - it is not and it is fully compliant with definition of const parameters. Problem arises because being passed as const - where reference cannot be changed triggers reference counting optimization for reference counted types. Since parameter will not be changed, there is not need to take care of reference counting. Because reference counting is not triggered, original string (same would apply for any other reference counted type) where Value reference points to, will be destroyed when s1 is assigned new value - in that moment Value reference that was not changed will point to now invalid memory location. 

It is also important to note that actual reference count is maintained as part of heap allocated data - not inside reference variable.

52 minutes ago, David Heffernan said:

There is a well known design flaw with passing arguments to const arguments.

I probably wasn't specific enough when I said there are no flaws in compiler. I meant only related to what @Kas Ob. was saying and his example does not have const parameters. Also, only object instances suffer from this particular bug.

21 minutes ago, Kas Ob. said:

Delphi compiler and RTL really need a review, and i hope that i explained how this is a bug and flawed design, also really sorry for the English (i tried!)

You are mixing a lot of things here. 

First, Java has garbage collection, so you absolutely cannot compare it to what Delphi does.

WriteItDown procedure is compiled independently of the call site - it has to cover all possible variants. So while your example only passes string literal that has special handling, WriteItDown still has to take care of situation where dynamically allocated string variable will be passed to it. Hence try...finally block. 

As for the rest is concerned I am not sure I am following what you want to say... what const are you talking about? WriteItDown does not have const parameter, so it increases and decreases reference count as expected. 

Again there is no flaw in compiler, is works exactly as intended. If you pass reference counted parameter as value it will trigger reference counting mechanism, if you pass it as const it will not. You can choose to make parameter const or not depending on what method does with it. And again special handling for string literals is required because compiler does not know what kind of string you passed in when you called the method. 

There are two problems with using Boolean parameters.

First, is when like in Nick's example you are literally calling two separate functions to do something based on boolean parameter. If there is common logic shared inside the method, where making two distinct methods would violate DRY principle, then it is fine to have parameter that will make your method do two things.

Another problem using Boolean parameters, when you have valid reason to make method do two things is that boolean on its own is very rarely descriptive. 

EnableDisableControls(True);

Does the above call enable or disable controls? 

If you need to make method do two things, it is better to use enum with two values that will give you more descriptive code on the call site.

In your example if the method would be called EnableControls, then using Boolean parameter would be self descriptive. It is pretty obvious what following line does:

EnableControls(True);

On the other hand boolean in following method makes code unreadable

ListFiles(const APath: string; IncludeSubFolders: Boolean) 

Can you tell what following code does, unless you know ListFiles declaration by heart?

ListFiles(Path, True);

In such case enum, is much better option even though it would have only two states.

ListFiles(Path, optIncludeSubFolders);

3 hours ago, A.M. Hoornweg said:

I find "const" parameters confusing at times, especially when passing stuff like arrays. If I pass an array as a const, am I supposed to be able to modify elements or not?   The Delphi documentation (link at he bottom) says "... constant parameters are similar to value parameters, except that you cannot assign a value to a constant parameter within the body of a procedure or function..." . 

 

I don't want to nit-pick, but I find it very confusing and counter-intuitive that I can modify the array in such a case.  

This is nature of reference types. It is possible to modify their content. Only reference is passed as constant, not the actual content. 

Being confused about reference types and how they are passed to the procedures is not Delphi specific thing. For instance, Java passes all parameters as values, but in case of reference types (objects) that does not mean content of the object cannot be changed, unless it is made immutable through some other means (for instance, not having setter or similar methods that would mutate content). That causes two misconceptions, first. that objects references in Java are passed by reference, not by value and second that otherwise mutable objects cannot be changed inside method.

Of course, comparing to Java, Delphi has many more parameter modifiers and that certainly complicates situation and possible variations. 

I wrote more elaborate post on const parameters https://dalijap.blogspot.com/2021/01/are-const-parameters-dangerous.html

This problem is pure ARC problem. One instance of something is destroyed when other one is created and assigned because const parameter did not trigger reference counting mechanism - this is why const is used as speed optimization in the first place.

Same thing that happens with strings happens with reference counted objects, but it is easier to see what happens with objects. You need form with memo and button to test following code.

  IFoo = interface
    function GetCount: Integer;
    function GetNumber: Integer;
  end;

  TFoo = class(TInterfacedObject, IFoo)
  private
    FNumber: Integer;
  public
    constructor Create(ANumber: Integer);
    destructor Destroy; override;
    function GetCount: Integer;
    function GetNumber: Integer;
  end;

constructor TFoo.Create(ANumber: Integer);
begin
  inherited Create;
  FNumber := ANumber;
  Form1.Memo1.Lines.Add('Created ' + FNumber.ToString);
end;

destructor TFoo.Destroy;
begin
  Form1.Memo1.Lines.Add('Destroyed ' + FNumber.ToString);
  inherited;
end;

function TFoo.GetCount: Integer;
begin
  Result := FRefCount;
end;

function TFoo.GetNumber: Integer;
begin
  Result := FNumber;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  s1: IFoo;

  procedure Test(const Value: IFoo);
  begin
    s1 := TFoo.Create(456);
    Memo1.Lines.Add('Test Value ' + Value.GetNumber.ToString);
  end;

begin
  s1 := TFoo.Create(123);
  Test(s1);
end;

Output is:

Created 123
Created 456
Destroyed 123
Test Value 123
Destroyed 456

It is pretty obvious that object instance holding 123 is destroyed before its value is printed. Value.GetNumber.ToString is accessing dangling pointer - if the memory where object or string previously occupied is not reused, you may get expected value even though that instance is dead, but if memory is reused in the meantime you will read content of some other data. Also the whole thing might crash, or it might not. Basically you get undefined behavior.

15 hours ago, Kas Ob. said:

Congratulations and wish you success.

Thanks!

15 hours ago, Kas Ob. said:

Only i don't understand this, did you added +100 page and finished a book, then released it in one month and called this a delay ?

How could you call that a delay ?!

About the delay... the final release was originally planned for December 11, but I had a minor accident on November 30, and ended up in ER, where I also tested positive for COVID! In less than a week whole family got infected. We all had mild symptoms and were left at home with no treatment., but living was pretty hectic for most of the following two weeks... Anyway, we managed to release it about 16 days later than expected (I already had a lot of it written before, just not fully ready for release)

I would say that all this is behind us, as it is... but as you may have heard it... Croatia got hit by major earthquake today. This one was 50km from Zagreb where we're located, but it was 30 times stronger than the one that hit Zagreb in March. After 6.2 by Richter scale, local Nuclear Power Plant (Krsko) was shut down, and power is all but stable, which is why I may not be online as regularly as before in the coming days!

5 hours ago, Rollo62 said:

@Dalija Prasnikar

I could download only the PDF-Version, not ePUB and MOBI, is that correct ?

Anyway, I prefer PDF anyway, but I never know what eReader gadgets I will buy in the future.

It looks like the download link was not reset properly for the original version buyers. It should work now! If you're still having problems, please contact us through the website form at dalija.prasnikar.info. We may need additional information.

Final version of my eBook Delphi Event-based and Asynchronous Programming has been released - 291 pages.

You can find more information and purchase option at: https://dalija.prasnikar.info/delphiebap/index.html

Thanks to all of you who purchased the incomplete pre-release version (179 pages) of my eBook!

After a minor delay, the full version is here! You can download it through the same PDF/epub/mobi links that you have received earlier from FastSpring via email. 

The subject line of that email message was: "Your Delphi Event-based and Asynchronous Programming - Part I Delivery Information".

If you have any problems, feel free to contact me via the contact form on my site.

Thanks again!

Happy Holidays to you all! 

15 minutes ago, vfbb said:

But at least this limitation should be documented, since crashes generated by this limitation are difficult to diagnose.

Well, there is really nothing to document because nothing is really thread safe in Delphi. Unless you want every single documentation page having "This is not thread safe"  (I am exaggerating a bit, but this is generally true in 99.99%)

13 minutes ago, vfbb said:

what is the scenario that does not fit to use Unsafe, that is really necessary Weak and that is really safe to use Weak? This scenario does not exist

You use weak when you cannot guarantee that weak reference lifespan will be less or equal to original strong reference.

For instance, let's pretend that TComponent is truly reference counted class, and that same goes for its descendants TEdit and TMenu. Edit has PopupMenu property that can be set to independent TMenu instance. Currently destroying such TMenu instance would leave dangling pointer in Edit PopupMenu property, but we have notification mechanism that notifies Edit that Menu will be destroyed and that Edit should clear reference to the menu.

In imagined reference counting scenario, you could have popup menu marked as weak reference and you would not need any notification mechanism to clear that reference when PopupMenu instance is deleted because weak would zero out (clear) that reference when menu is destroyed. If you use unsafe instead of weak, then you would have dangling pointer in PopouMenu property. 

8 minutes ago, vfbb said:

Yes Anders! There is no way to workaround this, the solution is redesign the code to doesn't depend or use weak attribute.

 

dalija, that's why Unsafe exists.

No. While unsafe can also be used to break reference cycles, it cannot be used interchangeably with weak, meaning unsafe used in wrong place can lead to dangling pointers. 

8 minutes ago, vfbb said:

if you cannot transform WeakToStrong reference safetly, then you should not use Weak, you need to redesign your code to not use Weak, or if you just need to avoid circular reference, use Unsafe.

 

Don’t use Weak, it is dangerous. This is the title and this is truly, once it is not thread safe.

Again, using weak is perfectly fine. If you read my book example again, you will see that you cannot even safely take strong reference from strong reference if original has been written to.

So now what? Don't use interfaces at all because they are not thread safe. Don't use strings and dynamic arrays because they are not thread safe?

You are mixing apples and oranges. 

Weak references work fine and have their purpose when used in single thread scenario. Their purpose is not achieving some magical thread safety, but breaking reference cycles.

I will say it again, assignment of any non trivial type in Delphi is not thread safe. Never has been. As soon as one thread is writing some data, all access to that data must be protected. There is no way around it.

Assignment operation in Delphi are not thread safe. If you have one thread writing and other threads reading, such access must be protected or you will get crashes or buggy behavior. 

You can safely write and read 32bit aligned simple types, in terms that such code will not cause any crashing, but such code will also not work as intended, unless you don't care about actual values stored.

So, weak references in Delphi are not thread safe. Also even strong interface references in Delphi are not thread safe. So there is no bug here, just bad test case.

Excerpt from my book Delphi Event-based and Asynchronous Programming (Part 5. Thread Safety) 

I just had to do it... Go with the flow!

It's the Black Friday / Cyber Monday "season", so I had to rush the book offer 

The book was actually scheduled for release in early December, but then the Black Friday deals started popping up all over the place... pressure was building... and I finally caved in!

I cut the darn thing in two, and decided to offer Part I at a discount, and will give Part II for free to all Buyers. Not really a 97% BF discount but hey... Junior is still studying, and we still have to pay the bills (Corona doesn't help, either)!

So, here it is! 

I hope you'll like it like you did Delphi Memory Management!

https://dalija.prasnikar.info/delphiebap/index.html

Thank you all for the support!

23 minutes ago, Der schöne Günther said:

Very true. The same applies to Embarcadero. If 64 Bit or DPI scaling is so easy, then why does their own IDE not support it yet?

64bit IDE is not question of easiness. It is also question of 3rd party libraries and design packages support. You cannot mix 32bit and 64bit DLLs so having 64bit IDE would mean everyone would need to provide their components and plugins as 64bit versions, too. Also, because you cannot mix 32bit and 64bit DLLs in a process, there is question of other parts of the Delphi IDE - this is not just Editor, but also compiler, linker, debugger... not everything is written in Delphi and porting any of the necessary parts to 64bit may not be straight forward.

Easy part of 64bit transition is about Delphi applications, even though depending on your code you may have smooth or bumpy ride, too. I didn't have to do any adjustments for my code, because I never used any hacks around integer - pointers casting and I always had bitness explicitly specified for any data types I have used. So, generally most of transition to 64bit should be easy, but it also depends on specific projects and other libraries, plugins and interactions they might have and use.

Scaling is bit different story... VCL Styles originally didn't have High DPI support, and when they were added to IDE theming in Rio they were still without High DPI support... now they have High DPI support, but tweaking the IDE to fully work under High DPI does require some work. This part would be the same with or without VCL Styles. 

I would first make IDE high DPI aware and then add styles... but that is me... why are some things done the way they are, I have no idea.

40 minutes ago, David Heffernan said:

I can't sign up for the beta because I don't have a subscription. Does partaking in the beta have much impact? 

Yes. It can have huge impact.

If some feature is scheduled for improvement, that means it will be looked at, but the more work feature needs, the more chances is that some parts will not be done right or some accidental bugs might creep in. 

Also public API can change only at major releases. If some API gets designed in a wrong way, it must stay like that until the next major release. 

I personally don't use floating point much, so I don't know how broad are required changes to improve performance and whether this will be some compiler tweaking or changes involved require significant API changes, too. In any way, it is always hard to predict different use cases and most important pain points, so having feedback from people that are experts in the area and that know what they need is crucial.

Subscription is needed for regular users, but there are also exceptions for cases where people are invited because they can provide valuable feedback in certain areas.

35 minutes ago, David Heffernan said:

If you really want to reduce the size of what you deploy then use runtime packages instead of DLLs and you won't duplicate RTL/VCL classes. Likely that would save you far more than you would save by stripping RTTI in the RTL/VCL code that you link, were it even possible for you to do that.

Runtime packages are not always ideal solution, so you have to weight your options.

Just for fun... my installer application written in D7 is 1.1MB, same code recompiled in XE4 is 3.5MB, 10.4 4.7 MB.  Windows 32bit

Of course, no DLLs here... but even if you can argue that there are substantial differences and needed improvements between D7 and XE4, between XE4 and 10.4 when it comes to VCL new features were sparse, and certainly not large enough to justify size difference. I mean the whole darn setup in D7 with several forms fits into that difference (XE4-10.4)