27 minutes ago, Kryvich said:

Scoped enumerations.

If you put in a record or a class, the type name will be a namespace.
To refer to a constant you use the dot notation

Record_type.Constant_name

13 minutes ago, Kryvich said:

When I need to add a new member to the enum, I ... just insert the new member into the enum. If I had a set of named constants, I would have to renumber all the constants starting from the insertion position.
So what about extensibility?

Or you can add a new value to the end of the list.

For example, we take Google protocol buffer, where tags are used to identify fields.
Do not touch the old field identifiers unless you want to break anything.

21 minutes ago, Kryvich said:

1. You cannot extend a set of constants if they are defined in a record type in a third party library. There is no record inheritance.

2. An enumeration give us more type safety than a set of integer constants.

3. Scoped enumerations.

4. I like prefixes. But if not, see 3.

 

I use enums with 100 or more members. If instead of them I used named integer constants, adding and subtracting them, there would be many hard-to-find errors.

Why do you need to inherit records?

If I know the value of the constant, I don't need anything else.

Also, no one forbids transferring the api header files.

Take tlb for example, constants are beautifully declared in it.
Within the module. Can be within a record or class.

When I use Windows dll I can perfectly use the constants from the header file.

1 hour ago, Stefan Glienke said:

Exactly what I have been doing recently during my refactoring - fun fact: sometimes you even want to rewrite that to avoid the conditional forward jump taken.

I do not understand the thought, perhaps I am already tired and my head is working so-so.

24 minutes ago, Stefan Glienke said:

If that would compile - and even if it would that would not prevent that set to be assigned [4] to.

I corrected the syntax of the example

34 minutes ago, David Heffernan said:

Don't think so

 

https://en.cppreference.com/w/cpp/language/enum

As for C ++, I was in a hurry. Although when porting, I very often port as a list of constants.
Wonderful sources for oberon do without enumerated types.

In Java, an enumerated type can be inherited and extended.

52 minutes ago, Stefan Glienke said:

And I thought enums and the possibility to build sets of enums is one of those unique features in Delphi that many other languages such as C++ don't have hence you have to and/or with damn bitmasks....

type
  TMyType = record
  const
    &Template = 0;
    &Static = 1;
    Header = 2;
    Custom = 3;
  type
    TMySetType = set of &Template..Custom;
  end;

For example, we took the direction of the cardinal points
North South West East.
Then he finds out that this is not enough, we decided to add. The question is where to add?
Now what to do with that code that uses only 4 directions?

52 minutes ago, Mike Torrettinni said:

Interesting. Are you really using this construct instead of enums, in your real code?

Of course I do.

All programmers, for example, in java-script are forced to use the same approach.

The original K&R C dialect did not have enumeration types, however they were added in the ANSI C standard.

The enumerated type has been removed in oberon.

When Niklaus Emil Wirth came to us in Tomsk.
We asked him a question, the reason for this was done?
We got an answer about his shortcomings.

1. Very often, an enumerated type cannot be extended,
This may be due to the fact that it is from a third party library
and the domain of its values is already sealed.

2. A set of constants can successfully perform the same role.

3. One and the same concept is often used in different classifications, which leads to the growth of similar names with distinguishing prefixes.

4. The prefix is essentially the same vicious Hungarian notation LPStrZ_to_hell

For convenient use, a related set of constants can be placed in a module or record.
The record will act as a namespace.
Here we will not be limited to expanding the list of values.
Also, we can always add related methods for working with these values.

THTMLType = record
public const
  Template = 0;
  Static = 1;
  Header = 2;
  Custom = 3;
end;

Criticism

The enumeration type is traditional in advanced programming languages, is widely used and is often taken for granted. However, this type is also not without criticism from programming theorists and practitioners. So, when developing the Oberon programming language, enumerated types were included in the list of features that were removed from the language. Niklaus Wirth, the language developer, gave the following reasons:

“In an increasing number of programs, the ill-considered use of enumerations ... leads to a population explosion among types, which, in turn, leads not to clarity of programs, but to verbosity” [1]; 

when an enumeration type is exported by a module (that is, it becomes part of the interface), the general rule is violated - the type export command simultaneously exports all its elements, while for all other types, the type export hides its internal structure;
from the point of view of ensuring readability of programs, nothing prevents you from using just a group of jointly defined named constants instead of an enumerated type, especially when there are language mechanisms such as modules or classes.

On the other hand, for example, in Java, which initially did not contain an enumerated type, this type was subsequently introduced for reasons of not only convenience, but also reliability: the problem of using named constant groups instead of enumerations is that there is no compiler control as to the uniqueness of values constants, and the possibility of random assignment to variables of values that do not correspond to any of these constants.

Quote

It would seem that we should get the same behavior when using the Inrange function from System.SysUtils,

but I didn't like the compiler code at all.

Oz.SGL.Test.pas.465: Assert(Inrange(index, 0, fHigh));
0066E41A 85DB             test ebx,ebx
0066E41C 0F9DC0           setnl al
0066E41F 3BFB             cmp edi,ebx
0066E421 0F9DC2           setnl dl
0066E424 84D0             test al,dl
0066E426 7514             jnz $0066e43c
0066E428 B9D1010000       mov ecx,$000001d1
0066E42D BA98E46600       mov edx,$0066e498
0066E432 B828E56600       mov eax,$0066e528
0066E437 E8A4C9D9FF       call @Assert
Oz.SGL.Test.pas.466: Assert(Inrange(Cardinal(index), 0, Cardinal(fHigh)));
0066E43C 8BC3             mov eax,ebx
0066E43E 33D2             xor edx,edx
0066E440 83FA00           cmp edx,$00
0066E443 7508             jnz $0066e44d
0066E445 83F800           cmp eax,$00
0066E448 0F93C1           setnb cl
0066E44B EB03             jmp $0066e450
0066E44D 0F9DC1           setnl cl
0066E450 8BC3             mov eax,ebx
0066E452 33D2             xor edx,edx
0066E454 52               push edx
0066E455 50               push eax
0066E456 8BC7             mov eax,edi
0066E458 33D2             xor edx,edx
0066E45A 3B542404         cmp edx,[esp+$04]
0066E45E 7508             jnz $0066e468
0066E460 3B0424           cmp eax,[esp]
0066E463 0F93C0           setnb al
0066E466 EB03             jmp $0066e46b
0066E468 0F9DC0           setnl al
0066E46B 83C408           add esp,$08
0066E46E 84C1             test cl,al
0066E470 7514             jnz $0066e486
0066E472 B9D2010000       mov ecx,$000001d2
0066E477 BA98E46600       mov edx,$0066e498
0066E47C B828E56600       mov eax,$0066e528
0066E481 E85AC9D9FF       call @Assert

 

I think the compiler developers have work to do.

In my opinion, of all the pascal compilers I have worked with, the best code generated was Pascal - 2 Oregon software for pdp-11.

Amazing register optimization. Carrying out expressions for cycles and much more.
This compiler generated better code than the C compiler.

How to check the zero-based index.

How many comparison operations it takes to check the occurrence of a value from 0 .. Count - 1. index, fCount: Integer

It would seem that it could be easier and here we write the code:

Assert((index >= 0) and (index < fCount), 'Array bounds error')

This is exactly the message I saw when rangecheck was on. on my very first and favorite compiler for Pascal - 1 for PDP-11. First love does not rust.

This is equivalent to two checks and if you open the CPU window we can see that this is indeed the case.

We use commands that take into account the upper sign bit.

Oz.SGL.Test.pas.459: fCount := 5;
0066E55D C745F405000000 mov [ebp-$0c],$00000005

Oz.SGL.Test.pas.460: index := -2;
0066E564 C745F8FEFFFFFFF mov [ebp-$08],$fffffffe

Oz.SGL.Test.pas.461: Assert((index >= 0) and (index < fCount), 'Array bounds error');
0066E56B 837DF800 cmp dword ptr [ebp-$08],$00
0066E56F 7C08 jl $0066e579
0066E571 8B45F8 mov eax,[ebp-$08]
0066E574 3B45F4 cmp eax,[ebp-$0c]
0066E577 7C14 jl $0066e58d

0066E579 B9CD010000 mov ecx,$000001cd
0066E57E BABCE56600 mov edx,$0066e5bc
0066E583 B818E66600 mov eax,$0066e618
0066E588 E853C8D9FF call @Assert

Some microprocessors have special commands to check the entry of the index, who can perform this check with a single command.

But if you use the command of comparing unsigned numbers we can simplify the expression and write the following code

Assert(Cardinal(index) < Cardinal(fCount), 'Array bounds error');

and then we can do the same check using a single comparison. This reduces the price of the check by exactly half.

Oz.SGL.Test.pas.462: Assert(Cardinal(index) < Cardinal(fCount), 'Array bounds error');
0066E58D 8B45F8 mov eax,[ebp-$08].
0066E590 3B45F4 cmp eax,[ebp-$0c]
0066E593 7214 jb $0066e5a9

0066E595 B9CE010000 mov ecx,$000001ce
0066E59A BABCE56600 mov edx,$0066e5bc
0066E59F B818E66600 mov eax,$0066e618
0066E5A4 E837C8D9FF call @Assert

Usually when debugging I try to remember to enable rangecheck. But in the release version all checks are usually turned off.

That is, in training flights we fly with a parachute. But in the combat flight (in the release version) we leave this parachute at home.

Then hackers use it to crack through our programs. Do not forget to check at least the input buffer of your program.

On 10/11/2020 at 2:21 AM, Stefan Glienke said:

6 hours ago, Stefan Glienke said:

So instead of adressing those statements you bring up some completely irrelevant and wrong things about my library? Well... I am all for criticism if you find issues in its design or implementation but that was just a low blow. 😉

 

I'm sorry, I was wrong in this case.

SgList doesn't really support managed types.

The implementation of this collection was one of the very first.
I actually used it for data that does not contain managed types.

It will be necessary to fix this shortfall and do it in about the same way as in other collections, in which this problem should not be.

You will need to look into the code of each collection again and provide test support for the managed types.

10 hours ago, Fritzew said:

I'm using Spring4D all the time. Run the Test without debugger......
for me It is one of the best, correction, the best library for Delphi. 

I have no complaints about the library. I already wrote about this.
I looked at the library code. The spirit of the library is close to java.

I was offended by several phrases that I considered incorrect.

1. What I observed though is that they are no general purpose collection classes but obviously tailored for some specific needs - you cannot use TsgList <T> for any type as it is very limited as what it handles (only non managed types).

I'm sorry, I was wrong in this case.

SgList doesn't really support managed types.

The implementation of this collection was one of the very first.
I actually used it for data that does not contain managed types.

It will be necessary to fix this shortfall and do it in about the same way as in other collections, in which this problem should not be.

You will need to look into the code of each collection again and provide test support for the managed types.

2. I also did not do a in depth performance comparison but just adding some Integers to a list was 3-4 times slower than in my latest Spring4D build (which is a faster than the RTL).

I looked at the test.
First, we specify the size of the list, and then we assign a scalar value to the allocated memory area.

In this case, a collection is used, which is essentially a clone of the standard collection, with the exception of the possibility to refer to a data element by a pointer and using a memory region.

I suggested changing the test a bit, bringing it closer to real conditions.

Added a level and we have indirection of access to the data item. In my opinion, an indirect base-index access method is obtained.
Instead of a base-index accessor.
In addition, I now have to check the correctness of the index.
 

procedure TMemSegment.CheckPointer (Ptr: Pointer; Size: Cardinal);
var
  lo, hi: NativeUInt;
begin
  lo: = NativeUInt (@Self) + sizeof (TMemSegment);
  hi: = NativeUInt (@Self) + HeapSize - Size;
  Check (InRange (NativeUInt (Ptr), lo, hi));
end;

I have been developing software for embedded microprocessors for six years and always took my parachute with me during a real flight.
This is me about checks when accessing an array.
Now, though, I have this option enabled at least during the development of the project.

This and some other checks can be left only for debugging, but I love to fly with a parachute and identify the problem at the point of origin.

The conversation is about the Spring4D project.
For simplicity of heart, I ran the test in the IDE.
program Spring.Tests;

3 hours ago, Mahdi Safsafi said:

There're a lot of place where an AV must be used/expected by tests to ensure a logic consistence.

 

I actually talked about "division by zero in the context of working with a library offering work with collections and IOC.

Why not check the parameter for zero and display a message in the place where it might appear?

Perhaps the error was in the test: "What will happen with our engine if we are dealing with a buggy UDF?"

Access Violation was not met!
I brought them alongside because they are hardware.

2 hours ago, David Heffernan said:

Ask yourself why the RTL raises more than a single class of exception. 

The Delphi Runtime Library is a collection of levels of abstraction.
RTL itself is implemented on top of the operating system API.

The operating system (already sometimes) runs using hardware and a processor.

Take an I / O library for example.
It is pertinent to see an exception like "Oh, I (i / o) cannot open the file".
Or "Oh, why am I (i / o) trying to read something beyond the end of the file."

When I see "division by zero" or "Access violation" it is generally a disaster - a tragedy.

This means that the problem did not occur at the library level, but at the hardware and processor levels.

For me, this is a signal that the memory is destroyed, or the code does not understand what.

I am an advocate of defensive programming. The library function must validate the input parameters.

On 10/11/2020 at 2:21 AM, Stefan Glienke said:

You know that unit testing also includes testing if exceptions are thrown properly, yes?

All of them are expected exceptions and you saw that all tests are green, yes?

Usually when I implement a library for it, I declare a separate type of exception.

When checking when throwing an exception, I try to use only this type of exception.

While running the tests, I added a bunch of "ignore on execution" exceptions.
The most incredible exceptions were thrown.

Especially, I was very confused by the "division by zero" exception.

I decided that it was not worth continuing further.

I really decided that the installation of the library was done incorrectly.

10 hours ago, Stefan Glienke said:

As I wrote that was not an in depth benchmark but just to get a rough idea and its apples and oranges anway, ymmv.

Your library code is functionally complete.
Written well and reliably, you can see a strong love for interfaces and OOP.
I also had a love stage with COM objects and interfaces.

I am now developing the protobuf-delphi project and was looking for a suitable collection implementation.
It will be possible to try to make code generation for your library.

The integer test will not be able to show the advantage of using memory regions.

I have an implementation of two buffers https://sourceforge.net/projects/protobuf-delphi/, which I used on the application server for folding 
there the results sent to the client. 
A segmented buffer is a connected chain of memory regions and when there is not enough memory, it asks for another piece of memory.

The second buffer uses a continuous piece of memory and has a different allocation strategy.
When there is insufficient memory, it asks for a larger piece of memory (usually ReallocMem),

copies the current data (Assign) and then deletes the old block (FreeMem).

Now we have to consider a typical memory manager implementation. 

And at my age, I have seen and studied the device dozens of their implementations
(starting with the implementation of Donald Knuth on his UM processor...).

Working with a heap in Delphi uses this structure

  TMemoryManager = record
    GetMem: function(Size: Integer): Pointer;
    FreeMem: function(P: Pointer): Integer;
    ReallocMem: function(P: Pointer; Size: Integer): Pointer;
  end;

  
At first, the memory in the heap is not defragmented and we have one solid piece of memory.
The mechanism of allocation at this stage is sequential.
A correct implementation of ReallocMem checks the input parameter of a variable and

if it is on the free memory pile boundary, it will simply reduce the amount of free memory and move the free memory pile boundary.

In this case there is no need to copy data and delete the old block.

You have TArray<Integer> used "overboard" your class. In other words, it's a dynamic array. 
In my case, it uses a continuous memory region.

If you are in a loop, you will increment a single collection 
we have one solid piece of memory and we test the dynamic array operation
in the best conditions imaginable.

So your test is in an ideal world.

In order to see the difference you need to get closer to the combat conditions of use.

Integer is not an object, not a string that is placed in a heap.

If it is possible to use records instead of an object I use a record.
I don't like to use controllable types (string, interface, reference to proc). for the record fields in a heavily loaded server.
Because Delphi's finalization operation is not very cheap, it uses rtti and will scan the record fields in order to take account of the referenced fields with a managed type.

Why don't we improve the test and bring it closer to the real world?

It is possible to simulate the real working conditions of the program.
First, we bring the heap to a defragmented state.

Defragmentation occurs after multiple memory allocation and memory return.

Then we interrupt the work of several collections including a linked list, a hash table and a tree. Memory freeing should be used as well.

BSP Test

Take for example the BSP algorithm and implement it 
1. on your collections using objects.
2. on SGL collections using records placed in memory regions.

Multithreaded application server

Simulate typical server operation state.
There are different types of requests from different clients.
We accept them, service and return the answer.
It can be json, xml or binary data format such as Google protocol buffer.
 

I always analyze the types of data being processed and the operations that will be performed on them.

It is possible to combine the merits of both array and linked list. To do this, it is enough to use data structures using memory regions.

If you arrange the list items in a memory region, the data will certainly be located side by side and the percentage of cache hits will be maximum.

Using structures using memory regions can also make the job of freeing memory easier.

I would also advise to take a closer look at the storage format of the buffer protocol. Which is equally good for storing any data. In addition, it shakes data quite well.
at the expense of default values and does not store empty values.

All messages in this format start with a length value. You can store pointers to their beginning in an array.
You can pass this set of bytes to the reader of an object that reads data. The writer of an object can write its state to a buffer.

If this is processing data from client applications, I would do something like a processing context associated with a specific client and perhaps a message builder based on a deterministic finite automaton.

https://github.com/marat1961/protobuf-delphi

https://github.com/marat1961/Oz-SGL

1 hour ago, FPiette said:

This article is about random insertion and random deletion, I guess they take into account the time required to locate the record in the list.  If random means generating a random index number, then it is the worst case possible for linked list and best case for array (direct access). If that is a main access pattern, then a linked list is NOT the correct data structure. Insertion/deletion in a linked list is only quick provided the insertion/deletion point is already available. Measuring the performance for a search separately from performance for insert/delete itself is what has to be done to truly compare linked-list and arrays or any other data structure such as a dictionary or hash table.

 

A linked list is excellent for a FIFO QUEUE or LIFO QUEUE (STACK). And that is the problem in the case of this message thread: communication which require a FIFO queue between receiving process and computing process. That is why I proposed a linked-list.

 

 

François, I was very glad to see you. I want to say a big thank you for your ICS components. In my server, I used them.

On 10/9/2020 at 6:38 PM, Stefan Glienke said:

@Edwin Yip

I also did not do a in depth performance comparison but just adding some Integers to a list was 3-4 times slower than in my latest Spring4D build (which is a faster than the RTL).

On 10/9/2020 at 6:38 PM, Stefan Glienke said:

@Edwin Yip The article you linked to is my blog, not Erics 😉

Yes, it also suffers from that however the types in that library are way smaller and have only limited functionality of just storing stuff, no rich IEnumerable API such as Spring4D. That makes the binary overhead very very small or even non existing.

What I observed though is that they are no general purpose collection classes but obviously tailored for some specific needs - you cannot use TsgList<T> for any type as it is very limited as what it handles (only non managed types).

 

I also did not do a in depth performance comparison but just adding some Integers to a list was 3-4 times slower than in my latest Spring4D build (which is a faster than the RTL).

 

If you watch the video by Herb Sutter I linked in the other thread I even wonder why someone coming from C++ would need to create some list type as he could just use TArray<T> because he should have used Vector<T> in C++.

21 hours ago, Mahdi Safsafi said:

 For example, the library has a sealed architecture ! Being based on records, it means that you can't roll easily your own collection based on the existing one (at least in the way we know all) just because the current status of the language does not permit record inheritance. In the other hand, RTL/Spring4D are more flexible in this area. 

Another thing, a quick look at the Heap manager, it appears that its based on the ordinal memory manager, in other word: a memory manager on the top of another memory manager. This technically may cause all sort of memory troubles.

 

I rarely inherit from collections, for some reason I rarely have a desire to add something.
If you want to extend functionality without adding fields, use a helper.

I am a fan of domain-driven design.
Using inheritance for collections does not seem to be a very good idea to me.
I am very suspicious of thoughtless inheritance.
Most often, inheritance does not solve a problem, usually it creates them.

If it is necessary to reuse code, use the aggregation relation.

You can replace inheritance with aggregation and declare only the methods you need in the public section.

1. Place the structure you want to aggregate as a private field
2. Next we open only the necessary part of the interface by redefining it to public 
sections of required methods and properties. 
If you set the inline option, we will avoid additional costs. 
The Delphi compiler will not generate code for overridden methods. 
At the place where the method is called, there will be a direct call to the aggregated structure method.

Selecting abstraction levels is not a rarity.
FastMM replaces the memory manager.
Delphi manager works on top of the operating system manager.

The memory organization system is very well written here.

https://www.gunsmoker.ru/2009/01/blog-post.html

https://www.gunsmoker.ru/2011/04/windows.html

https://en.wikipedia.org/wiki/Region-based_memory_management

I use FastMM with fully enabled diagnostics, no problems with memory leaks.

The code of this library is used in real projects.
Practice is a criterion of truth.

On 10/9/2020 at 6:38 PM, Stefan Glienke said:

@Edwin Yip

Yes, it also suffers from that however the types in that library are way smaller and have only limited functionality of just storing stuff, no rich IEnumerable API such as Spring4D. That makes the binary overhead very very small or even non existing.

What I observed though is that they are no general purpose collection classes but obviously tailored for some specific needs - you cannot use TsgList<T> for any type as it is very limited as what it handles (only non managed types).

Why did you decide that the type of something is limited?
You want to use a record, an object or a string ...

This is not quite true, managed types are supported.

First, when type assignment occurs, all counters for managed types will be updated (Delphi string, interfaces, etc.).

Second, when deleting data, the cleanup method is passed, which will be used when deleting.

If it is a record with managed types, it can be a procedure inside which the Default (TMyRecord) operator contains.