Stefan Glienke

Please describe the specs for a real set type.

From my experience it never does that - if you have 3 different lines with function calls returning a string the compiler will generate 3 distinct hidden local variables. Even if it's guaranteed they will never be used both such as calling different string returning functions in both branches of an if or in a case. To understand what Peter showed before and how strings (and other managed type) results are handled look at this code: function Test: string; begin Result := Result + 'hello'; end; procedure Main; var s: string; begin s := Test; s := Test; s := Test; Writeln(s); end; var s: string; i: Integer; begin // case 1 Main; // case 2 s := Test; s := Test; s := Test; Writeln(s); // case 3 for i := 1 to 3 do s := Test; Writeln(s); end. 1. When you assign the string result to a local variable the compiler directly passes the local variable as that hidden result var parameter - hence you will see the output 'hellohellohello' 2. When you assign the string result to a global variable (or a field of an object) the compiler generates distinct hidden variables for each occurence of the call - three in this case - they are also initialized to zero like any explicit variable. It then assigns that to s. This is why you will see 'hello' 3. When doing multiple calls in a loop the compiler had only generated one hidden variable - of course which will be reused for each call - now again you will see 'hellohellohello'

Delphi does not do caller site checks such as Java or .NET do.

As a solo maintainer of an open-source library myself I can say this: if that person does not have a working dev environment installed anymore and does not have time to work on it - I personally consider that project being de facto dead. That is not an offense but there has to be taken action. Nobody can blame you for not working on a spare time project if you got no time or the mood to. But if that project is of fundamental importance - and I consider Indy as a crucial piece of the 3rd party ecosystem for Delphi - there needs to be something done about it. Not having an up-to-date networking library available (yes I know there are several others, but Indy always has been out of the box and freely available) is one of the many reasons that make Delphi unappealing for decision-makers.

Given the activity on Indy, I am tempted to call that project dead.

While it cannot squeeze the maximum it is far far away from performing anywhere near "well" for larger amounts - simply because a) the x86 implementation only moves 8 bytes at once in a loop using FILD and FISTP which is slower than an SSE loop using instructions that are available since like 2000. and b) because Win64 uses a rather terrible purepascal implementation which also at most moves 8 byte at a time.

That is why FastMM and probably other memory managers have diagnostic features such as setting freed memory to a certain pattern. If you enable that your ShowMessage will most likely blow up because Danger would be an invalid string pointer.

No, that is a const record - which is not really const but just a write protected variable. You know how to declare integer consts, do you?

The point that others already have expressed is that despite being interested in a topic as performance improvement so low level (as in instruction-level instead of algorithmic level) you seem to lack some important knowledge to do so such as assembly - it does not require as much as it does to write assembly code but to understand it in order to be able to look at the code in the debugger and see that some comparisons are apples and bananas. I did not even read through your code but simply placed a breakpoint into your IsIN function and noticed that it contained a function call to System.SetElem (that even was the first time I have ever seen that function being called so a TIL for me). Why was that the case? Because you are not using consts here but variables. Had you simply made consts for all those IDs the code would have almost as fast as the IsOR which does not suffer to extra function calls but from memory reads (not noticeable in the benchmark because its all in L1 cache already). On my CPU InOR is still a little faster than IsIN which is due to the fact how the compiler builds the in - you can see that for yourself in the disassembly and then look at instruction timings, read up on macro-operation fusion and data dependency For reference, this is the assembly for the two functions when using consts Project1.dpr.40: Result := aID in [xControlsRec.ButtonID, xControlsRec.FormID, xControlsRec.ListBoxID, xControlsRec.TabControlID, xControlsRec.ComboBoxID]; 004CEE7C 83E802 sub eax,$02 004CEE7F 7417 jz $004cee98 004CEE81 83E802 sub eax,$02 004CEE84 7412 jz $004cee98 004CEE86 83E802 sub eax,$02 004CEE89 740D jz $004cee98 004CEE8B 83E802 sub eax,$02 004CEE8E 7408 jz $004cee98 004CEE90 83E802 sub eax,$02 004CEE93 7403 jz $004cee98 004CEE95 33C0 xor eax,eax 004CEE97 C3 ret 004CEE98 B001 mov al,$01 Project1.dpr.41: end; 004CEE9A C3 ret 004CEE9B 90 nop Project1.dpr.45: Result := (aID = xControlsRec.ButtonID) or (aID = xControlsRec.FormID) or (aID = xControlsRec.ListBoxID) or (aID = xControlsRec.TabControlID) or (aID = xControlsRec.ComboBoxID); 004CEE9C 83F802 cmp eax,$02 004CEE9F 7417 jz $004ceeb8 004CEEA1 83F804 cmp eax,$04 004CEEA4 7412 jz $004ceeb8 004CEEA6 83F806 cmp eax,$06 004CEEA9 740D jz $004ceeb8 004CEEAB 83F808 cmp eax,$08 004CEEAE 7408 jz $004ceeb8 004CEEB0 83F80A cmp eax,$0a 004CEEB3 7403 jz $004ceeb8 004CEEB5 33C0 xor eax,eax 004CEEB7 C3 ret 004CEEB8 B001 mov al,$01 Project1.dpr.46: end; 004CEEBA C3 ret Depending on the number of IDs you have it might be worth using power of two and bitmasks or an enum directly because that would only require one cmp/test making the function twice as fast and perfect for inlining which would then also eliminate the function call overhead at all.

I am usually happy to help people improve in that subject but seeing that you simply ignored half the advice given to you before in countless similar threads I consider this a waste of time.

Does this help you? https://stackoverflow.com/questions/43023217/delphi-linux-ld-linux-exe-error-cannot-find-lgcc-s

Little explanation of how the RTL dictionary works: - as Capacity, it always uses n^2 - that enables a fast calculation from the hashcode to the bucket index by simply bitmasking it (which makes it important to have a good hash function that also has good distribution in the lower bits of the hashcode - since 11 the RTL uses FNV1a which is quite good). - now how to deal with collisions (and I mean those in the buckets, not the hashcode itself because for 300 entries there would be a capacity of 512). When distributing 300 items across 512 slots it is very likely that you get a few collisions. The RTL uses a very simple so-called linear probing for that - it keeps looking at the next bucket until it finds a free one. While this provides good locality it naturally leads to primary clustering which you can see by the Collision count. This number tells the number of items that don't sit in the slot where their hashcode would have put them. As of 2.0 Spring uses a different hashtable implementation which is using a clever algorithm from the Python dictionary which mitigates this quite nicely. FWIW especially if you have a fixed amount of strings such a keywords it might be better to not use a hashtable because of the hash function overhead but rather some handcrafted lookup table. Since keywords most likely always start with a letter you can do something like DWS does: see https://github.com/EricGrange/DWScript/blob/master/Source/dwsTokenizer.pas#L798 just with the first letter you get the very tiny array of the keywords starting with that letter and then you do a linear search - which beats a hashtable any time.

I am very sure that record finalization is way faster than heap allocations caused by SetString - if after profiling it still has noticeable impact then one could extract the string keeping from the record into some other structure - the point is they need to be kept alive as long as they are inside the dictionary - how does not matter.

This is how I would solve this - using the record type to preserve the original string and a dedicated equality comparer. Using Spring4d collections here, but would be similar when using the RTL. {$APPTYPE CONSOLE} uses Spring.Collections, Spring.Comparers, System.SysUtils, System.StrUtils, System.Generics.Defaults; type TTokenKey = record s: string; p: PChar; len: Integer; constructor Create(const s: string); overload; constructor Create(p: PChar; len: Integer); overload; end; TTokenKeyComparer = class(TInterfacedObject, IEqualityComparer<TTokenKey>) public function Equals(const left, right: TTokenKey): Boolean; reintroduce; function GetHashCode(const value: TTokenKey): Integer; reintroduce; end; { TTokenKey } constructor TTokenKey.Create(const s: string); begin self.s := s; p := Pointer(s); len := Length(s); end; constructor TTokenKey.Create(p: PChar; len: Integer); begin self.p := p; self.len := len; end; { TTokenKeyComparer } function TTokenKeyComparer.Equals(const left, right: TTokenKey): Boolean; begin Result := (left.len = right.len) and CompareMem(left.p, right.p, left.len); end; function TTokenKeyComparer.GetHashCode(const value: TTokenKey): Integer; begin Result := DefaultHashFunction(value.p^, value.len * SizeOf(Char)); end; const sentence = 'the quick brown fox jumps over the lazy dog'; begin var tokens := TCollections.CreateSet<TTokenKey>(TTokenKeyComparer.Create); for var s in SplitString(sentence, ' ') do tokens.Add(TTokenKey.Create(s)); Writeln(tokens.Count); Writeln(tokens.Contains(TTokenKey.Create(@sentence[5], 5))); // 'quick' ->true Writeln(tokens.Contains(TTokenKey.Create(@sentence[6], 4))); // 'uick' -> false Readln; end.

Yeah, they could at least have made Fishfacts, amirite?!

I assume you are not building a debugger because it says:

Doing their best to avoid an even bigger disaster?

Nice idea - however I dislike using some special kind of comment format for that including the string for some kind of parsing. As so often .NET solves this pretty nicely. I would even claim this could be already be done by a third party and requires no work from Embarcadero - no claims on performance. The debugger afaik is able to read RTTI from an inspected object and thus the attribute. It can then execute the proxy code which generates the output for the debugger. ToolsAPI has interfaces for that.

If you look at https://www.w3.org/TR/2010/REC-xquery-20101214/#EBNFNotation and https://www.w3.org/TR/xml/#sec-notation you will see that it uses some in between format of BNF and EBNF - using the ::= from BNF but not the angle brackets.

Could be DirectWrite related if you have a rather poor onboard GPU. Check CPU and GPU usage when it's lagging.

Took me less than 5 minutes to find this and another 5 minutes to try it out with chrome dev tools.

The early 2000s want their website design back

When dragging the Grep window (and also other dockable windows from GExperts) they show as too large - I would say by exactly the scaling amount (the screenshot is from a system with 175%)

Yes, otherwise I would have reported that to Embarcadero

I still have to prepare the blog article about the beta. Vincent already summarized what 2.0 provides. Testing early is always appreciated as it helps find any possible issues. There might be some minor breaking changes which I will outline in the article. Still no response from google though after sending feedback about the blocked forum. Their reaction times are atrocious.