Leaderboard

Actually the defect is in your test code. Set the capacity when you create the list. Just as you do with the array. Obviously you need to compare like with like.

Please google "virtual memory".

The problem is, that RTTI doesn't provide the T in TSomeGenericClass<T> in the first place. This works for arrays because arrays have information about the element type in their RTTI. Of course there are workarounds to find out T for an TObjectList<T>, but looking at any code doing that: how would you declare some class type derived from TObjectList<T> without providing T? BTW, the helpers described in my blog posts do save a ton of scaffolding code. I was able to reduce a unit containing all the serialized classes from over 2000 lines downto less than 1200 lines. If only I were in charge of designing the relevant Delphi classes, you could bet on some significant improvements

David, sorry but that's not helpful. This is not some "library" and there is no "consumer". The code I'm talking about is internal and scientific. It does stuff like calculating the 3-d trajectory of curved oil wells, calculates dynamic pressure losses caused by drilling fluid being pumped downhole through a complex drillstring and then up again through the annular space between the drillstring and the wellbore (the flow can behave laminar in some segments, turbulent in others) etc. and tons of other things. There is no way to make such code self-explanatory, it requires knowledge of physics, fluid dynamics, geometry and engineering. So I want to document this code in great detail to explain what's going on inside, with hyperlinks to scientific literature, Wikipedia etc. Because when I retire in ten years, the code must be maintained and enhanced with new fluid dynamic models by my successor and he'd better know what he's doing. Since the comments are going to be as large as the code itself, a side-by side solution would be ideal for me. Using Xmldoc or Javadoc/pasdoc only interrupts the flow of things, making it more difficult to read.

What happens, if you add vDataList.Capacity := vMax; before filling vDataList. vDataList is using array internally, but it is constantly grown in your example. It might be the cause of those issues. Also, you really should run those test so, that one test is only one run on that execution of test program. Otherwise testing array might make use of memory that was already allocated during list testing (Delphi memory manager does not return memory to windows, when it is freed.

Two names for the same thing in my eyes. But if the official title is page file then that's what we should say, I agree.

No. You still don't understand the difference between physical and virtual memory.

Please google "64-bit"

What you describe could avoid memory reallocations and moving items if reallocation could not happen inplace but has a lot of other performance considerations - see https://softwareengineering.stackexchange.com/questions/267870/are-noncontiguous-arrays-performant

No it doesn't. It's just a wrapper for an array. Effectively it's the exact same data type but with a lot of helper methods to let you write higher level code. And an allocation strategy that overallocates to avoid performing heap allocation every time you add an item. I'm not sure that there are many collection libraries that implement array like collections as anything other than a contiguous array. I know my own personal code has some classes that do this, to avoid address space fragmentation. Less of a problem now that we have 64 bit.

I agree. But others before me pointed that already out 🙂 I was trying to explain (not justify) the differences between the two samples given, the one with 17% more memory and the second with 68% more. You can explain the +17% with the missing SetLength, but not the +68% more memory. More knowledge doesn't hurt, and if someone uses the task manager to compare memory usage, I suppose it is nice to know about the memory manager itself, isn't it?

Like others said, TList has a capacity growing strategy. The second, and more important thing is, that the memory manager doesn't release the memory back to windows. So what you might getting is this: when you add items to the list, and it tries to resize, it might happen, that there is not sufficient memory in the current place to just in place resize. so the memory manager copies the whole array to a new location, that can hold the size of the new array. but the old memory space is still kept reserved by the memory manager. So when you use the task manager, windows sees both. Andd BTW, the memory manager doesn't request the exact size either. Like TList, it has a growth strategy, too.

TList grows its internal array by some factor (1.5 IIRC). So if you use just Add, there are big chances that list will have free space at the end of test

For local storage that won't have any chance to become client/server - SQLite. Otherwise FB embedded with elementary transition to FB C/S

While anything is possible, I don't think there is too much ARC only code around. For anything TComponent based ARC was never working in the first place (you had to use DisposeOf) so most of UI code should work as-is. Since manual memory management is what people are used to, migration path should be relatively easy, although it may be harder to find exact places where object needs releasing. I would follow constructors .Create - usually destruction is somewhere nearby - either in Destroy, or in the same method. Also, on mobile small leaks are not that critical, usually applications are killed anyway before such leaks cause substantial problems, and larger leaks are easily found just by looking at code. With Instruments and Android Studio you can easily spot problematic parts of application with large leaks. Not ideal situation... but what is done is done.

Unification is good. Going to cause headaches for anybody with ARC only code. Don't think that's going to be an easy migration path. Such a shame that this happened at all, but good for Emba for having the strength to change direction.

Currently this will only work in Delphi 10.3.3 Rio thanks to implementing a code change I suggested in QP together with a couple of failing test cases (unfortunately only people with access to that beta are able to see it). I just have none, but after all I am not the Delphi compiler engineer.

@Lars Fosdal Serializing Objects with TJson and Serializing Generic Object Lists with TJson