David Heffernan

Smells like a memory leak, or perhaps the loss of contiguous address space due to fragmentation from reallocation.

It's very plausible that you have a memory leak. Just because fastmm4 says you don't doesn't mean you don't. It's one thing returning memory when the program closes, another returning it in a timely fashion during execution.

Didn't you read that topic? The entire topic was based on misconception.

No. It's a suggestion that if you want an improvement to an open source project then one option is that you contribute it yourself. ALL-CAPSing the project maintainer is a strategy that in my opinion is sub optimal for your goals.

Why don't you have a go at fixing it, and contributing the fix?

Perhaps you know what this is. We don't. Try to imagine that we can't see your screen.

Isn't this something you set when you create the dictionary? Does the spring4d implementation offer more functionality?

What is your actual problem for which you need a solution? Because what you are asking doesn't actually make sense.

Flip side, you can have loads of available memory but no available contiguous address space. That's the advantage of a 64 bit process. It removes the limitations on address space.

I have no qualms about working with a 2G memory block under 64 bit. What exactly are you getting at? Can you explain in technical terms what problem you envisage?

Two names for the same thing in my eyes. But if the official title is page file then that's what we should say, I agree.

Then you'll fragment your address space and perhaps worse spend time doing expensive heap allocation. I don't know why you think SetLength(arr, Length(arr) + 1); arr[High(arr)] := item; is preferable to list.Add(item); And what about if you hold classes in a list, how do you manage lifetime? Lots of good reasons to use a good library of collection classes.

So when you populate an array, and its full but you need to add more items, how do you grow it?

How could lists possibly consume less than an array?!!! Let me ask you a question, do you have any code that does SetLength(arr, Length(arr) + 1)?

No. Wrong conclusion. That memory will be backed by the swap file. User may not have one. And even if they do perf is terrible. Seriously, you are way down the rabbit hole. Give up thinking about the advanced stuff and concentrate on the basics. Make sure you know how lists work.

Yes, it would definitely help you to learn the difference between virtual and physical memory.

Dude, we are talking about 64 bit code. You can't expect to make a 1GB array and do anything useful with it in a 32 bit process. Every chance that the virtual memory manager won't be able to find 1GB contiguous address space. This is virtual memory, and in 32 bit code usually the limits are from address space rather than physical memory.

Why? What is special about 1G?

Doubt usage was widespread

I already said. It's to avoid memory address space fragmentation when growing an array, even with the TList growth strategy. Address space fragmentation was a big issue when we had no 64 bit compiler. This is way beyond where you currently are. You need to fully understand how TList works first.

{$POINTERMATH ON} function TBlockAllocatedList<T>.GetItem(Index: Integer): P; var BlockIndex, ItemIndex: Cardinal; begin Assert(InRange(Index, 0, Count-1)); DivMod(Index, FItemsPerBlock, BlockIndex, ItemIndex); Result := P(FBlocks[BlockIndex]) + ItemIndex; end; {$POINTERMATH OFF} That's the key function in my implementation.

64 bit makes that problem go away.

I'm talking about an array like structure (i.e. O(1) random access, contiguous indices) whose backing allocation is not contiguous. I'm not aware of many (if any) collections in widespread use that are like this. I introduced such classes in my codebase in the bad old 32 bit days when address space fragmentation was killing me.

No it doesn't. It's just a wrapper for an array. Effectively it's the exact same data type but with a lot of helper methods to let you write higher level code. And an allocation strategy that overallocates to avoid performing heap allocation every time you add an item. I'm not sure that there are many collection libraries that implement array like collections as anything other than a contiguous array. I know my own personal code has some classes that do this, to avoid address space fragmentation. Less of a problem now that we have 64 bit.

How could it possibly know how many items were going to be added? In reality you often don't know this. I would not be surprised if your array code has lots of lines like SetLength(arr, Length(arr) + 1) which are hideously inefficient.