memory; The function of EmptyWorkingSet

[DelphiUdIT 176]

In Windows you can use the function " EmptyWorkingSet(GetCurrentProcess) " (from WinAPI.PsAPI) to release all memory to OS. I use it on my application when the memory used is over 50% of all system memory (this help also to reduce the effects of fragmentation).

 

My applications use lot of images, near 2000 raw buffers / second (form some KB to some MB) and fragmentation is high. Every 4 / 5 days I look (from logs) that the application use that function. Normally the application stay on near one year, actively working 10 hours per day.

 

You should look for equivalent function in Linux (I'm quite sure that should exists).

Bye

 

 

 

[Lars Fosdal 1792]

A workaround could be to explicitly allocate system memory outside of the regular memory manager, but that would require using specific methods for each platform.

1 hour ago, DelphiUdIT said:

EmptyWorkingSet(GetCurrentProcess)

How does that work with memory allocations in FastMM?

[Lajos Juhász 293]

You should read more about memory managers. The memory manager will not always return the allocated memory to OS. It's not only Delphi but every single other language works that way. You can google it easily.

[DelphiUdIT 176]

5 hours ago, Lars Fosdal said:

How does that work with memory allocations in FastMM?

I don't know how "working set"  interact with memory (page, virtual, etc..) and standard FastMM, but the function is really working freeing memory not released. I used FastMM5 for test and not side effects will be notice. I tested also EmptyWorkingSet in leak condition and in this case the effects are near to NULL (only little parts of memory freed).

Some links: https://learn.microsoft.com/en-us/windows/win32/memory/working-set

                  https://learn.microsoft.com/en-us/windows/win32/psapi/working-set-information

                  https://learn.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-emptyworkingset

 

Edited May 1 by DelphiUdIT

[Anders Melander 1783]

It would be so sweet if people voicing opinions about how [insert topic here] should work actually understood the topic.

[David Heffernan 2345]

EmptyWorkingSet removes pages from the working set. Doesn't this mean your program won't be able to take advantage of memory caching? 

[DelphiUdIT 176]

28 minutes ago, David Heffernan said:

EmptyWorkingSet removes pages from the working set. Doesn't this mean your program won't be able to take advantage of memory caching? 

Yes, you're right. There is a little latency (in my application measured in 10 milliseconds near) when the program start again to works and use (allocate) memory.
But whit this, I can maintain the memory free, especially when a lot of gigabytes are used.

 

The last post is the example that also in Windows there is a memory that was not freed, like in Linux and in others OS probably, and using that API you can free memory (cache, but not only) if you need.

And I think the should be a function in Linux too that can do this (although I don't know in depth the memory management system in Linux, so I might be talking nonsense).

Edited May 1 by DelphiUdIT

[Anders Melander 1783]

You are.

 

The working set are the pages of a process' virtual memory that resides in physical memory. They are there because the process has a need for them to be there (e.g. it has referenced an address in virtual memory causing a working set page to be mapped to that address).

If something else in the system has a need for physical memory, and it's all in use (which, by design, is normally is - because why not) then the least recently used pages will be paged out and eventually written to the page file, so the physical page can be mapped to the other process' working set.

The above is just a simplification of the virtual memory management but the point is that, unless what you are doing is really extreme, then you don't have to think about it; It just magically works.

 

I know everybody has to go through the phase of thinking that they can outsmart the OS virtual memory management by messing with the working set but you really should leave it alone. The OS virtual memory system was designed 50 years ago by people who actually knew what they were doing.

 

You might very well be having memory issues but look to the memory manager instead. The working set isn't the problem.

[DelphiUdIT 176]

@Anders Melander

I'm not completely in line with you.
It's true that memory management is automatic and normally you don't have to worry about it, but have you ever tried to observe the performance of an application with more or less available memory (I'm talking about Windows)?
I did in the past some tests using more or less memory, both "privately" allocated and used in cache, and performance drops in proportion to availability.
Testing up to 40% of occupied memory there are no problems whatsoever but from 50% and above there are performance losses of 5% on average, from 75% I have detected performance losses of up to 10%.
All analyzes were done using the exact same dataset (specifically images) and occupancy was simulated with both real occupancy and cached occupancy (approximately estimated), measuring average processing times. The memory occupation was carried out both inside the application and externally, and the two situations gave comparable results.
There is no virtual disk memory in my systems, and no file operations were performed during testing.
I use PC with 32GB memory and peak usage of 24GB is normal. The processor is normally used at around 75% and the CPU temperature varies between 75 and 85 degrees.
I can assure you that without the use of the EmptyWorkingSet I would have to slow down the industrial line at peak times, obviously automatically and in real time (which is what I did in the past).

[David Heffernan 2345]

10 hours ago, DelphiUdIT said:

Yes, you're right. There is a little latency (in my application measured in 10 milliseconds near) when the program start again to works and use (allocate) memory.
But whit this, I can maintain the memory free, especially when a lot of gigabytes are used.

This just makes your program slower though. Forcing the pages out of memory and onto disk before other programs need that memory takes time and can only lead to performance reduction. 

 

You may as well open the computer up and remove some memory sticks! 

[DelphiUdIT 176]

1 hour ago, David Heffernan said:

This just makes your program slower though. Forcing the pages out of memory and onto disk before other programs need that memory takes time and can only lead to performance reduction. 

 

You may as well open the computer up and remove some memory sticks! 

Like I told I don't use virtual memory on disk (it's disabled). And always like I told that function is called one time every 4 o 5 days ... in the PC run's only my software (and of course normal Windows services) and it's for this that cache memory become very high.

I call the function when a memory occupation over 50% is detected and when the machine has the first functional stop. After that (always when the machine in in the stopped mode) a simulated run (one shot) is done to resolve the latency.

 

All that is about 100 ms. (include one simulated round of application).

The EmptyWorkSet doesn't discharge the memory pages onto disk, he release the memory to OS (from Application workspace to OS).

[David Heffernan 2345]

So why would spending the time to do this be better than letting the system do it on demand?

[Anders Melander 1783]

5 hours ago, DelphiUdIT said:

The EmptyWorkSet doesn't discharge the memory pages onto disk, he release the memory to OS

You still haven't understood how this works.

 

The memory in your working set isn't "owned" by the process; It's owned by Windows and the process only has it on loan. If Windows needs the memory for something else (e.g. another process) it will take it back without any action needed (or possible) from your process.

 

The working set pages that you are trimming are unused by the process but was assigned to it at some point because it needed them. The reason that the OS hasn't removed them from your working set by itself is that there has been no need for it; Nobody else has made a memory request that couldn't be satisfied elsewhere. The whole idea behind the working set model is to allow the system to grow and shrink the working sets based on demand.

 

Why not spend 30 minutes to read up on how it works. Just Google windows working set.

 

Or simply search for EmptyWorkSet and see if you can find a single person who thinks it's a good solution.

[DelphiUdIT 176]

2 hours ago, Anders Melander said:

You still haven't understood how this works.

 

The memory in your working set isn't "owned" by the process; It's owned by Windows and the process only has it on loan. If Windows needs the memory for something else (e.g. another process) it will take it back without any action needed (or possible) from your process.

 

The working set pages that you are trimming are unused by the process but was assigned to it at some point because it needed them. The reason that the OS hasn't removed them from your working set by itself is that there has been no need for it; Nobody else has made a memory request that couldn't be satisfied elsewhere. The whole idea behind the working set model is to allow the system to grow and shrink the working sets based on demand.

 

Why not spend 30 minutes to read up on how it works. Just Google windows working set.

 

Or simply search for EmptyWorkSet and see if you can find a single person who thinks it's a good solution.

 

4 hours ago, David Heffernan said:

So why would spending the time to do this be better than letting the system do it on demand?

I understood like it works, and I described what happens ... but you don't have catch the problem and why I use it.

I use it because when the memory "in use", "locked", "booked", "cached", or like you want to describe it, is too high there is performance degradation.

 

(to respond to @David, i check normally the memory status, like temperature and others, and launch the function is two lines, of code: "if ... then Empty...." not so much time or energy spent)

 

Why there is that degradation I don't know, but for sure is that when the memory .... is to high, using EmptyWorkingSet is useful.

And this every 4 or 5 days ... I don't use it every single machine cycle.

 

But if EmptyWorkingSet is so "objectless", why exists ? It may be that whoever created the automatic memory management also foreseen that perhaps this mechanism may not be infallible, and has given a tool to force "the hand".

 

But all that (especially how I handle the lines) is not correlated with the original question of the topic, I'm OT so I suggest to start a new topic if needed.

Bye

 

P.S.: @Lars Fosdal  or some moderators, move my posts and correlated to other new topic ... except for one or two, others are nothing to do with the original topics. Thanks

Edited May 2 by DelphiUdIT

[Anders Melander 1783]

7 minutes ago, DelphiUdIT said:

It may be that whoever created the automatic memory management also foreseen that perhaps this mechanism may not be infallible, and has given a tool to force "the hand".

Nope.

It's there so that a process which

1. Knows that its access and allocation patterns will have caused the working set to grow.
2. Knows that it will no longer need that working set for the foreseeable future.
3. Has virtual memory backing store (i.e. page file).

can trim the working set in order to lessen the memory pressure on the rest of the system. But you don't have virtual memory so this don't apply to you.

 

By trimming your working set the only thing you have accomplished is to move pages to the working set cache. Since you don't have virtual memory, nobody else can use those pages anyway because their content can't be saved anywhere. So the only thing that can happen to them is that they will be paged back into your working set at some point when your process needs them. End result: Nothing but overhead.

[Brandon Staggs 278]

[Brandon Staggs 278]

12 minutes ago, Anders Melander said:

End result: Nothing but overhead.

Well, it also gives the false impression of lower memory usage if you're watching a number.

[David Heffernan 2345]

How are you measuring the performance degradation?

Edited May 3 by David Heffernan

[DelphiUdIT 176]

1 hour ago, David Heffernan said:

How are you measuring the performance degradation?

The test were done some years ago, during the covid era (2020). I used two PCs with Windows 10, processor I7-10700 and I7-9700 at that time with 16 GB of DDR4 RAM. ALL TEST WERE DONE WITH 64 bit APP.
I try test allocated stable memory and cache memory (or like you named it) allocated (and release) increased raw data (images) in a quickly way to stress the OS. This result was a variable size of working set, but the same sequence had done about the same results.

I used only one file to load an image at the start of the program, after that only copy in memory was done and no activity on disk were done.

 

With diagnostic and monitoring tools like those of "SysInternals" (I remember that I used RAMMAP, but also others that I don't have in mind now), I look for composition of working set.

 

After that simple run a part of analysis (some functions of vision analysis that I normally use), of course repeated some millions time and catch the global time of these analysis. Note that these timing test were done after that the working set are big enough. The same analysis were done after using the EmptyWorkingSet and the timing were restores.

 

The timing showed that with about 50% of memory used (eg. wotking set from API "GetProcessMemoryInfo")  by the the program (or programs when external loads were used) compared to that present in the PC, the are some degradation (with peaks of 5%) and with more used memory 75% the peaks go to 10%.

 

These was done with various mode: using the same application to allocate the memory and do tests, using external application to allocate the memory.

 

The two PCs does different results of course, with smaller percentages of variations with I7-9700.

 

During all these process Internet was disabled but Windows Defender was enabled.

 

After those tests I never repeats them.

 

EDIT: ... I talking about 200 ms of timing. One normal program cycle is from 90 ms. to 200 ms.

Edited May 3 by DelphiUdIT

[PeterBelow 238]

On 5/1/2024 at 12:12 PM, DelphiUdIT said:

In Windows you can use the function " EmptyWorkingSet(GetCurrentProcess) " (from WinAPI.PsAPI) to release all memory to OS. I use it on my application when the memory used is over 50% of all system memory (this help also to reduce the effects of fragmentation).

 

My applications use lot of images, near 2000 raw buffers / second (form some KB to some MB) and fragmentation is high. Every 4 / 5 days I look (from logs) that the application use that function. Normally the application stay on near one year, actively working 10 hours per day.

 

You should look for equivalent function in Linux (I'm quite sure that should exists).

Bye

 

 

 

You misunderstand what this API function does. It does not release any memory to the OS, it just moves in-memory pages the application uses to the system swap file. That reduces the amount of RAM shown in use by the app in tools like Taskmanager, but it is in fact completely pointless to use since the OS is smart enough to move memory pages accessed infrequently to the swap file on its own when more physical RAM is needed for another process. All it does is slow down the app performance due to having to load memory pages back from swap when they are accessed.

If your app is actually running out of virtual memory (the address space available for it) you have a problem in how you use memory in your application, e. g. trying to keep too much data in memory, not releasing memory correctly, or using a problematic allocation scheme that increases fragmentation of the memory blocks your app has obtained from the OS.

[DelphiUdIT 176]

5 hours ago, PeterBelow said:

You misunderstand what this API function does. It does not release any memory to the OS, it just moves in-memory pages the application uses to the system swap file. That reduces the amount of RAM shown in use by the app in tools like Taskmanager, but it is in fact completely pointless to use since the OS is smart enough to move memory pages accessed infrequently to the swap file on its own when more physical RAM is needed for another process. All it does is slow down the app performance due to having to load memory pages back from swap when they are accessed.

If your app is actually running out of virtual memory (the address space available for it) you have a problem in how you use memory in your application, e. g. trying to keep too much data in memory, not releasing memory correctly, or using a problematic allocation scheme that increases fragmentation of the memory blocks your app has obtained from the OS.

Like I wrote, I don't use swap file and I don't have any issues, except the performance (little loss). May be, with Windows 11 that is not a problem, but the software works perfectly in this way: next time that I'll refactor it I'll try to retest all.

Edited May 3 by DelphiUdIT

[David Heffernan 2345]

13 hours ago, DelphiUdIT said:

Like I wrote, I don't use swap file and I don't have any issues, except the performance (little loss). May be, with Windows 11 that is not a problem, but the software works perfectly in this way: next time that I'll refactor it I'll try to retest all.

If you aren't using a swap file, where do those pages go when removed from the working set? I mean, these are pages allocated by your process. 

 

My best guess is that you did a benchmarking exercise that reached an incorrect conclusion and then you have continued using this function erroneously. 

Edited May 4 by David Heffernan

[DelphiUdIT 176]

9 hours ago, David Heffernan said:

If you aren't using a swap file, where do those pages go when removed from the working set? I mean, these are pages allocated by your process. 

You look this post with result ?

 

 

There are simple memory non release to OS (paged, not paged, virtual, not virtual, ghost, not ghost, imaginary, not imaginary ...) and the acts like "cache".  This memory will be releases directly by OS if it's needed.

 

I simply forced that (there is not only EmptyWorkingSet APi but others that do the same) when the size of the WorkingSet is too high.

That's all, really simple. No side effects (except for the first "round" of application where I have latency, OF COURSE).

 

AND EmptyWorkingSet doesn't works (I means that the results are not constant) when there are leaks: I had done some tests with memory leaks, and the base memory, as expected, increased every time that a leak was generated. You can see it very well and is very clear with EmptyWorkingSet.

 

EDIT: this is the only way I know, but like always I might miss something of course, to view a leaks "on the line" that you have, when using third-party libraries.

 

If you don't have leaks, after three years (the "ancient" report with live application that I have) the base memory (simply measured after call the API) is exactly the same measured with byte precision.

 

9 hours ago, David Heffernan said:

My best guess is that you did a benchmarking exercise that reached an incorrect conclusion and then you have continued using this function erroneously. 

This is possible, I agreed with you: but until the next refactoring the things stay in this way. I promise that will do some tests to view if the conclusions are wrong, and report here that.

Edited May 4 by DelphiUdIT