Jump to content

DelphiUdIT

Members
 View Profile  See their activity

  • Content Count

    445

  • Joined

  • Last visited

  • Days Won

    8

Posts posted by DelphiUdIT

  1. Posted · Edited by DelphiUdIT

    9 hours ago, David Heffernan said:

    If you aren't using a swap file, where do those pages go when removed from the working set? I mean, these are pages allocated by your process. 

    You look this post with result ?

     

    image.png

     

    There are simple memory non release to OS (paged, not paged, virtual, not virtual, ghost, not ghost, imaginary, not imaginary ...) and the acts like "cache".  This memory will be releases directly by OS if it's needed.

     

    I simply forced that (there is not only EmptyWorkingSet APi but others that do the same) when the size of the WorkingSet is too high.

    That's all, really simple. No side effects (except for the first "round" of application where I have latency, OF COURSE).

     

    AND EmptyWorkingSet doesn't works (I means that the results are not constant) when there are leaks: I had done some tests with memory leaks, and the base memory, as expected, increased every time that a leak was generated. You can see it very well and is very clear with EmptyWorkingSet.

     

    EDIT: this is the only way I know, but like always I might miss something of course, to view a leaks "on the line" that you have, when using third-party libraries.

     

    If you don't have leaks, after three years (the "ancient" report with live application that I have) the base memory (simply measured after call the API) is exactly the same measured with byte precision.

     

    9 hours ago, David Heffernan said:

    My best guess is that you did a benchmarking exercise that reached an incorrect conclusion and then you have continued using this function erroneously. 

    This is possible, I agreed with you: but until the next refactoring the things stay in this way. I promise that will do some tests to view if the conclusions are wrong, and report here that.

  2. Posted · Edited by DelphiUdIT

    5 hours ago, PeterBelow said:

    You misunderstand what this API function does. It does not release any memory to the OS, it just moves in-memory pages the application uses to the system swap file. That reduces the amount of RAM shown in use by the app in tools like Taskmanager, but it is in fact completely pointless to use since the OS is smart enough to move memory pages accessed infrequently to the swap file on its own when more physical RAM is needed for another process. All it does is slow down the app performance due to having to load memory pages back from swap when they are accessed.

    If your app is actually running out of virtual memory (the address space available for it) you have a problem in how you use memory in your application, e. g. trying to keep too much data in memory, not releasing memory correctly, or using a problematic allocation scheme that increases fragmentation of the memory blocks your app has obtained from the OS.

    Like I wrote, I don't use swap file and I don't have any issues, except the performance (little loss). May be, with Windows 11 that is not a problem, but the software works perfectly in this way: next time that I'll refactor it I'll try to retest all.

  3. Posted · Edited by DelphiUdIT

    1 hour ago, David Heffernan said:

    How are you measuring the performance degradation?

    The test were done some years ago, during the covid era (2020). I used two PCs with Windows 10, processor I7-10700 and I7-9700 at that time with 16 GB of DDR4 RAM. ALL TEST WERE DONE WITH 64 bit APP.
    I try test allocated stable memory and cache memory (or like you named it) allocated (and release) increased raw data (images) in a quickly way to stress the OS. This result was a variable size of working set, but the same sequence had done about the same results.

    I used only one file to load an image at the start of the program, after that only copy in memory was done and no activity on disk were done.

     

    With diagnostic and monitoring tools like those of "SysInternals" (I remember that I used RAMMAP, but also others that I don't have in mind now), I look for composition of working set.

     

    After that simple run a part of analysis (some functions of vision analysis that I normally use), of course repeated some millions time and catch the global time of these analysis. Note that these timing test were done after that the working set are big enough. The same analysis were done after using the EmptyWorkingSet and the timing were restores.

     

    The timing showed that with about 50% of memory used (eg. wotking set from API "GetProcessMemoryInfo")  by the the program (or programs when external loads were used) compared to that present in the PC, the are some degradation (with peaks of 5%) and with more used memory 75% the peaks go to 10%.

     

    These was done with various mode: using the same application to allocate the memory and do tests, using external application to allocate the memory.

     

    The two PCs does different results of course, with smaller percentages of variations with I7-9700.

     

    During all these process Internet was disabled but Windows Defender was enabled.

     

    After those tests I never repeats them.

     

    EDIT: ... I talking about 200 ms of timing. One normal program cycle is from 90 ms. to 200 ms.

  5. Posted · Edited by DelphiUdIT

    2 hours ago, Anders Melander said:

    You still haven't understood how this works.

     

    The memory in your working set isn't "owned" by the process; It's owned by Windows and the process only has it on loan. If Windows needs the memory for something else (e.g. another process) it will take it back without any action needed (or possible) from your process.

     

    The working set pages that you are trimming are unused by the process but was assigned to it at some point because it needed them. The reason that the OS hasn't removed them from your working set by itself is that there has been no need for it; Nobody else has made a memory request that couldn't be satisfied elsewhere. The whole idea behind the working set model is to allow the system to grow and shrink the working sets based on demand.

     

    Why not spend 30 minutes to read up on how it works. Just Google windows working set.

     

    Or simply search for EmptyWorkSet and see if you can find a single person who thinks it's a good solution.

     

    4 hours ago, David Heffernan said:

    So why would spending the time to do this be better than letting the system do it on demand?

    I understood like it works, and I described what happens ... but you don't have catch the problem and why I use it.

    I use it because when the memory "in use", "locked", "booked", "cached", or like you want to describe it, is too high there is performance degradation.

     

    (to respond to @David, i check normally the memory status, like temperature and others, and launch the function is two lines, of code: "if ... then Empty...." not so much time or energy spent)

     

    Why there is that degradation I don't know, but for sure is that when the memory .... is to high, using EmptyWorkingSet is useful.

    And this every 4 or 5 days ... I don't use it every single machine cycle.

     

    But if EmptyWorkingSet is so "objectless", why exists ? It may be that whoever created the automatic memory management also foreseen that perhaps this mechanism may not be infallible, and has given a tool to force "the hand".

     

    But all that (especially how I handle the lines) is not correlated with the original question of the topic, I'm OT so I suggest to start a new topic if needed.

    Bye

     

    P.S.: @Lars Fosdal  or some moderators, move my posts and correlated to other new topic ... except for one or two, others are nothing to do with the original topics. Thanks

  6. Posted

    1 hour ago, David Heffernan said:

    This just makes your program slower though. Forcing the pages out of memory and onto disk before other programs need that memory takes time and can only lead to performance reduction. 

     

    You may as well open the computer up and remove some memory sticks! 

    Like I told I don't use virtual memory on disk (it's disabled). And always like I told that function is called one time every 4 o 5 days ... in the PC run's only my software (and of course normal Windows services) and it's for this that cache memory become very high.

    I call the function when a memory occupation over 50% is detected and when the machine has the first functional stop. After that (always when the machine in in the stopped mode) a simulated run (one shot) is done to resolve the latency.

     

    All that is about 100 ms. (include one simulated round of application).

    The EmptyWorkSet doesn't discharge the memory pages onto disk, he release the memory to OS (from Application workspace to OS).

  7. Posted

    @Anders Melander

    I'm not completely in line with you.
    It's true that memory management is automatic and normally you don't have to worry about it, but have you ever tried to observe the performance of an application with more or less available memory (I'm talking about Windows)?
    I did in the past some tests using more or less memory, both "privately" allocated and used in cache, and performance drops in proportion to availability.
    Testing up to 40% of occupied memory there are no problems whatsoever but from 50% and above there are performance losses of 5% on average, from 75% I have detected performance losses of up to 10%.
    All analyzes were done using the exact same dataset (specifically images) and occupancy was simulated with both real occupancy and cached occupancy (approximately estimated), measuring average processing times. The memory occupation was carried out both inside the application and externally, and the two situations gave comparable results.
    There is no virtual disk memory in my systems, and no file operations were performed during testing.
    I use PC with 32GB memory and peak usage of 24GB is normal. The processor is normally used at around 75% and the CPU temperature varies between 75 and 85 degrees.
    I can assure you that without the use of the EmptyWorkingSet I would have to slow down the industrial line at peak times, obviously automatically and in real time (which is what I did in the past).

  8. Posted · Edited by DelphiUdIT

    28 minutes ago, David Heffernan said:

    EmptyWorkingSet removes pages from the working set. Doesn't this mean your program won't be able to take advantage of memory caching? 

    Yes, you're right. There is a little latency (in my application measured in 10 milliseconds near) when the program start again to works and use (allocate) memory.
    But whit this, I can maintain the memory free, especially when a lot of gigabytes are used.

     

    The last post is the example that also in Windows there is a memory that was not freed, like in Linux and in others OS probably, and using that API you can free memory (cache, but not only) if you need.

    And I think the should be a function in Linux too that can do this (although I don't know in depth the memory management system in Linux, so I might be talking nonsense).

  12. Posted · Edited by DelphiUdIT

    5 hours ago, Lars Fosdal said:

    How does that work with memory allocations in FastMM?

    I don't know how "working set"  interact with memory (page, virtual, etc..) and standard FastMM, but the function is really working freeing memory not released. I used FastMM5 for test and not side effects will be notice. I tested also EmptyWorkingSet in leak condition and in this case the effects are near to NULL (only little parts of memory freed).

    Some links: https://learn.microsoft.com/en-us/windows/win32/memory/working-set

                      https://learn.microsoft.com/en-us/windows/win32/psapi/working-set-information

                      https://learn.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-emptyworkingset

     

  13. Posted

    In Windows you can use the function " EmptyWorkingSet(GetCurrentProcess) " (from WinAPI.PsAPI) to release all memory to OS. I use it on my application when the memory used is over 50% of all system memory (this help also to reduce the effects of fragmentation).

     

    My applications use lot of images, near 2000 raw buffers / second (form some KB to some MB) and fragmentation is high. Every 4 / 5 days I look (from logs) that the application use that function. Normally the application stay on near one year, actively working 10 hours per day.

     

    You should look for equivalent function in Linux (I'm quite sure that should exists).

    Bye

     

     

     

  15. Posted

    It's more complex than this ...

    The wrapper (tlb) (it's the "ASCOM Device Interfaces for Platform 6") expose only the interfaces, there is a "scopeSim" that expose similar interface and a full class.

     

    He must explorer all type libraries registered about that (or read the full documentation) and find the class that use that interface (or how to use if it's exposed in the documentation or in the examples like @Anders told).

    Bye

  16. Posted

    40 minutes ago, baka0815 said:

    However I'm not using different ports, but maybe I will still figure something out.

    In the article that I signal  in the last post there are also some suggestions to use only one port ... like you need.

  18. Posted · Edited by DelphiUdIT

    5 hours ago, baka0815 said:

    Wouldn't that be something that could be automated in the CI of the Indy project (the creation of the cmd, without the txt file I mean)?

    In the Embarcadero release, Indy packages, components and other stuffs may be differ from GitHub maintained release. For example IndyIPServer, IndyIPClient and others are not present in the GitHub version.

    EDIT: also the runtime version may be differ like 12.0 and 12.1 where in the 12.1 release the new WIN64X "platform" exists.

  19. Posted · Edited by DelphiUdIT

    (This is for TIdHTTPServer component, hope to be useful for you component).

    You must use the "TIdSSLIOHandlerSocketBase.PassThrough" property ... if you set it to FALSE then TLS is required, if you set to TRUE only clear connection (NO TLS) are managed.

    Do it in the "create connection" event:

      

    procedure TForm1.Server1Connect(AContext: TIdContext);
begin
  If AContext.Connection.IOHandler is TIdSSLIOHandlerSocketBase then
    TIdSSLIOHandlerSocketBase(AContext.Connection.IOHandler).PassThrough := false;     //If True the authentication is not managed (only in clear text) or is used for STARTTLS, if false it manages the TLS/SSL authentication
end;

     

  20. Posted · Edited by DelphiUdIT

    EDIT: This is for TComboBox, I never used TComboBoxEx and I don't know if that I suggest is working with that component.

     

    You can use OnDrawItem to draw what you want this is an example that draw a flag (form an array of pngimages), you have to see if you can also use this method for the Edit position:

      

    //FlagListPNG is a list of pngimages
//NameLangLisTXT is a list of string (name of the languages)
//You must adapt of course at your use

procedure TLanguageSelect.ComboBox1DrawItem(Control: TWinControl;
  Index: Integer; Rect: TRect; State: TOwnerDrawState);
begin
  try
    TComboBox(Control).Canvas.Draw(Rect.Left + 8, Rect.Top + (Rect.Height div 2 - FlagListPNG[Index].Height div 2), FlagListPNG[Index]);
    TComboBox(Control).Canvas.TextOut(Rect.Left + 8 + FlagListPNG[Index].Width + 8, Rect.Top, NameLangListTXT[Index]+'        ');
  except
    TComboBox(Control).Canvas.TextOut(Rect.Left + 10, Rect.Top, NameLangListTXT[Index]);
  end;
end;

    image.thumb.png.b47a6e58c7df18a51c6ee683a0434d1f.png

×