In Windows you can use the function " EmptyWorkingSet(GetCurrentProcess) " (from WinAPI.PsAPI) to release all memory to OS. I use it on my application when the memory used is over 50% of all system memory (this help also to reduce the effects of fragmentation).
My applications use lot of images, near 2000 raw buffers / second (form some KB to some MB) and fragmentation is high. Every 4 / 5 days I look (from logs) that the application use that function. Normally the application stay on near one year, actively working 10 hours per day.
You should look for equivalent function in Linux (I'm quite sure that should exists).
Bye