Newbie: memory management

[Sonjli 6]

Hi,

I am trying to use your fantastic library (thanks for it).

I have a doubt about freeing memory. I have a master thread (a standard TThread by Delphi) that is responsible to run some omni tasks.

I use to do it in this way:

procedure TThreadMonitorBroadcast.Execute;
begin
...
			   while not GetArticoloVersato.getTabella.Eof do
               begin
                  ...
                  ATaskList := CreateTaskControlList;
                  try
                     // Run all registered threads
                     FBroadcastActions.ForEach(Action<IBroadcastAction>(
                        procedure(ABroadcastAction: IBroadcastAction)
                        begin
                           ATaskList.Add(CreateTask(
                              procedure(const Task: IOmniTask)
                              begin
                                 Sleep(100); // Only for test
                                 // ABroadcastAction.Run(Task, GetMacchina);
                              end).MonitorWith(FEventMonitor).Join(otlWaitGroup).Schedule(otlPool));
                        end
                        ));

                     otlWaitGroup.WaitForAll;

                     if otlPool.CountExecuting > 0 then
                        raise Exception.Create('Oh my..., thread still running');

                     ATaskList.Clear;
                  finally
                     ATaskList := nil;
                  end;
					...
                end;
                ...
end;

It looks like everything works finem but RAM is increasing at every loop. What am I wrong?

I tried to use a TaskControlList, but without success....

 

Thanks in advance

Eddy

[Sonjli 6]

While experimenting I found that the problem is the creation of the otlWaitGroup.

I created it out of the main loop; so now I moved it inside the loop like in the code below.

I have some doubts:

Why the reference to OmniTasks stay inside the WaitGroup until it is detroyed? Are they weak references?

Why do I have to create many waitGroups instead of creating only one, and use it many times?

Thanks anyone.

	   while not GetArticoloVersato.getTabella.Eof do
               begin
                  ...
                  ATaskList := CreateTaskControlList;
                  otlWaitGroup := CreateTaskGroup; // Here create it
                  try
                     // Run all registered threads
                     FBroadcastActions.ForEach(Action<IBroadcastAction>(
                        procedure(ABroadcastAction: IBroadcastAction)
                        begin
                           ATaskList.Add(CreateTask(
                              procedure(const Task: IOmniTask)
                              begin
                                 Sleep(100); // Only for test
                                 // ABroadcastAction.Run(Task, GetMacchina);
                              end).MonitorWith(FEventMonitor).Join(otlWaitGroup).Schedule(otlPool));
                        end
                        ));

                     otlWaitGroup.WaitForAll;

                     if otlPool.CountExecuting > 0 then
                        raise Exception.Create('Oh my..., thread still running');

                     ATaskList.Clear;
                  finally
                     otlWaitGroup := nil; // Here free it
                     ATaskList := nil;
                  end;
					...
                end;

[Olivier EXEL. 1]

18 minutes ago, Sonjli said:

While experimenting I found that the problem is the creation of the otlWaitGroup.

I created it out of the main loop; so now I moved it inside the loop like in the code below.

I have some doubts:

Why the reference to OmniTasks stay inside the WaitGroup until it is detroyed? Are they weak references?

Why do I have to create many waitGroups instead of creating only one, and use it many times?

Thanks anyone.

	   while not GetArticoloVersato.getTabella.Eof do
               begin
                  ...
                  ATaskList := CreateTaskControlList;
                  otlWaitGroup := CreateTaskGroup; // Here create it
                  try
                     // Run all registered threads
                     FBroadcastActions.ForEach(Action<IBroadcastAction>(
                        procedure(ABroadcastAction: IBroadcastAction)
                        begin
                           ATaskList.Add(CreateTask(
                              procedure(const Task: IOmniTask)
                              begin
                                 Sleep(100); // Only for test
                                 // ABroadcastAction.Run(Task, GetMacchina);
                              end).MonitorWith(FEventMonitor).Join(otlWaitGroup).Schedule(otlPool));
                        end
                        ));

                     otlWaitGroup.WaitForAll;

                     if otlPool.CountExecuting > 0 then
                        raise Exception.Create('Oh my..., thread still running');

                     ATaskList.Clear;
                  finally
                     otlWaitGroup := nil; // Here free it
                     ATaskList := nil;
                  end;
					...
                end;

I've just experimented an equivalent scenario and had an increasing memory too. I moved for an Parallel.for loop and it has resolved the issue ; but I had some freeing for long tasks.

I noted that, in this case (parallel.for), if you have an exception not taken in charge  internaly in the thread (try ...), then the threads is killed without freeing and u may experiment not finishing the group, for what I understand.

For now, for what I need  OTL Background Workers is the best way (has Gab wrote me in a precedent flow) ; one interesting thing is ... the exception management 'cause I had some in the threads with SQL timeout (that's I have to treat anyway).

 

Your use of TaskGroup is once, before ce loop ; then your "join" in the loop attach the task to the group..

[Sonjli 6]

Thanks Olivier. I'll try the background workers.

 

For the sentence:

Quote

Your use of TaskGroup is once, before ce loop ; then your "join" in the loop attach the task to the group.. 

Sorry, but I didn't understand... (maybe my poor english? 🙂 )

[Olivier EXEL. 1]

On 2/15/2019 at 4:23 PM, Sonjli said:

Thanks Olivier. I'll try the background workers.

 

For the sentence:

Sorry, but I didn't understand... (maybe my poor english? 🙂 )

I meant to do that once only before the loop : 

otlWaitGroup := CreateTaskGroup; // Here create it

[Sonjli 6]

1 hour ago, Olivier EXEL. said:

I meant to do that once only before the loop : 

otlWaitGroup := CreateTaskGroup; // Here create it

OK, but THAT is the problem... if I create the group outside the loop then memory increase; if I put that inside the loop, memory is stable.

Anyway, I am trying with background workers. They are awesome.