How to free PythonEngine in background thread?

[DennisTW 1]

I seem to have successfully created PythonEngine inside a thread:

          RichEdit1 := TMemo.Create(nil);

          PythonGUIInputOutput := TPythonGUIInputOutput.Create(nil);
          PythonGUIInputOutput.UnicodeIO := true;
          PythonGUIInputOutput.Output := RichEdit1;

          modDBFireDac := TPythonModule.Create(nil);
          modDBFireDac.OnInitialization := modDBFireDacInitialization;
          modDBFireDac.ModuleName := 'DBFireDac';

          PyDelphiWrapper := TPyDelphiWrapper.Create(nil);

          PythonEngine := TPythonEngine.Create(nil); // 4.17
          PythonEngine.IO := PythonGUIInputOutput;

          modDBFireDac.Engine := PythonEngine;
          PyDelphiWrapper.Engine := PythonEngine;

          PythonEngine.LoadDll;
          
However, when I tried to unload and free the pythonengine inside the same background thread later, it hanged at PythonEngine.Free:
          FreeAndNil(PyDelphiWrapper);
          FreeAndNil(modDBFireDac);
          FreeAndNil(PythonGUIInputOutput);
          FreeAndNil(RichEdit1);
          if PythonEngine <> nil then begin
             PythonEngine.UnloadDll;
             PythonEngine.Free;//  program hanged here!!!!
             PythonEngine := nil;
          end;
          
If I don't PythonEngine.Free, Delphi will report memory leak.
What should I do?

 

[DennisTW 1]

After debugging, I traced the problem to the Py_Finalize inside the procedure TPythonEngine.Finalize.

Calling Py_Finalize will freeze the program.

And   Py_Finalize                 := Import('Py_Finalize'); so, the problem is within the dll.

 

If I skip calling Py_Finalize, there will be a small memory leak of 89 - 104 bytes of Unknown x 1

[Dalija Prasnikar 1397]

I don't use Python4Delphi so I cannot accurately tell what can and what cannot be done with it in context of threads. 

 

But there is one thing definitely wrong in your code and that is constructing TMemo in the context of the background thread. VCL controls can only ever be used from the main thread. Because how that memo control is connected to the Python engine sounds like it is TComponent based. Just being TComponent based does not necessarily mean that it cannot be used in the background threads, but in such cases all related components must belong to the same thread. 

 

If we know that TMemo must be used in the main thread, then if you connect it to the Python engine implies that such configured Python engine also must be constructed and used from the main thread.

 

Just because you managed to "make it work" without obvious error (if you ignore the leak), does not mean that this kind of setup actually works.

[SwiftExpat 65]

First note Dalija's response above about the TMemo.

 

Your memory leak will be solved by calling finalize on modDBFiredac before you call free.

 

If you are going to use python on a thread replace TPythonGUIInputOutput with TPythonInputOutput and consider using DEB to send messages to the main thread, it handles the sync.

[DennisTW 1]

3 hours ago, Dalija Prasnikar said:

I don't use Python4Delphi so I cannot accurately tell what can and what cannot be done with it in context of threads. 

 

But there is one thing definitely wrong in your code and that is constructing TMemo in the context of the background thread. VCL controls can only ever be used from the main thread. Because how that memo control is connected to the Python engine sounds like it is TComponent based. Just being TComponent based does not necessarily mean that it cannot be used in the background threads, but in such cases all related components must belong to the same thread. 

 

If we know that TMemo must be used in the main thread, then if you connect it to the Python engine implies that such configured Python engine also must be constructed and used from the main thread.

 

Just because you managed to "make it work" without obvious error (if you ignore the leak), does not mean that this kind of setup actually works.

Thanks for your advice.

[DennisTW 1]

3 hours ago, SwiftExpat said:

First note Dalija's response above about the TMemo.

 

Your memory leak will be solved by calling finalize on modDBFiredac before you call free.

 

If you are going to use python on a thread replace TPythonGUIInputOutput with TPythonInputOutput and consider using DEB to send messages to the main thread, it handles the sync.

Excuse me. What is DEB?  I googled but can't find it. 

[SwiftExpat 65]

My Bad, DEB is Delphi Event Bus, from your thread you just call : https://github.com/spinettaro/delphi-event-bus

 GlobalEventBus.Post(lEsa, 'Processed');

On the form you can subscribe and handle the sync by choosing the thread mode:

    [Subscribe(TThreadMode.Main, 'HostUpdated')]
    procedure OnEventHostAdminUpdate(AEvent: IDEBEvent<TSERTCCollectionHostSSH>);