Xdata Rest server request & Python4Delphi

[J. Robroeks 1]

Hi

 

I've looked into the threading demo examples and read a bit about the GIL and other issues related to threading and p4d.

 

My issue is related to https://github.com/pyscripter/python4delphi/issues/47.

 

I've a REST server running where there are several endpoints that use Python script of P4D. The Pythonengine is created during the start of the server as there may only be one engine created in an application. Every request to an endpoint is running in its own thread. Therefore calling PythonEngine.ExecStrings(Script); directly is not thread safe. I prefer the rest request not to be async. 

 

Option 1: using the TPythonThread

The TPythonThread ensures that the script is executed in a thread safe manner. However, since the endpoint is not async and already running in a thread, there is no need to use a seperate thread other than for using the GIL lock.

 

Creating the thread:

OwnThreadState := PythonEngine1.PyEval_SaveThread;
Thread1 := TSortThread.Create(ThreadExecMode, script);
Thread1.WaitFor;
GetPythonEngine.PyEval_RestoreThread(OwnThreadState);

 

Executing the thread (pythonmodule is not needed)

procedure TSortThread.ExecuteWithPython;
begin
  running := true;
  try
    with GetPythonEngine do
    begin
      if Assigned(fScript) then
        ExecStrings(fScript);
    end;
  finally
    running := false;
  end;
end;

Questions:

• TThreadExecMode = (emNewState, emNewInterpreter); Is it allowed to use emNewState when TPythonVars and the script change per thread creation? (this way the import of certain packages is not done for each thread creation seperately)

 

Option 2: using the GIL (similar to what TPythonThread does)

OwnThreadState := PythonEngine1.PyEval_SaveThread;
gilstate  :=  PythonEngine.PyGILState_Ensure; 
PySUBJECT := TPythonDelphiVar.Create(nil);
try
    PySUBJECT.Engine := PyEngine;
    PySUBJECT.VarName := 'SUBJECT';
    PySUBJECT.Module := '__main__';
    PySUBJECT.Initialize;
    PySUBJECT.Value := onderwerp;	
    PythonEngine.ExecStrings(Stringlist);
finally
    PythonEngine.PyGILState_Release(gilstate); 
    PySUBJECT.Free;
end;
GetPythonEngine.PyEval_RestoreThread(OwnThreadState);

 

Questions:

• Why is the PythonEngine.PyEval_SaveThread; necessary for using the PyGILState_Ensure? 
• When testing with a parallel.for and the GIL ensures that the TPythonDelphiVar is working correctly. Or am I wrong?

 

Overall question:

 

• Is it OK to call the  PythonEngine.PyEval_SaveThread during the startup of the RestServer and PyEval_RestoreThread(OwnThreadState) when the server stops, otherwise the GIL lock will not work?

 

 

Thank you! 

[pyscripter 689]

In your code above you should destroy PY_SUBJECT before releasing the GIL

Python has a lesser known feature called sub-interpreters, which allows you to use the interpreter from a clean state.  This is what emNewInterpreter does.  Normally I would not bother with that.

 

The pattern you need to follow using the latest version of P4D:

 

In your main thread to release the GIL after loading the Python dll:  

TPythonThread.Py_Begin_Allow_Threads  (calls PyEval_SaveThread)

 

In your Delphi threads including the main thread (or if you use ITask or  TParallel) that execute python code (this is what TPythonThread does):

  fGILState := .PyGILState_Ensure;
  try
    Do python staff
  finally
     PyGILState_Release(fGILState);
  end;

 

In your main thread before unlolading Python

TPythonThread.Py_End_Allow_Threads  (calls PyEval_RestoreThread)

 

if you have a long running thread that does python stuff and you want allow other threads to do python stuff as well then the pattern is:

 

  fGILState := .PyGILState_Ensure;
  try
    Do python staff
    TPythonThread.Begin_Allow_Threads;
    try
      Other threads can run python code
    finally
      TPythonThread.End_ALlow_Threads;
    end;
    Do more python stuff.
  finally
     PyGILState_Release(fGILState);
  end;

 

In PyScripter I have a utility function:

type
  IPyEngineAndGIL = interface
    function GetPyEngine: TPythonEngine;
    function GetThreadState: PPyThreadState;
    property PythonEngine: TPythonEngine read GetPyEngine;
    property ThreadState: PPyThreadState read GetThreadState;
  end;

function SafePyEngine: IPyEngineAndGIL;
begin
  Result := TPyEngineAndGIL.Create
end;


type
  TPyEngineAndGIL = class(TInterfacedObject, IPyEngineAndGIL)
  fPythonEngine: TPythonEngine;
  fThreadState: PPyThreadState;
  fGILState: PyGILstate_STATE;
  private
    function GetPyEngine: TPythonEngine;
    function GetThreadState: PPyThreadState;
  public
    constructor Create;
    destructor Destroy; override;
  end;

{ TPyEngineAndGIL }

constructor TPyEngineAndGIL.Create;
begin
  inherited Create;
  fPythonEngine := GetPythonEngine;
  fGILState := fPythonEngine.PyGILState_Ensure;
  fThreadState := fPythonEngine.PyThreadState_Get;
end;

destructor TPyEngineAndGIL.Destroy;
begin
  fPythonEngine.PyGILState_Release(fGILState);
  inherited;
end;

function TPyEngineAndGIL.GetPyEngine: TPythonEngine;
begin
  Result := fPythonEngine;
end;

function TPyEngineAndGIL.GetThreadState: PPyThreadState;
begin
  Result := fThreadState;
end;

which is used in the main or other threads as:

var Py: IPyEngineAndGIL;

begin

   Py := SafePyEngine;

   Py.Engine.

 

whenever I need to execute Python code

Edited March 10, 2021 by pyscripter

[J. Robroeks 1]

Thank you for your quick and elaborate answer. The py_subject was indeed freed at the wrong place. 

 

In case someone else wonders the following:

There are several funtions in Python that are non-blocking. For example:

1. sleep
2. urlopen

 

[pyscripter 689]

On 3/10/2021 at 5:55 PM, J. Robroeks said:

Thank you for your quick and elaborate answer. The py_subject was indeed freed at the wrong place. 

 

In case someone else wonders the following:

There are several funtions in Python that are non-blocking. For example:

1. sleep
2. urlopen

 

None of the python code is blocking other threads.  But the following functions are blocking:

• PyGILState_Ensure
• PyEval_RestoreThread

In other words you need to get and hold to the GIL to execute any python code.  So take for example sleep.  It works like windows sleep.   But to let other threads to execute python code you need to use Py_Begin_Allow_Threads /Py_End_Allow_Threads   before/after the sleep.  Python deals with the threads it creates.  (threading module).

Edited March 13, 2021 by pyscripter

[wuwuxin 28]

On 3/10/2021 at 8:09 AM, pyscripter said:

In your code above you should destroy PY_SUBJECT before releasing the GIL

Python has a lesser known feature called sub-interpreters, which allows you to use the interpreter from a clean state.  This is what emNewInterpreter does.  Normally I would not bother with that.

 

The pattern you need to follow using the latest version of P4D:

 

In your main thread to release the GIL after loading the Python dll:  

TPythonThread.Py_Begin_Allow_Threads  (calls PyEval_SaveThread)

 

In your Delphi threads including the main thread (or if you use ITask or  TParallel) that execute python code (this is what TPythonThread does):

  fGILState := .PyGILState_Ensure;
  try
    Do python staff
  finally
     PyGILState_Release(fGILState);
  end;

 

In your main thread before unlolading Python

TPythonThread.Py_End_Allow_Threads  (calls PyEval_RestoreThread)

 

if you have a long running thread that does python stuff and you want allow other threads to do python stuff as well then the pattern is:

 

  fGILState := .PyGILState_Ensure;
  try
    Do python staff
    TPythonThread.Begin_Allow_Threads;
    try
      Other threads can run python code
    finally
      TPythonThread.End_ALlow_Threads;
    end;
    Do more python stuff.
  finally
     PyGILState_Release(fGILState);
  end;

 

In PyScripter I have a utility function:

type
  IPyEngineAndGIL = interface
    function GetPyEngine: TPythonEngine;
    function GetThreadState: PPyThreadState;
    property PythonEngine: TPythonEngine read GetPyEngine;
    property ThreadState: PPyThreadState read GetThreadState;
  end;

function SafePyEngine: IPyEngineAndGIL;
begin
  Result := TPyEngineAndGIL.Create
end;


type
  TPyEngineAndGIL = class(TInterfacedObject, IPyEngineAndGIL)
  fPythonEngine: TPythonEngine;
  fThreadState: PPyThreadState;
  fGILState: PyGILstate_STATE;
  private
    function GetPyEngine: TPythonEngine;
    function GetThreadState: PPyThreadState;
  public
    constructor Create;
    destructor Destroy; override;
  end;

{ TPyEngineAndGIL }

constructor TPyEngineAndGIL.Create;
begin
  inherited Create;
  fPythonEngine := GetPythonEngine;
  fGILState := fPythonEngine.PyGILState_Ensure;
  fThreadState := fPythonEngine.PyThreadState_Get;
end;

destructor TPyEngineAndGIL.Destroy;
begin
  fPythonEngine.PyGILState_Release(fGILState);
  inherited;
end;

function TPyEngineAndGIL.GetPyEngine: TPythonEngine;
begin
  Result := fPythonEngine;
end;

function TPyEngineAndGIL.GetThreadState: PPyThreadState;
begin
  Result := fThreadState;
end;

which is used in the main or other threads as:

var Py: IPyEngineAndGIL;

begin

   Py := SafePyEngine;

   Py.Engine.

 

whenever I need to execute Python code

I am still somewhat confused - if I call PyGILState_Ensure in one thread,  will that block other thread calling PyGILState_Ensure?

 

If I use your utility function IPyEngineAndGIL, do I still need to call TPythonThread.Py_Begin_Allow_Threads  (calls PyEval_SaveThread) in the main thread, after loading the Python DLL?

 

Thank you.

 

 

[wuwuxin 28]

I am still somewhat confused - if I call PyGILState_Ensure in one thread,  will that block other threads calling PyGILState_Ensure?

 

If I use your utility function IPyEngineAndGIL, do I still need to call TPythonThread.Py_Begin_Allow_Threads  (calls PyEval_SaveThread) in the main thread, after loading the Python DLL?

 

Thank you.

Edited June 15, 2021 by wuwuxin

[wuwuxin 28]

10 hours ago, wuwuxin said:

I am still somewhat confused - if I call PyGILState_Ensure in one thread,  will that block other threads calling PyGILState_Ensure?

 

If I use your utility function IPyEngineAndGIL, do I still need to call TPythonThread.Py_Begin_Allow_Threads  (calls PyEval_SaveThread) in the main thread, after loading the Python DLL? 

 

Thank you.

OK.  After studying Python documentation https://docs.python.org/3/c-api/init.html

 

I think I found the answer.  Python internally has a thread-switching mechanism, controlled by sys.setswitchinterval.   PyGILState_Ensure tries to get the GIL lock - which is blocking.  It seems when the thread context is switched to the current thread,  GIL lock held by a previously running thread will have already been (automatically) released, and the current running thread can get the GIL lock.  So in that sense,  PyGILState_Ensure is not blocking at all (i.e., with an actively running thread,  it won't "block" since it can always get the GIL lock) - is my understanding correct?

 

 

Edited June 15, 2021 by wuwuxin

[pyscripter 689]

 

3 hours ago, wuwuxin said:

PyGILState_Ensure is not blocking at all

Of course it is.   If another thread holds the lock, PyGILState_Ensure blocks until the GIL becomes available, i.e. the holding thread calls PyGILState_Release.  If more than one threads are waiting for GIL then I think it is assigned of FCFS basis.  The automatic switching that you described only applies to threads created with the threading code.  All other threads need to get the GIL before executing python code and then release it.

 

14 hours ago, wuwuxin said:

If I use your utility function IPyEngineAndGIL, do I still need to call TPythonThread.Py_Begin_Allow_Threads  (calls PyEval_SaveThread) in the main thread, after loading the Python DLL?

Yes you do need to call Py_Begin_Allow_Threads in the main thread.   When the python DLL loads, the main thread owns the GIL and it needs to release it before other threads can execute python code.

 

 

Edited June 15, 2021 by pyscripter

[wuwuxin 28]

To be exact,  Py_Begin_Allow_Threads should be called inside PythonEngine.OnAfterInit event,  NOT OnAfterLoad event as advised by @pyscripter.  Otherwise,  error will be thrown, because the gPythonEngine at this point is still nil.

 

Py_End_Allow_Threads  should NOT be called inside OnBeforeUnload event.  If put inside OnBeforeUnload event, an access violation error will be thrown - because Py_Finialize will be called BEFORE OnBeforeUnload,  thus would trigger an access violation error at the point where Py_Finalize is called, which is before firing OnBeforeUnload event.

 

I think a new event OnBeforeFinalize should be added, to address this problem.  @pyscripter   What am I missing here?

Edited June 15, 2021 by wuwuxin

[pyscripter 689]

1 hour ago, wuwuxin said:

To be exact,  Py_Begin_Allow_Threads should be called inside PythonEngine.OnAfterInit event,  NOT OnAfterLoad event as advised by @pyscripter.  

I said "after loading" loosely speaking and not "in the OnAfterLoad event".  Indeed you can only call Py_Begin_Allow_Threads after the engine is initialized.

 

I have not tested, but you can set AutoFinalize to False and in the OnBeforeUnload do

Py_End_Allow_Threads;
Finalize;

 

Also probably, skipping the above when the application closes down may not be necessary.

Edited June 16, 2021 by pyscripter