J. Robroeks

Hi I've looked into the threading demo examples and read a bit about the GIL and other issues related to threading and p4d. My issue is related to https://github.com/pyscripter/python4delphi/issues/47. I've a REST server running where there are several endpoints that use Python script of P4D. The Pythonengine is created during the start of the server as there may only be one engine created in an application. Every request to an endpoint is running in its own thread. Therefore calling PythonEngine.ExecStrings(Script); directly is not thread safe. I prefer the rest request not to be async. Option 1: using the TPythonThread The TPythonThread ensures that the script is executed in a thread safe manner. However, since the endpoint is not async and already running in a thread, there is no need to use a seperate thread other than for using the GIL lock. Creating the thread: OwnThreadState := PythonEngine1.PyEval_SaveThread; Thread1 := TSortThread.Create(ThreadExecMode, script); Thread1.WaitFor; GetPythonEngine.PyEval_RestoreThread(OwnThreadState); Executing the thread (pythonmodule is not needed) procedure TSortThread.ExecuteWithPython; begin running := true; try with GetPythonEngine do begin if Assigned(fScript) then ExecStrings(fScript); end; finally running := false; end; end; Questions: TThreadExecMode = (emNewState, emNewInterpreter); Is it allowed to use emNewState when TPythonVars and the script change per thread creation? (this way the import of certain packages is not done for each thread creation seperately) Option 2: using the GIL (similar to what TPythonThread does) OwnThreadState := PythonEngine1.PyEval_SaveThread; gilstate := PythonEngine.PyGILState_Ensure; PySUBJECT := TPythonDelphiVar.Create(nil); try PySUBJECT.Engine := PyEngine; PySUBJECT.VarName := 'SUBJECT'; PySUBJECT.Module := '__main__'; PySUBJECT.Initialize; PySUBJECT.Value := onderwerp; PythonEngine.ExecStrings(Stringlist); finally PythonEngine.PyGILState_Release(gilstate); PySUBJECT.Free; end; GetPythonEngine.PyEval_RestoreThread(OwnThreadState); Questions: Why is the PythonEngine.PyEval_SaveThread; necessary for using the PyGILState_Ensure? When testing with a parallel.for and the GIL ensures that the TPythonDelphiVar is working correctly. Or am I wrong? Overall question: Is it OK to call the PythonEngine.PyEval_SaveThread during the startup of the RestServer and PyEval_RestoreThread(OwnThreadState) when the server stops, otherwise the GIL lock will not work? Thank you!

Do you have a link to SA? I'll post my answer there as well. That might be useful for others. I'll check the quality portal. Returning values in DML's would be neat.

I was encountering an issue that is related to this topic: Inserting a parent record, with a FK to a newly inserted child record. Both the parentid as well as the childid are needed for further processing Array DML is needed as the inserts can reach up to 30.000 parent records (single inserts take 1 min. while a DML query takes 1s) There is no way to retrieve the returning ID's when performaning a array DML with firedac. (If I'm wrong, please let me know). But this can achieved with simple SQL: In case the returning ID value is only needed to relate the child record to the parent record then the following does the trick: with rows as (insert into parent DEFAULT VALUES returning parentid) INSERT INTO child (parentid) SELECT parentid FROM rows In case the returning ID value is needed, then you can solve this with a temporary table and select the records you need. You can leverage options that PG offers such as upserts, unique etc. Also make sure to drop the temp table (or use DROP ON COMMIT) and set the cursorkind of the FDQuery to default. with rows as (insert into parent DEFAULT VALUES returning parentid) INSERT INTO temp_table (parentid) SELECT parentid FROM rows

Hi, I'm trying to get started with the python4delphi with Ubuntu 18.04 on Subsytem for Windows. But I always receive the error: Could not open DLL "libpython3.3m.so". What I've tried so far: Downloaded the packages via Git on 26-2 Installation did trow errors of unavailable packages. I finally build the non-dcl projects again and installed the dcl as 'Explicit rebuild'. (I don't know whether it had something to do with the $(auto).bpl suffix). Building and Running Demo project files for 32 bits Windows. (works) Building and deploying for 64 bit Linux (works) Installing Python 3.8 via get-apt and tested it via CLI (works) Running Python commands in Ubuntu after installation via get-apt (works) Changing the standard python verison on ubuntu to 3. (works, even though this is not recommended) Located the .so file via locate libpython3. Changed the PythonEnging paths and DLL's accordingly. (no succes) Result: /usr/lib/x86_64-linux-gnu/libpython3.6m.so.1 /usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0 /usr/lib/config-3.6m-x86_64-linux-gnu/libpython3.6.so /usr/lib/config-3.6m-x86_64-linux-gnu/libpython3.6m.so I don't know why the 3.8 .so files are not found. Searching for Linux examples, but only found one for Lazarus. But I think I've overlooked something here. (I tried the source, bit without luck) I've changed the DLL name to several different versions. libpython3.8m.so, libpython3.8.so and even older 2.7 versions. (No luck) Played with several PythonEngine settings, such as uselastknownversion, redirect IO, Autoload etc. Changing dllpaths. (No luck) The project file so far as Console application: PythonEngine1:= TPythonEngine.Create(nil); PythonInOut := TPythonInputOutput.Create(nil); try PythonEngine1.IO := PythonInOut; PythonEngine1.DllName := 'libpython3.6m.so'; PythonEngine1.AutoLoad := true; PythonEngine1.DllPath := '/usr/lib/x86_64-linux-gnu/'; PythonEngine1.AutoFinalize := true; PythonEngine1.AutoLoad := true; PythonEngine1.AutoUnload := true; PythonEngine1.RedirectIO := true; PythonEngine1.UseLastKnownVersion := true; PythonEngine1.Loaddll; finally PythonEngine1.Free; PythonInOut.Free; end; except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; Is there anyone who knows what I'm doing wrong here? Is it probably my Delphi environment, or Ubunut? Kind regards, Jan

Hi, The following code has a memory leak (73-88 bytes: Unknown x 1). Why is that? PySUBJECT := TPythonDelphiVar.Create(nil); try PySUBJECT.Engine := self.PythonEngine1; PySUBJECT.VarName := 'SUBJECT'; PySUBJECT.Initialize; finally PySUBJECT.Free; end; The memory leak does not occure when setting the owner to the pythonengine. Moreover, not freeing the var and setting the owner to the pythonengine will just consume memory until the pythonengine if freed. TParallel.For(1, 10, procedure(I: Integer) var PySUBJECT : TPythonDelphiVar; Py : IPyEngineAndGIL; begin Py := TPyEngineAndGIL.Create; PySUBJECT := TPythonDelphiVar.Create(nil); try PySUBJECT.Engine := Py.GetPyEngine; PySUBJECT.VarName := 'SUBJECT'; PySUBJECT.Module := TGuid.NewGUID.ToString; PySUBJECT.Initialize; PySUBJECT.Value := randomvalue; Py.GetPyEngine.ExecModule := PySUBJECT.Module; Py.GetPyEngine.ExecStrings(script); finally PySUBJECT.Free; end; end); Thanks!

The var has to be finalized before freeing the object: PySUBJECT := TPythonDelphiVar.Create(nil); try PySUBJECT.Engine := self.PythonEngine1; PySUBJECT.VarName := 'SUBJECT'; PySUBJECT.Initialize; finally PySUBJECT.Finalize; PySUBJECT.Free; end;

Thank you for your quick and elaborate answer. The py_subject was indeed freed at the wrong place. In case someone else wonders the following: There are several funtions in Python that are non-blocking. For example: sleep urlopen

The scripter works after installing python3.8-venv when setting the dllname to 'libptyhon3.6m.so.1.0'. The libptyhon3.8 files are neither found via p4d nor via locate libpython3. So there must be still an issue with sharing the 3.8 version on ubuntu. Installation command: sudo apt-get install python3.8-venv https://dev.to/mortoray/how-to-install-python-3-8-on-ubuntu-1bp4