Soulflesh

Hi uefi, have a look at Running Python scripts in threads - Python4Delphi - Delphi-PRAXiS [en] (delphipraxis.net). Unfortunately, this still does not work in the form of true parallelism at the moment. Only the pseudo Python thread parallelism. From my point of view, there is currently only the possibility/attempt to multiply the Python dlls and load them per thread. Bets regards

Hi FabDev, In my estimation, this is not a good idea due to the lack of parallelisation with Python. We also use the TMS Scripter for web applications (f.e. https://help.clarc-software.de/display/CD7/WEB+VALIDATION+Arbeitsbereich). Here, however, you also reach the limits very quickly as soon as you use the generic classes in the scripter. These require a lot of performance and about 1MB per object instance. We have therefore transferred a lot to the client/browser. We were able to achieve acceptable results with this. Best regards

Hi, I have a problem running Python scripts in threads. The execution stucks in TPythonThread.Execute --> gilstate := PyGILState_Ensure(); and never return. Here is my example console application. program p4dtest; {$APPTYPE CONSOLE} {$R *.res} uses System.SysUtils, System.Classes, PythonEngine; type TMyPythonThread = class(TPythonThread) protected procedure ExecuteWithPython; override; public constructor Create; end; TMyServiceHelper = class public ThreadCount : Integer; public procedure ThreadDone(Sender: TObject); end; { TMyPythonThread } constructor TMyPythonThread.Create; begin Self.ThreadExecMode := emNewInterpreter; Self.FreeOnTerminate := true; inherited Create(true); end; procedure TMyPythonThread.ExecuteWithPython; begin Writeln('Test1'); GetPythonEngine.ExecString('print("Test2")'); end; { TMyServiceHelper } procedure TMyServiceHelper.ThreadDone(Sender: TObject); begin Dec(ThreadCount); end; var MyPythonEngine : TPythonEngine; MyPyInOut : TMyPythonGUIInputOutput; MySvcHelper : TMyServiceHelper; Threads : Array of TMyPythonThread; i : Integer; begin try MyPythonEngine := TPythonEngine.Create(nil); MySvcHelper := TMyServiceHelper.Create; try MyPythonEngine.InitThreads := true; MyPythonEngine.LoadDll; MySvcHelper.ThreadCount := 1; SetLength(Threads,1); for i:=0 to 0 do begin Threads[i] := TMyPythonThread.Create; Threads[i].OnTerminate := MySvcHelper.ThreadDone; Threads[i].Start; end; repeat CheckSynchronize; sleep(10); until MySvcHelper.ThreadCount = 0; Writeln('Done'); ReadLn; finally FreeAndNil(MyPythonEngine); FreeAndNil(MySvcHelper); end; except on E: Exception do begin Writeln(E.ClassName, ': ', E.Message); ReadLn; end; end; end. Any ideas? Best regards.

Hello Harald, Unfortunately, I have not yet found a way to implement this. I think at the moment the only possibility would be, as I wrote at the time, to multiply the dlls and then load them several times so that you have an independent dll environment for each thread. However, this would probably mean quite a lot of effort to rewrite this in the core libraries. Many greetings

My topic is more in the sense of a web service, whereby it is more a matter of the overall speed and less of that of individual processes. In my view, true parallelism is the only way here.

Additional question: 😀 I did some performance tests and unfortunately came to the expected result that normal thread processing does not bring any significant performance benefits. For my test I used the Fibonacci algorithm to generate a high CPU load and a low IO load. Are there ways, apart from additional processes, to improve the performance of parallel processing? I am talking here in the context of processing a wide variety of scripts or problems that cannot be parallelized themselves. Is there a possibility to use Python derivatives like CPython or IronPython without or with a disableableable GIL? From another area of resource sharing of dll libraries, a viable way was to duplicate the dll's and load them multiple times. Would this also be a possible approach here with Python and Delphi? Best regards

Thank you very much. That solves my issue!

Unfortunately I cannot find anything like "PyBeginAllowThreads" or "AllowThreads" in all sources... 🤔

Hi, unfortunately it's not possible to compile PythonEngine.pas for Linux64 with Delphi 10.4. [DCC Fehler] PythonEngine.pas(198): E2003 Undeklarierter Bezeichner: 'PUCSChar' [DCC Fehler] PythonEngine.pas(199): E2007 Konstante oder Typenbezeichner erwartet [DCC Fehler] PythonEngine.pas(4396): E2010 Inkompatible Typen: 'Dynamic array' und 'UCS4String' Any ideas? Best regards

Thank you!

Hi pyscripter, Unfortunately a wrong character has crept into the code in the PythonEngine.pas in line 4396 for the "I": Best regards

This raises an exception: SetLength(Test,0); Result := UCS4StringToUnicodeString(Test); Your suggested code has solved the problem! 😀 Python Scripts runs now on Linux (Ubuntu 18.04.5 LTS with Python 3.9.6). Best regards

The exception is raised in the UCS4StringToUnicodeString function. Called by function TPythonEngine.GetProgramName: UnicodeString; And FProgramName is "{...}" ??!

Unfortunately now I get a "range check" exception here: [PythonEngine.pas / Line 4219] if Assigned(Py_SetProgramName) and (ProgramName <> '') then Py_SetProgramName(PWCharT(FProgramName)); My "configuration" is: PythonEngine.UseLastKnownVersion := false; PythonEngine.DllPath := '/usr/lib/x86_64-linux-gnu'; PythonEngine.DllName := 'libpython3.9.so.1.0'; Any ideas? Best regards

Oh yes!! Sorry.

Hi pyscripter, Thank you. I have additionally changed the following lines and now I'm able to compile: 199 --> PPWCharT = ^PUCS4Char; 4396 --> WL := UnicodeStringToUCS4String(TempS); Best regards