Celebr0

Hi, I encountered the same problem. I need to run a ready-made .py script in several windows/threads simultaneously, while also setting launch parameters and reading input/output for further parsing. I've been struggling for 2 years now and can't do it. Probably the Python4Delphi library only works in a single thread. I also tried to make my own simplified module for working with the library: unit pythonmodule; interface uses System.SysUtils, System.IOUtils, System.Types, System.StrUtils, System.Classes, PythonEngine; procedure InitPython(const Data: TSendUniDataEvent; const Path: string); procedure ExecPythonFile(const FFile, Params:string); procedure FreePython; function CommandLineToArgvW(lpCmdLine: PWideChar; out pNumArgs: Integer): PPWideChar; stdcall; external 'shell32.dll'; function LocalFree(hMem: Pointer): Pointer; stdcall; external 'kernel32.dll'; var PythonEngine: TPythonEngine; script: string=''; implementation procedure InitPython(const Data: TSendUniDataEvent; const Path: string); var pyio:TPythonInputOutput; begin script:=TFile.ReadAllText(Path + '\script.py'); pyio := TPythonInputOutput.Create(nil); pyio.UnicodeIO := True; pyio.DelayWrites := False; pyio.OnSendUniData := Data; PythonEngine := TPythonEngine.Create(nil); PythonEngine.IO := pyio; try PythonEngine.UseLastKnownVersion := False; PythonEngine.DllPath := ExtractFilePath(ParamStr(0)) + 'python'; PythonEngine.DllName := 'python314.dll'; PythonEngine.AutoLoad := False; PythonEngine.AutoFinalize := False; PythonEngine.AutoUnload := False; PythonEngine.RedirectIO := True; PythonEngine.UseWindowsConsole:=False; PythonEngine.InitScript.Text:='import sys'#13#10+'sys.path.append("'+Path+'")'; PythonEngine.LoadDll; TPythonThread.Py_Begin_Allow_Threads; finally // end; end; procedure ExecPythonFile(const FFile, Params:string); const Arg = 'import sys, shlex' + #13#10 + 'sys.argv = ["script.py"] + shlex.split("'; var ArgCount: integer; ArgList: PPWideChar; config: PyConfig; FGILState: PyGILstate_STATE; begin //ArgList:=CommandLineToArgvW(PWideChar(FFile+' '+Params), ArgCount); //FGILState := PythonEngine.PyGILState_Ensure; try PythonEngine.ExecString(Arg + StringReplace(Params, '"', '\"', [rfReplaceAll]) + '")'); //PythonEngine.ExecStrings(TStrings(script)); PythonEngine.ExecFile(FFile); finally //PythonEngine.PyGILState_Release(FGILState); end; end; procedure FreePython; begin TPythonThread.Py_End_Allow_Threads; FreeAndNil(PythonEngine); end; end. I'm trying: procedure TForm2.OnData(Sender: TObject; const Data: string); begin TPythonThread.Synchronize(nil, procedure begin Form2.Memo1.Lines.Add(Data); end); end; procedure TForm2.Button2Click(Sender: TObject); var pt:TPythonThread; begin home:=ExtractFilePath(ParamStr(0))+'myscript'; InitPython(OnData, home); pt.CreateAnonymousThread( procedure begin pt.ThreadExecMode:=emNewInterpreterOwnGIL; ExecPythonFile(home + '\script.py', '-n -m -o "result.txt"'); end).Start; end; end;

Hello, I think I checked everything 7 times, I seem to be doing everything correctly according to the code, but I get an error: Subinterpreter with own GIL: Traceback (most recent call last): File "<string>", line 1, in <module> NameError: name 'HW' is not defined My Code: program ParallelPython; {$APPTYPE CONSOLE} uses System.SysUtils, System.Diagnostics, System.Variants, System.SyncObjs, PythonEngine; var PythonEngine: TPythonEngine; procedure CreatePyEngine; begin PythonEngine := TPythonEngine.Create(nil); PythonEngine.InitScript.Text:='HW="Hello World!"'; PythonEngine.Name := 'PythonEngine'; PythonEngine.IO:=nil; PythonEngine.RedirectIO:=False; PythonEngine.LoadDll; TPythonThread.Py_Begin_Allow_Threads; end; procedure DestroyEngine; begin TPythonThread.Py_End_Allow_Threads; PythonEngine.Free; end; type TPyThread = class(TPythonThread) protected procedure ExecuteWithPython; override; public constructor Create(ThreadMode: TThreadExecMode); end; procedure TPyThread.ExecuteWithPython; begin writeln(PythonEngine.EvalStringAsStr('HW')); end; constructor TPyThread.Create(ThreadMode: TThreadExecMode); begin inherited Create; ThreadExecMode := ThreadMode; FreeOnTerminate := True; end; var I: Integer; begin try CreatePyEngine; try WriteLn('Subinterpreter with own GIL:'); TPyThread.Create(emNewInterpreterOwnGIL); finally //Sleep(10); // allow some time for the threads to terminate //DestroyEngine; end; except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; ReadLn; end.

Hello, the thing is that Python has something that Delphi does not have and will never have, for example, Tensorflow AI and so on.

Hi, for some reason, I'm experiencing massive memory leaks when reading a file and parallelizing by lines procedure ReadF(const fname: string); const BUFSIZE = 1024 * 18; var Line:string; List: TStringList; Reader: TStreamReader; Buf: array [0..BUFSIZE] of char; begin List := TStringList.Create; Reader := TStreamReader.Create(fname, TEncoding.Default, True, 4096); while not Reader.EndOfStream do begin Reader.ReadBlock(@Buf, 0, BUFSIZE); List.Text := Buf; parallel.&For(0, List.Count - 1).Execute( procedure(value: integer) begin Line:=List[value]; end); end; List.Free; Reader.Close; FreeAndNil(Reader); end;

Hi, where and to whom have I been disrespectful, arrogant, or insulting? Why are you blatantly lying about it ?

No, it looks like you don't understand multithreaded development at all if you're suggesting using critical sections. If you put critical sections and threads have to wait for each other every time, what's the point of using threads then?

How much are you betting ?

I checked and solved this issue, and yes, that is indeed the case: Memory Leak: procedure ReadF(const fname: string); const BUFSIZE = 1024 * 18; var Line:string; List: TStringList; Reader: TStreamReader; Buf: array [0..BUFSIZE] of char; begin List := TStringList.Create; Reader := TStreamReader.Create(fname, TEncoding.Default, True, 4096); while not Reader.EndOfStream do begin Reader.ReadBlock(@Buf, 0, BUFSIZE); List.Text := Buf; parallel.ForEach(0, List.Count - 1).Execute( procedure(value: integer) begin Line:=List[value]; end); end; List.Free; Reader.Close; FreeAndNil(Reader); end; No memory Leak: procedure ReadF(const fname: string); const BUFSIZE = 1024 * 18; var Line:string; List: TStringList; Reader: TStreamReader; Buf: array [0..BUFSIZE] of char; begin List := TStringList.Create; Reader := TStreamReader.Create(fname, TEncoding.Default, True, 4096); while not Reader.EndOfStream do begin Reader.ReadBlock(@Buf, 0, BUFSIZE); List.Text := Buf; parallel.ForEach(0, List.Count - 1).NoWait.Execute( procedure(value: integer) begin Line:=List[value]; end); end; List.Free; Reader.Close; FreeAndNil(Reader); end;

I think I figured it out when I was writing my own module for parallelizing the for loop, this is what I encountered: 1. There will be memory leaks: procedure ParallelFor(const AStart, AEnd: Integer; AProc: TProc<Integer>); var i, ThreadCount, RangeStart: Integer; begin ThreadCount:=GetCPUCount; RangeStart:=AStart-1; for i := 0 to ThreadCount - 1 do WaitForSingleObject(TThread.CreateAnonymousThread( procedure begin while RangeStart < AEnd do begin AProc(InterlockedIncrement(RangeStart)); end; end).Create(False).Handle, INFINITE); end; 2. There will be no memory leaks here: procedure ParallelFor(const AStart, AEnd: Integer; AProc: TProc<Integer>); var i, ThreadCount, RangeStart: Integer; begin ThreadCount:=GetCPUCount; RangeStart:=AStart-1; for i := 0 to ThreadCount - 1 do TThread.CreateAnonymousThread( procedure begin while RangeStart < AEnd do begin AProc(InterlockedIncrement(RangeStart)); end; end).Create(False); end; It looks like OtlParallel.For is written based on the same principle as in the first case !

Delphi XE2 I wrote my own module for parallelizing threads using the same code as here, and there are no memory leaks at all!

Working with memory, since it is fast, is always absolutely thread-safe. However, working with form1 for example Memo1, ListBox, and so on is not thread-safe. If you are implying that I should wrap Line := List[value]; in a critical section, it makes absolutely no difference—memory leaks still remain.

Who told you such nonsense? The variable will just be overwritten!

Hello, after the last update 5 days ago, python4delphi crashes immediately after launch: Delphi XE2

Hello, the problem has not yet been resolved. I downloaded the newest version of Python4Delphi from GitHub and Python 3.13.0, but it still crashes

Hello, I just started studying the OmniThreadLibrary library, but I can’t figure out how to parallelize this Regex procedure ? procedure pars(const Regex: TRegEx; const str: string); var Match: TMatch; begin Match := Regex.Match(str); //Parallel.ForEach(???).Execute( ??????????????? while Match.Success do begin //Match.Value; Match := Match.NextMatch; end; end;

That even Regex cannot be accelerated in any way through OmniLibrary ?

I guess I found something CRASH in AssignPyFlags(Config); from here: var i : Integer; Config: PyConfig; Status: PyStatus; ErrMsg: string; begin if Assigned(gPythonEngine) then raise Exception.Create('There is already one instance of TPythonEngine running' ); gPythonEngine := Self; FIORedirected := False; if FInExtensionModule then FInitialized := True else begin // Fills Config with zeros and then sets some default values if pfIsolated in FPyFlags then PyConfig_InitIsolatedConfig(Config) else PyConfig_InitPythonConfig(Config); try AssignPyFlags(Config); // CRASH HERE // Set programname and pythonhome if available if FProgramName <> '' then PyConfig_SetString(Config, PPWcharT(PByte(@Config) + ConfigOffests[MinorVersion, TConfigFields.program_name]), PWCharT(StringToWCharTString(FProgramName))); if FPythonHome <> '' then PyConfig_SetString(Config, PPWcharT(PByte(@Config) + ConfigOffests[MinorVersion, TConfigFields.home]), PWCharT(StringToWCharTString(FPythonHome))); // Set venv executable if available if FVenvPythonExe <> '' then PyConfig_SetString(Config, PPWcharT(PByte(@Config) + ConfigOffests[MinorVersion, TConfigFields.executable]), PWCharT(StringToWCharTString(FVenvPythonExe))); // Set program arguments (sys.argv) SetProgramArgs(Config); // PythonPath SetPythonPath(Config);

I don’t understand how and where initialization is performed, you should know this

Hello debugger brought me here: procedure TPythonEngine.AfterLoad; begin inherited; Initialize; end; If I comment out Initialize; then there is no crash

I can’t debug it because it’s crashing here please fix it somehow or try a virtual machine with windows 7 + delphi XE2

I was wondering if this error could be related to the fact that I am using Python under Windows 7 https://github.com/adang1345/PythonWin7 ? Although this version https://github.com/pyscripter/python4delphi/commit/46cd7a66dd9a249b3bc6bac9c2be611f2bde2069 python4delphi works successfully with this !

Yes, I was really mistaken and the error is not where I previously thought the error was with LoadDll: procedure CreatePyEngine; begin PythonEngine := TPythonEngine.Create(nil); PythonEngine.LoadDll; // CRASH HERE //TPythonThread.Py_Begin_Allow_Threads; end;

No, the problem is not in demo36, but in pythonengine.pas itself, I tried my ready-made projects, which worked and are now working with previous versions of python4delphi. The problem most likely manifested itself after editing: Paths := string(FPythonPath).Split([PathSep]); on Paths := SplitString(string(FPythonPath), PathSep); When previous versions did not work because of this syntax, I myself tried to fix it on SplitString and immediately such crashes appeared last work version from Delphi XE2 https://github.com/pyscripter/python4delphi/commit/46cd7a66dd9a249b3bc6bac9c2be611f2bde2069 UPD: Most likely I think that there is an incorrect conversion here: PyWideStringList_Append(PWSL, PWCharT(StringToWCharTString(Paths))); Please study this demo TStringDynArray is not friendly with such transformations, the output turns out to be crooked program Project2; {$APPTYPE CONSOLE} uses System.SysUtils, System.StrUtils, System.Types; var s:string; ar:TStringDynArray; begin s:='python4 delphi'; ar:=splitstring(s,' '); writeln(PansiChar(ar)); writeln(PChar(ar)); writeln(PansiChar(ansistring(ar))); readln; end.

1. https://github.com/pyscripter/python4delphi 2. Python version 3.12.1 3. Demo36

Hello, I am faced with a problem that I need to run one .py script in several instances, but I can’t achieve this through python4delphi ( I'm trying to do it like this, but since the script has been running for some time, the program just crashes: program ParallelPython; {$APPTYPE CONSOLE} {$R *.res} uses System.SysUtils, System.Diagnostics, System.Variants, System.SyncObjs, PythonEngine; var PythonEngine: TPythonEngine; type TPyThread = class(TPythonThread) protected procedure ExecuteWithPython; override; public constructor Create(createsuspended:boolean); end; procedure CreatePyEngine; begin PythonEngine := TPythonEngine.Create(nil); PythonEngine.LoadDll; TPythonThread.Py_Begin_Allow_Threads; end; procedure DestroyEngine; begin TPythonThread.Py_End_Allow_Threads; PythonEngine.Free; end; procedure TPyThread.ExecuteWithPython; const Arg = 'import sys'+#13#10+ 'def run_python_script(args_str):'+#13#10+ ' args_list = args_str.split()'+#13#10+ ' sys.argv = [sys.argv[0]] + args_list'; begin inherited; while true do begin PythonEngine.ExecString(Arg); PythonEngine.ExecFile(extractfilepath(paramstr(0))+'script.py'); end; end; constructor TPyThread.Create(createsuspended:boolean); begin inherited Create(CreateSuspended); ThreadExecMode := emNewInterpreterOwnGIL; FreeOnTerminate := True; end; var I: Integer; begin try CreatePyEngine; for I := 1 to 10 do TPyThread.Create(False); finally //DestroyEngine; end; ReadLn; end. And it probably also crashes because the script is trying to be executed in one interpreter (