Celebr0

Hi, I encountered the same problem. I need to run a ready-made .py script in several windows/threads simultaneously, while also setting launch parameters and reading input/output for further parsing. I've been struggling for 2 years now and can't do it. Probably the Python4Delphi library only works in a single thread. I also tried to make my own simplified module for working with the library: unit pythonmodule; interface uses System.SysUtils, System.IOUtils, System.Types, System.StrUtils, System.Classes, PythonEngine; procedure InitPython(const Data: TSendUniDataEvent; const Path: string); procedure ExecPythonFile(const FFile, Params:string); procedure FreePython; function CommandLineToArgvW(lpCmdLine: PWideChar; out pNumArgs: Integer): PPWideChar; stdcall; external 'shell32.dll'; function LocalFree(hMem: Pointer): Pointer; stdcall; external 'kernel32.dll'; var PythonEngine: TPythonEngine; script: string=''; implementation procedure InitPython(const Data: TSendUniDataEvent; const Path: string); var pyio:TPythonInputOutput; begin script:=TFile.ReadAllText(Path + '\script.py'); pyio := TPythonInputOutput.Create(nil); pyio.UnicodeIO := True; pyio.DelayWrites := False; pyio.OnSendUniData := Data; PythonEngine := TPythonEngine.Create(nil); PythonEngine.IO := pyio; try PythonEngine.UseLastKnownVersion := False; PythonEngine.DllPath := ExtractFilePath(ParamStr(0)) + 'python'; PythonEngine.DllName := 'python314.dll'; PythonEngine.AutoLoad := False; PythonEngine.AutoFinalize := False; PythonEngine.AutoUnload := False; PythonEngine.RedirectIO := True; PythonEngine.UseWindowsConsole:=False; PythonEngine.InitScript.Text:='import sys'#13#10+'sys.path.append("'+Path+'")'; PythonEngine.LoadDll; TPythonThread.Py_Begin_Allow_Threads; finally // end; end; procedure ExecPythonFile(const FFile, Params:string); const Arg = 'import sys, shlex' + #13#10 + 'sys.argv = ["script.py"] + shlex.split("'; var ArgCount: integer; ArgList: PPWideChar; config: PyConfig; FGILState: PyGILstate_STATE; begin //ArgList:=CommandLineToArgvW(PWideChar(FFile+' '+Params), ArgCount); //FGILState := PythonEngine.PyGILState_Ensure; try PythonEngine.ExecString(Arg + StringReplace(Params, '"', '\"', [rfReplaceAll]) + '")'); //PythonEngine.ExecStrings(TStrings(script)); PythonEngine.ExecFile(FFile); finally //PythonEngine.PyGILState_Release(FGILState); end; end; procedure FreePython; begin TPythonThread.Py_End_Allow_Threads; FreeAndNil(PythonEngine); end; end. I'm trying: procedure TForm2.OnData(Sender: TObject; const Data: string); begin TPythonThread.Synchronize(nil, procedure begin Form2.Memo1.Lines.Add(Data); end); end; procedure TForm2.Button2Click(Sender: TObject); var pt:TPythonThread; begin home:=ExtractFilePath(ParamStr(0))+'myscript'; InitPython(OnData, home); pt.CreateAnonymousThread( procedure begin pt.ThreadExecMode:=emNewInterpreterOwnGIL; ExecPythonFile(home + '\script.py', '-n -m -o "result.txt"'); end).Start; end; end;

Hello, the thing is that Python has something that Delphi does not have and will never have, for example, Tensorflow AI and so on.

Hello, I think I checked everything 7 times, I seem to be doing everything correctly according to the code, but I get an error: Subinterpreter with own GIL: Traceback (most recent call last): File "<string>", line 1, in <module> NameError: name 'HW' is not defined My Code: program ParallelPython; {$APPTYPE CONSOLE} uses System.SysUtils, System.Diagnostics, System.Variants, System.SyncObjs, PythonEngine; var PythonEngine: TPythonEngine; procedure CreatePyEngine; begin PythonEngine := TPythonEngine.Create(nil); PythonEngine.InitScript.Text:='HW="Hello World!"'; PythonEngine.Name := 'PythonEngine'; PythonEngine.IO:=nil; PythonEngine.RedirectIO:=False; PythonEngine.LoadDll; TPythonThread.Py_Begin_Allow_Threads; end; procedure DestroyEngine; begin TPythonThread.Py_End_Allow_Threads; PythonEngine.Free; end; type TPyThread = class(TPythonThread) protected procedure ExecuteWithPython; override; public constructor Create(ThreadMode: TThreadExecMode); end; procedure TPyThread.ExecuteWithPython; begin writeln(PythonEngine.EvalStringAsStr('HW')); end; constructor TPyThread.Create(ThreadMode: TThreadExecMode); begin inherited Create; ThreadExecMode := ThreadMode; FreeOnTerminate := True; end; var I: Integer; begin try CreatePyEngine; try WriteLn('Subinterpreter with own GIL:'); TPyThread.Create(emNewInterpreterOwnGIL); finally //Sleep(10); // allow some time for the threads to terminate //DestroyEngine; end; except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; ReadLn; end.

Hi, where and to whom have I been disrespectful, arrogant, or insulting? Why are you blatantly lying about it ?

No, it looks like you don't understand multithreaded development at all if you're suggesting using critical sections. If you put critical sections and threads have to wait for each other every time, what's the point of using threads then?

How much are you betting ?

I checked and solved this issue, and yes, that is indeed the case: Memory Leak: procedure ReadF(const fname: string); const BUFSIZE = 1024 * 18; var Line:string; List: TStringList; Reader: TStreamReader; Buf: array [0..BUFSIZE] of char; begin List := TStringList.Create; Reader := TStreamReader.Create(fname, TEncoding.Default, True, 4096); while not Reader.EndOfStream do begin Reader.ReadBlock(@Buf, 0, BUFSIZE); List.Text := Buf; parallel.ForEach(0, List.Count - 1).Execute( procedure(value: integer) begin Line:=List[value]; end); end; List.Free; Reader.Close; FreeAndNil(Reader); end; No memory Leak: procedure ReadF(const fname: string); const BUFSIZE = 1024 * 18; var Line:string; List: TStringList; Reader: TStreamReader; Buf: array [0..BUFSIZE] of char; begin List := TStringList.Create; Reader := TStreamReader.Create(fname, TEncoding.Default, True, 4096); while not Reader.EndOfStream do begin Reader.ReadBlock(@Buf, 0, BUFSIZE); List.Text := Buf; parallel.ForEach(0, List.Count - 1).NoWait.Execute( procedure(value: integer) begin Line:=List[value]; end); end; List.Free; Reader.Close; FreeAndNil(Reader); end;

I think I figured it out when I was writing my own module for parallelizing the for loop, this is what I encountered: 1. There will be memory leaks: procedure ParallelFor(const AStart, AEnd: Integer; AProc: TProc<Integer>); var i, ThreadCount, RangeStart: Integer; begin ThreadCount:=GetCPUCount; RangeStart:=AStart-1; for i := 0 to ThreadCount - 1 do WaitForSingleObject(TThread.CreateAnonymousThread( procedure begin while RangeStart < AEnd do begin AProc(InterlockedIncrement(RangeStart)); end; end).Create(False).Handle, INFINITE); end; 2. There will be no memory leaks here: procedure ParallelFor(const AStart, AEnd: Integer; AProc: TProc<Integer>); var i, ThreadCount, RangeStart: Integer; begin ThreadCount:=GetCPUCount; RangeStart:=AStart-1; for i := 0 to ThreadCount - 1 do TThread.CreateAnonymousThread( procedure begin while RangeStart < AEnd do begin AProc(InterlockedIncrement(RangeStart)); end; end).Create(False); end; It looks like OtlParallel.For is written based on the same principle as in the first case !

Delphi XE2 I wrote my own module for parallelizing threads using the same code as here, and there are no memory leaks at all!

Working with memory, since it is fast, is always absolutely thread-safe. However, working with form1 for example Memo1, ListBox, and so on is not thread-safe. If you are implying that I should wrap Line := List[value]; in a critical section, it makes absolutely no difference—memory leaks still remain.

Who told you such nonsense? The variable will just be overwritten!

Hi, for some reason, I'm experiencing massive memory leaks when reading a file and parallelizing by lines procedure ReadF(const fname: string); const BUFSIZE = 1024 * 18; var Line:string; List: TStringList; Reader: TStreamReader; Buf: array [0..BUFSIZE] of char; begin List := TStringList.Create; Reader := TStreamReader.Create(fname, TEncoding.Default, True, 4096); while not Reader.EndOfStream do begin Reader.ReadBlock(@Buf, 0, BUFSIZE); List.Text := Buf; parallel.&For(0, List.Count - 1).Execute( procedure(value: integer) begin Line:=List[value]; end); end; List.Free; Reader.Close; FreeAndNil(Reader); end;

Hello, the problem has not yet been resolved. I downloaded the newest version of Python4Delphi from GitHub and Python 3.13.0, but it still crashes

That even Regex cannot be accelerated in any way through OmniLibrary ?

Hello, I just started studying the OmniThreadLibrary library, but I can’t figure out how to parallelize this Regex procedure ? procedure pars(const Regex: TRegEx; const str: string); var Match: TMatch; begin Match := Regex.Match(str); //Parallel.ForEach(???).Execute( ??????????????? while Match.Success do begin //Match.Value; Match := Match.NextMatch; end; end;