True thread parallelism with P4D and Python 12

[pyscripter 689]

In case you have not heard, Python 12 made a breakthrough towards running threads in parallel.  For now, the feature is only available through the C-API and there are limitations.   But still, the performance boost that comes from the exploitation of multiple cores makes this feature very desirable.  P4D makes this feature very easily accessible by adding a new execution mode to TPythonThread.  There is also a new demo (Demo 36) that shows how to run threads in parallel.  If you are wondering about the performance boost, see the results below:

 

Classic Subinterpreters:
prime count 78498
prime count 78498
prime count 78498
prime count 78498
prime count 78498
Elapsed ms: 13695

Subinterpreters with own GIL:
prime count 78498
prime count 78498
prime count 78498
prime count 78498
prime count 78498
Elapsed ms: 3482

You can find more details in this P4D announcement.

Edited October 30, 2023 by pyscripter

[Sherlock 663]

Noice! And which breaking changes happened this time? Python is sweet when you need short scripts but a pain for large projects. I would like to see any one of our colleagues here with their million LOC projects using Python.

[David Heffernan 2345]

3 hours ago, Sherlock said:

Noice! And which breaking changes happened this time? Python is sweet when you need short scripts but a pain for large projects. I would like to see any one of our colleagues here with their million LOC projects using Python.

Python is very good at managing change in my experience. 

[Fr0sT.Brutal 900]

21 hours ago, Sherlock said:

I would like to see any one of our colleagues here with their million LOC projects using Python.

Django, Numpy

However compiled language with strict typing is always better for big projects

[pyscripter 689]

Let's not start language wars in this thread, which is about a significant development in Python 12 and how take advantage of it using P4D.   P4D is about combining the strengths of python and Delphi, not choosing one versus the other.

Edited November 1, 2023 by pyscripter

[Sherlock 663]

 

1 hour ago, pyscripter said:

Let's not start language wars

You are absolutely right. Sorry about that.

[David Heffernan 2345]

I'm quite excited about escaping GIL tyrrany but shared global state in the python extension module I develop is going to be tricky. 

 

I'm looking forward to seeing what you've done here and drawing some inspiration!! 

[shineworld 73]

4 hours ago, pyscripter said:

Let's not start language wars in this thread, which is about a significant development in Python 12 and how take advantage of it using P4D.   P4D is about combining the strengths of python and Delphi, not choosing one versus the other.

This was my case, in which the main UI and image analysis, with OpenCV, was managed with Python + DelphiVCL, and faster IP Camera commands and frames receiving were managed by a Delphi P4D, in this case, D4P, module using native Threads of Delphi.

The attempt to manage frame acquisition via socket with python threads failed miserably.

The combo was winning !!!

I the future of Python I will be happy to have more power on threads and cores.

Edited November 1, 2023 by shineworld