DelphiVCL: Fantastic! But there are some issues

[Reinier Sterkenburg 3]

Let me first say that I am very enthusiastic about Python4Delphi.
I also find the possibility to create a DelphiVCL dll (.pyd) that makes 'the whole VCL' available in Python quite spectacular!

 

I am having a few issues with it though, and before opening issues in GitHub I'd like to put a post here to see if others have encountered these as well and what they do about it.

First of all, I tried DelphiVCL as found under Modules\DelphiVCL. That worked nicely out of the box, at least after choosing the right Python dll.

(perhaps we can use UseLastKnownVersion there as well?).

 

What doesn't work well is closing the application.

Whether I run it from Spyder, from PyCharm or from PyScripter, just pressing the close button does not make the app close.

In Spyder it takes a restart of the kernel, in PyCharm after a few tries, the program can't started at all anymore and in PyScripter it takes Ctrl-F2. 

Any ideas where this comes from?

 

Then some more VCL related issues.

I wanted to use a ComboBox so I added one to my form.

Setting its Style to csDropDownlist worked, after I had added code to register the TComboBoxStyle constants to WrapDelphiStdCtrls.

However, the ComboBox now works a bit strange: the down arrow that should make the items list appear is gone.

Another really strange thing that I encountered is that string items that I have added to ComboBoxes get converted to lowercase.

Any ideas?

Shall I added these issues in GitHub?

Shall I make a PR for WrapDelphiStdCtrls (and add all the enumeration constants from StdCtrls)?

 

[Anders Melander 1784]

Hey Reinier!

I think it must be over 20 years since I last heard from you. AFAIR you were the original maintainer of The Delphi Bug List.

Ah, yes: https://web.archive.org/web/20001025065121/http://www.dataweb.nl/~r.p.sterkenburg/indexpag.htm

 

Good to see you're still around.

[pyscripter 689]

Please file appropriate bug reports and open PRs,

 

On note regarding WrapDelphi:

The WrapDelphiXYZ units were written before the enhanced RTTI was implemented.  Nowadays you do not need to wrap enumerations sets and the like.  Here is an example from WrapDelphi unit tests:

 

type
  TFruit = (Apple, Banana, Orange);
  TFruits = set of TFruit;

  TTestRttiAccess = class
  private
    FFruit: TFruit;
    FFruits: TFruits;
  public
    FruitField :TFruit;
    FruitsField: TFruits;
    StringField: string;
    DoubleField: double;
    ObjectField: TObject;
    RecordField: TTestRecord;
    InterfaceField: ITestInterface;
    procedure BuyFruits(AFruits: TFruits);
    property Fruit: TFruit read FFruit write FFruit;
    property Fruits: TFruits read FFruits write FFruits;
  end;

 

With the above declarations and if you Wrap a variable say test of type TTestRttiAccess you can write python code such as:

 

test.Fruit = 'Apple'
test.Fruits = ['Apple', 'Banana']

 

 

So there is no need to create special wrappers of enumerations and sets.

 

The WrapDelphi unit tests are worth studying in some depth.

 

Edited October 24, 2020 by pyscripter

[pyscripter 689]

6 hours ago, Reinier Sterkenburg said:

 

What doesn't work well is closing the application.

Did you try handling the Form OnClose event or use Release instead of Close?

Edited October 24, 2020 by pyscripter

[Reinier Sterkenburg 3]

9 hours ago, Anders Melander said:

Hey Reinier!

I think it must be over 20 years since I last heard from you. AFAIR you were the original maintainer of The Delphi Bug List.

Ah, yes: https://web.archive.org/web/20001025065121/http://www.dataweb.nl/~r.p.sterkenburg/indexpag.htm

 

Good to see you're still around.

Hi Anders,

Yes I'm still around 🙂 and still using Delphi. Yeah, the Delphi Bug List, that as a nice project, which got so much better when you started helping.

Likewise, good to see you are still here as well!

[Reinier Sterkenburg 3]

6 hours ago, pyscripter said:

...

With the above declarations and if you Wrap a variable say test of type TTestRttiAccess you can write python code such as:

 

test.Fruit = 'Apple'
test.Fruits = ['Apple', 'Banana']

 

So there is no need to create special wrappers of enumerations and sets.

 

The WrapDelphi unit tests are worth studying in some depth.

 

Thanks for that answer!

Indeed, I could do 

 

    myComboBox.Style = 'csDropDownList'

 

and now it works as expected.

Also the items don't get converted to lowercase anymore.

 

..

Now that this is cleared up, I'd like to propose that this gets documented a bit better.

I made this mistake because I started with the DelphiVCL demo and drew a wrong conclusion from it.

I saw the Action.Value = caFree assignment in the OnClose handler.

caFree is an enumerated constant which was apparently imported from DelphiVCL.

However, csDropDownList was NOT available from DelphiVCL.

So I looked at how caFree was made available and found it in TFormsRegistration.DefineVars in WrapDelphiForms.

Following that same pattern, I added code in WrapDelphiStdCtrls to define csDropDownList. 

So far so good, and my code compiled and ran without (explicit) errors.

But it was wrong.

Actually, ComboBox.Style is a string property when imported from DelphiVCL, and Python did not complain when I assigned an integer/enumerated.

 

To avoid confusion, it would probably be better if caFree was also a string constant. 

Is that feasible, to drop all the explicit DefineVars in WrapDelphiForms and other units and let *all* the enumeration values become strings through RTTI?

(would that even work? I didn't understand yet how TComboBox.Style became a string property instead of an integer)

 

In any case, we could also add a little bit of extra comment in the DelphiVCL demo sources that explains this.

Edited October 24, 2020 by Reinier Sterkenburg

[Reinier Sterkenburg 3]

6 hours ago, pyscripter said:

Did you try handling the Form OnClose event or use Release instead of Close?

I did try a few things but can't get the form to close.

When you (or anyone else reading this) try the DelphiVCL demo as is, can you close the application normally?

[pyscripter 689]

The application terminates fine when run from the command line.  You are right that when is run from PyScripter the form does not close.  I am not fully sure, but it may have to do with the fact that PyScripter prevents the script from terminating so that the state of the interpreter can be examined after running a script.

 

However you can hide the form by adding f.Hide() as the last statement of the main procedure.

[Guest]

I can't run that demo as it complain about Python version, and i am not ready to mess with my setup, so i am sorry if my input on this in theory, which will sound like guess work.

 

So the points are

1) Delphi application like any application will terminate and exit with ExitProcess, but we have a DLL here, so what is the default behaviour to exit, no DLL should be allowed to call ExitProcess, right ?

2) DLL should notify or signal its parent process to do the free, exit, cleanup etc..   i think this is the missing link here

3) It would be better to hook ExitProcessProc, by hooking i mean just fill it to make sure that DLL is not calling Application.Terminate and if it does then you use (2) to clean up and free that module/DLL.

 

From all the above, i suggest to investigate the following 

1) You should pass callback function to the module to be called when it need to terminate/exit.

2) have one extra function like "PyFree_[ProjectName]" or "PyExit_[ProjectName]"...  to call cleanup in the module, This might be not so essential, but you can think of a use for it when the DLL specially with VCL need to be forced to exit from python the container, like timed execution to prevent freezing...

 

Hope that help.

[pyscripter 689]

3 hours ago, Kas Ob. said:

I can't run that demo as it complain about Python version,

    You only need to change the following two lines to match your python version:

 

    // Adapt to the desired python version - Will only work with this version
    gEngine.RegVersion := '3.8';
    gEngine.DllName := 'python38.dll';

I am happy to evaluate PRs in relation to the termination issue.  As I said it does work well when you run the python script standalone (not from a python IDE)

 

[Guest]

I version is 2.7 and it is not working, i am not expecting it to work.

 

I can't go through all sources to figure it out, but i believe can draw a path that can cause this, for me as thoughts, and hope they do help you track this and fix it for the running in the IDE,

After Application.Run() been called, where is the exit point? in Delphi it is between ExitThread and ExitProcess as usual for any thread, but in this case both can't be called from code, so the execution should be in python part.

Also for the thread that handled visual part, when it will be exited? and where? i have no idea where did that thread come from to begin with, is it the main thread in py.exe been passed down, or a background one...

 

That what i was pointing to.

[pyscripter 689]

43 minutes ago, Kas Ob. said:

I version is 2.7 and it is not working, i am not expecting it to work.

Python 2 support was dropped recently.  But you can use the last release with Python 2 support.

 

The VCL code is executed in the main thread.  ExitThread and ExitProcess are called by python on shutdown.  After Run is executed control is passed back to python.  In a standalone run python would exit at that point.   When run inside an IDE things may be different since the IDE will terminate the python process when appropriate.

Edited October 25, 2020 by pyscripter

[Guest]

Two hours of debugging using external debugger, and here is the results

 

There is an AV raised when the DLL (*.pyd) been released, this AV is raised from within python, and because it is not handled it been logged into the OS EventLog

Following the stack, it did pointed to TEventHandler.Destroy, this been reached from TPyDelphiWrapper.Finalize, which all comes from TPythonEngine.Finalize and from System.FinalizeUnits

To confirm the culprit, add try..except around that "Clients.Finalize" and you will have the fast solution (workaround), and of course something to track and fix this once and for all.

procedure TPythonEngine.Finalize;
....
  // First finalize our clients
  if Initialized then
    for i := 0 to ClientCount - 1 do
      with Clients[i] do
        begin
          if Initialized then
            Finalize;            //  AV been raised and for me it is always when i=1 !, though ClientCount is +60, try except here will fix this 
        end;
  // Then finalize Python, if we have to
  if Initialized and FAutoFinalize then begin
    try
      try
        FFinalizing := True;
        Py_Finalize;
      finally
        FFinalizing := False;
        FInitialized := False;
      end;
    except
    end;
  end;

 

Hope that help and save you time, as finding the exact EventHandler causing this will cost me hours and might not yield a result, while you can spot and debug it faster.

[pyscripter 689]

After some investigation I have committed changes that eliminate the exceptions during library finalization.

The comment above about hiding the form still stands.

 

If the main form is closed with action caFree, all VCL does, is send a Quit message (i.e. does not hide or free the main form).  If the application is not terminated (as is the case when running in PyScripter) the form is not closed.   You need to hide it using f.Hide().  In PyScripter the python application is truly terminated when you reinitialize the interpreter or close down PyScripter.

[Guest]

DelphiVCL from Python is fantastic, now i want to ask for another demo or extending the existing one, you already added full VCL, the default library and that is huge work, so what if i want to use 3rdparty library ?

 

The missed things in a demo is the minimum code to be able to use DataModule or a Delphi form containing 3rd-party library/component, it doesn't need to go through all its element, just some procedures and functions, your notes on that will have great value, as your code for the VCL is great but intimidating, so a simple form created from Delphi library and can communicate with the one created from Python through one or more function will be great, as for the DataModule, i really want to test the ability for getting data from DB using Delphi or using an IPC mechanism, then feed it to Python, i am aware of the reversed process, like using python from Delphi app, but i think using Delphi from Python is neat, shorter and more portable.

[pyscripter 689]

Wrapping third party components is no different to wrapping VCL controls.   Consider all the WrapDelphiXYZ units as demos on how to do that.

- Please note that with EnhancedRTTI, the only thing that needs wrapping is events other than simple TNotifyEvent.  So the wrapping code would be minimal.

- There is no special wrapping needed for DataModules (TComponent descendent).  Access to the Screen.DataModules property is already wrapped.

- Data Access wrapping is provided by the WrapFireDAC unit

- Demo 31 has extensive coverage of Forms (creation, subclassing etc.)