Data exchange between Delphi and SciKit Learn

[Jan Lutgert 0]

Dear reader,

 

I use Delphi 10.4.1 on Win10 and I have recently started working with P4D and Python 3.9.  I watched the two presentations by Jim McKeeth and studied and tried some of the the demos that came closest to my intentions, in particular demo 17 (variant array manipulations) and demo 25 (VarPyth). Next I tried to modify them to suit my needs, unfortunately only with partial succes. Thats why I ask for your guidance.

 

Over the years I have created a statistical evaluation environment that I use to apply various kinds of multivariate statistics to geological data and to analyse their results, see Kmeans.jpg. To extent its capabilities I would like to include some of the many statistical algorithms in SciKit Learn. In my sample project I am using kmeans clustering as an example. The challenge is to transfer the data and settings I have in Delphi to the python environment and to transfer the results back to Delphi.

 

Input issue
Using demo17 as an example, I have created a variant array the can hold my data. That works just fine. However, when I want to use the variant array that I have defined as input for the the SKlearn kmeans clustering module it does not recognize it. Using the examples in the demo's I can create, manipulate and read the array from the Delphi side, but I can not 'expose' it to the Python side. Apparently I am missing a crucial step where I have to register it as valid input (2D numpy ndarray) in the Python environment. Can you please provide some clues how that should be done?

 

Output Issue

The second issue is very similar and concerns how to transfer the clustering results back to Delphi. The SKlearn module creates a 1D array of variants containing the clustering labels (type numpy ndarray). In the script the function that returns the array is named 'kmeans.labels_'. It produces an array of integers indicating the cluster membership of each of the samples. I want to be able to use that array in my program but I don't understand how to get it out of the Python environment. Using the P4D procedure:  getVarAsVariant('kmeans.labels_') does not seem to do the trick... 

 

The associated example project now loads the data from an excel file using the Pandas library. Next the Pandas data frame is converted to a numpy ndarray. And after that the SciKit kmeans clustering programs is run. After finishing, the input and output data and their data types are all added to a (synedit) memo. All this works just fine. But I don't need Delphi to do all that, just the Python script would be sufficient. If we can make data exchange fully dynamic the full power of all the great Python libraries will become avalilable! Some promiss... Your assistence is greately appereciated!

 

Kind regards,

 

Jan Lutgert

 

The example code is in project SKM04, the data is in P14_NUZ.xls

 

SKM04.dpr

SKM04.dproj

SKM04_Unit.dfm

SKM04_Unit.pas

P14_NUZ.xls

[pyscripter 689]

Have you seen python4delphi/Tutorials/Webinar II at master · pyscripter/python4delphi (github.com)?  The VarPyth demo shows how to work with numpy arrays (two way link).

[Jan Lutgert 0]

Thank you for your speedy reaction! I will install myself behind my monitor with a pot of coffee and review the presentation. 
 

[tomye 1]

On 4/11/2022 at 6:37 AM, pyscripter said:

Have you seen python4delphi/Tutorials/Webinar II at master · pyscripter/python4delphi (github.com)? The VarPyth demo shows how to work with numpy arrays (two way link).

does P4D support real Object transfer?

 

for example we define a class in Python, it includes some data types of list , array, bytes stream etc. and it could be returned to Delphi 

 

then we can use the Object in Delphi with all data which are under the object.

[shineworld 73]

Do you have tried the tutorials from Webinar II?

Perhaps they cover your question.

https://github.com/pyscripter/python4delphi/tree/master/Tutorials/Webinar II