fjames

This is not true. "exit()" raises a Python SystemExit exception which you can certainly catch. If your change your Demo01 code to be: try: exit() except SystemExit as e: print("Python tried to Exit!") The top TMemo will be populated with "Python tried to Exit!" and the application will stay open. So you could easily modify an app to do this in the background. Oddly enough, looking at TPythonEngine.RaiseError, the code is attempting to catch SystemExits and raise a Delphi exception.

I originally tried Option 1 as you suggested, however, passing empty dictionaries caused me to lose the builtins, could no longer call 'import' for example. Additionally P4D is modifying the Main module, and by using a blank module you do not get these modifications (For example, I had trouble printing in certain circumstances, since the IO redirection is also setup in the main module, as well as any init scripts) Thanks for the suggestions!

That is the answer I was afraid of. That means crashes like this are almost impossible to prevent. Making "sure that your environment has the correct packages installed" is very tricky it seems. For example in the matplotlib example, matplotlib seems to be installed correctly and only produces error calling 'show()', and actually if I start python directly from the activated cmd prompt, matplotlib executed successfully, so the problem was resulting from some dlls not being in the correct place when being run from within a Delphi Application (but installing qt and pyqt put the dlls in a place that works from both within and outside a Delphi application). So to properly ensure the environment was correct, I would need to know all the dlls needed by each python module, and ensure they are in the place that the module is expecting. I guess if you are willing to allow the importing of third-party modules, this is a risk you have to take. Thanks for the quick response!

I was wondering if there is a way to prevent any external DLL calls from closing the Application. I ran into an issue with matplotlib raising an error, and after the error, the Application closed, and I cannot find a way of 'catching' the exception to prevent the crash. Stepping through the execution it seems that the call to PyRun_String in TPythonEngine.Run_CommandAsObjectWithDict never returns, and the application is terminated, and would like a way of preventing this. I have been able to reproduce this problem in Demo01 by performing the following steps: Install Anaconda 32bit Create new environment (Python 3.6.12) and install matplotlib using 'conda install matplotlib' in the new environment Start Demo01 in the new environment Can either start Demo01.exe from activated cmd prompt, or just open the IDE from an activated cmd prompt that way you can debug in the environment as well Change script to simple: 'from matplotlib import pyplot as plt; plt.plot(range(10)); plt.show()' What results is a popup much like the one here. After clicking OK, the program exits Its worth noting, that I was able to overcome the above issue by following the steps in this answer, however I foresee more issues like this in the future, and I would like some way of preventing my application from crashing if one of Python's third party modules encounters an error. Thanks for any advice!

If anyone is curious, I did find a workaround to this issue. If I create a new module using PyImport_AddModule, and get the __dict__ for this module, I can pass that into ExecuteString as the "locals". You can then clear our this dictionary when the script changes, and that will clear the namespace. When testing this on Demo01, I could actually get away with calling PyDict_Clear, and everything still seemed to work. However in a more complicated use case, this did not work. After the first time clearing the Dictionary (not in Demo01), I lost the builtins as would be expected, and then you can't do things like 'import'. As as alternative, I did find that clearing the dictionary was okay, if I used the MainModule __dict__ for the globals parameter of ExecString. However then if the caller writes, "global somevar; somevar = 0;" then even after clearing the local dictionary, somevar would still persist as it was written to global dictionary. This may be okay depending on the use case, however I decided against using the MainModule as the global namespace, as I actually want multiple active modules with separate, non-interacting namespaces. In order to achieve this, I am still using PyImport_AddModule to create or retrieve a module given a name, and then I am copying the attributes from __main__ into the new module, and whenever the script changes, I clear all attributes in the module except name, and reinitialize to __main__. This way each module has all of the __builtins__, as well as the class "DebugOutput" that is placed into the module by the PythonEngine for redirecting IO. While this module/dictionary manipulation could be done using the PythonEngine, I found it easier to just write two small python scripts for handling the module copy and module clear. To copy the main module into my new module ("%s" is name of module being manipulated, so that I can Format this string in delphi and call ExecString): import sys as new_sys for attribute in dir(new_sys.modules["__main__"]): if attribute not in ["new_sys", "__name__"]: setattr(new_sys.modules["%s"], attribute, getattr(new_sys.modules["__main__"], attribute)) delattr(new_sys.modules["__main__"], "new_sys"); And then to clear the module being used when the script changes: import sys as new_sys for attribute in dir(new_sys.modules["%s"]): if (not attribute.startswith("__") and (attribute != "new_sys")): delattr(new_sys.modules["%0:s"], attribute) delattr(new_sys.modules["%0:s"], "new_sys");

Is there a way to reset/clear the main module? For example, if I call PythonEngine.ExecString('test = 100') and then check PythonEngine.PyObject_GetAttrString(PythonEngine.GetMainModule,'test'), it rightly points me to the PPyObject that can be converted to Integer. However if I then do PythonEngine.ExecString('newTest = 1') and check for 'test', the variable still exists in the main module. I want to be able to reset the Main Module back to a clean start. Is this possible, or is there an alternative way to keep environments separate from each other? Currently, if I have two delphi methods and method1 for example executes 'import time' in the PythonEngine, then in method2 I could execute 'print(time.time())' without an error, where as I want method2 to return a Name Error, unless method2's script had 'import time' in it. This is very easily seen in Demo01. Change the code to: 'a = 5 print(a)' Click Execute Script Change code to: 'b = 1 print(a)' Click Execute Script See that the Output Memo contains two '5's meaning that 'a' was never removed from the main module. It just keeps accumulating.

The main points for accessing numpy to and from delphi are that first you need to hook up the numpy c-api methods (similar to how the Python c-api methods are hooked up in MapDLL). This will allow you to create numpy arrays using memory allocated by Delphi.

I also got this same behavior after upgrading >= v15.0.28. This made the indexer unusable for me. Once I downgraded back to v15.0.27 the indexer worked without AVs again.

There should be no reason you cant use a single engine, and then have a separate module for being imported by separate scripts. The same engine can execute all the scripts, (albeit sequentially due to issues described above), but for your use case I don't see this being an issue.

Thanks again for your responses. I think I will end up having some questions much more suited for SO as I get more into the nitty gritty.

Reading up on Python Buffers, it seems that its possible in C - And since you said a Delphi array is: It seems I can give the memory of a Delphi array to a Python Buffer, the way I would give a C array to a Python Buffer, unless I am misunderstanding something. While I could not find an example using python arrays, I found an example using numpy, and according to the post, the numpy array will have access to the values in the C array until the C array is freed (Source). PyObject *makearray(int array[], size_t size) { npy_int dim = size; return PyArray_SimpleNewFromData(1, &dim, (void *)array); }

Thanks for taking the time to try and help. I guess I just don't understand how your original post is a solution to the problem. I don't really care about it being numpy, I care about it being Python. For example, I want to be able to do the exact same thing I did in the code snippet above using a Python list. # Where this is a Python List using values from Delphi inputData = DelphiModule.InputArray # I use this data to create new data # (should be able to use inputData like any # other list, just cannot modify) ResultData = [x**2 for x in inputData] So here there is no numpy whatsoever, but my goal is the same, allow python to manipulate memory that was allocated by Delphi, and allow Delphi to manipulate memory that was allocated by python. Right now, it seems like I should be able to create a new Python List as a PPyObject, and then get the Buffer associated with the list (using PyObject_GetBuffer), and then tell the buffer to point to the memory allocated by Delphi for an array. Does this sound possible?

Yes, the goal is to be able to operate on this shared data in Python. As well as allocate memory within Python and share that with Delphi. I wrote up a very basic snippet of what I was thinking to hopefully aide in clarifying the goal. // In Delphi var myArray: array of Integer; numpyArray: PPyObject; // I have an array allocated in Delphi // myArray := [1,2,3,4,5,6,7,8,9,10]; numpyArray := CreateNumpyArrayFromArray(myArray); // Trying to create this method PythonModule.SetVar('InputArray', numpyArray); PythonEngine.Py_DecRef(numpyArray); # In Python import numpy as np import DelphiModule # Where this is a numpy array using values from Delphi inputData = DelphiModule.InputArray # I use this data to create new data # (should be able to use inputData like any # other ndArray, just cannot modify) ResultData = np.square(inputData) // Back In Delphi numpyArray := MainModule.GetVar('ResultData'); newArray := GetArrayFromNumpyArray(numpyArray); // With the goal being that I dont need to copy // the values from this array, but can now just use them // newArray = [1,4,9,16,25,36,49,64,81,100] With the idea being that these arrays are going to be very large and I would like to share multiple, which is why I am trying to avoid copying back and forth.

Apologies for having missed that comment hidden in the code snippet. Activating a Conda environment, essentially changes your Windows Path to point to the Bin folder for that environment. If I had that Bin folder in my path as the comment suggests, that would defeat the purpose of having separate environments. I see that what I need to do is essentially, while searching for Python distributions, if I find one within a Conda environment folder, I need to temporarily adjust the path to point to the env bin folder, and then restore it back to the original path later.

Thank you for your reply! I guess I should rephrase my question then. How would you get Delphi values (for example an array of Double) into a Python Buffer Object, does this involve just changing the 'void *buf' on the PyBuffer?. This is what I want to be doing. I want to point a Buffer to the memory of a Delphi array, then use the values in that Buffer within Python. Similarly, I want to be able to create a list (or ndarray) in Python and then by accessing the buffer of this object in Delphi, pull out the values into a Delphi array without copying them. This is what I want to know if is possible. Is there a way to give the Buffer the memory address of the Values in an array so that the buffer uses that memory and now consists of those values? If am I understanding you correctly, you are saying that once I get the internal buffer, I can modify it to point to memory I allocated in Delphi, then in python do something, and then again access the buffer in Delphi to get access to the memory with updated values?