Clearing the Main Module

[fjames 0]

Is there a way to reset/clear the main module?

 

For example, if I call PythonEngine.ExecString('test = 100') and then check PythonEngine.PyObject_GetAttrString(PythonEngine.GetMainModule,'test'), it rightly points me to the PPyObject that can be converted to Integer.

However if I then do PythonEngine.ExecString('newTest = 1') and check for 'test', the variable still exists in the main module.

 

I want to be able to reset the Main Module back to a clean start. Is this possible, or is there an alternative way to keep environments separate from each other?

 

Currently, if I have two delphi methods and method1 for example executes 'import time' in the PythonEngine, then in method2 I could execute 'print(time.time())' without an error, where as I want method2 to return a Name Error, unless method2's script had 'import time' in it.

 

 

This is very easily seen in Demo01.

Change the code to:

'a = 5

print(a)'

Click Execute Script

Change code to:

'b = 1

print(a)'

Click Execute Script

 

See that the Output Memo contains two '5's meaning that 'a' was never removed from the main module. It just keeps accumulating.

Edited January 22, 2021 by fjames

[fjames 0]

If anyone is curious, I did find a workaround to this issue.

 

If I create a new module using PyImport_AddModule, and get the __dict__ for this module, I can pass that into ExecuteString as the "locals".
You can then clear our this dictionary when the script changes, and that will clear the namespace.

When testing this on Demo01, I could actually get away with calling PyDict_Clear, and everything still seemed to work. However in a more complicated use case, this did not work.

After the first time clearing the Dictionary (not in Demo01), I lost the builtins as would be expected, and then you can't do things like 'import'.

 

As as alternative, I did find that clearing the dictionary was okay, if I used the MainModule __dict__ for the globals parameter of ExecString. However then if the caller writes, 

"global somevar; somevar = 0;" then even after clearing the local dictionary, somevar would still persist as it was written to global dictionary. This may be okay depending on the

use case, however I decided against using the MainModule as the global namespace, as I actually want multiple active modules with separate, non-interacting namespaces. In 

order to achieve this, I am still using PyImport_AddModule to create or retrieve a module given a name, and then I am copying the attributes from __main__ into the new module,

and whenever the script changes, I clear all attributes in the module except name, and reinitialize to __main__. This way each module has all of the __builtins__, as well as the class

"DebugOutput" that is placed into the module by the PythonEngine for redirecting IO.

 

While this module/dictionary manipulation could be done using the PythonEngine, I found it easier to just write two small python scripts for handling the module copy and module clear.

 

To copy the main module into my new module ("%s" is name of module being manipulated, so that I can Format this string in delphi and call ExecString):

import sys as new_sys                                           
for attribute in dir(new_sys.modules["__main__"]):            
    if attribute not in ["new_sys", "__name__"]:                
        setattr(new_sys.modules["%s"], attribute,                
            getattr(new_sys.modules["__main__"], attribute))    
                                                                
delattr(new_sys.modules["__main__"], "new_sys");

And then to clear the module being used when the script changes:

import sys as new_sys
for attribute in dir(new_sys.modules["%s"]):                       
    if (not attribute.startswith("__") and (attribute != "new_sys")):
        delattr(new_sys.modules["%0:s"], attribute)                
                                                                    
delattr(new_sys.modules["%0:s"], "new_sys");

 

[pyscripter 689]

There are many ways to achieve what you want:

 

1. The ExecString has an overloaded form:

procedure TPythonEngine.ExecString(const command : AnsiString; locals, globals : PPyObject );

You can pass a new empty dictionary to the locals and globals argument if you do not want to "pollute" the interpreter namespace.

 

2.  You can manually do the clean up as you suggest above.

 

3.  A more advanced way is to use subinterpreters  (have a look at TPythonThread.Execute), but this is more error prone.

Edited February 8, 2021 by pyscripter

[fjames 0]

I originally tried Option 1 as you suggested, however, passing empty dictionaries caused me to lose the builtins, could no longer call 'import' for example.
Additionally P4D is modifying the Main module, and by using a blank module you do not get these modifications (For example, I had trouble printing in certain circumstances, since the IO redirection is also setup in the main module, as well as any init scripts)

 

Thanks for the suggestions!