How Delphi manages mapping data aware control to a data source on a data module

[Martyn Spencer 0]

This is a bit of a general question and I may not be wording it particularly well. It's also pretty fundamental and in my time using Delphi, I have seen a number of approaches and I am just wondering what the general consensus is. I hope my explanation is clear enough.

 

Let's take a simple example. Form class TfA, has a data aware control ctrl1. There is a data module class TdmB with a data source ds1 linked to a query q1. I want to allow multiple instances of TfA, each with it's own data module instance of TdmB. In of itself, this is really straightforward. I can create multiple instances of TfA and TfA can be responsible for creating its own instance of TdmB and the data source for ctrl1 can be set in code to the instance of TdmB's ds1. This seems to be how some programmers do it. With anything other than a simple form, this becomes a bit of a headache. I appreciate that some people would argue that the data source should be placed on TfA, not in TdmB. In fact, doing this could well simplify the process I am asking about, but I personally have a loathing for too many non-visual controls on a form (I know they can be hidden) and I prefer to see data sources and data sets on data modules. Delphi's documentation suggests that it is OK to work this way.

 

When using the standard Delphi approach, the form designer creates an instance variable for form TfA (fA) and an instance variable for data module TdmB (dmB). When designing TfA, I can simply set the data source and field name for ctrl1 (dmB.ds1) and when the application runs, fA and dmB are auto-created. Delphi automatically handles the correct assignment. How is this handled internally by Delphi? Is it each data aware control that is responsible for picking up the appropriate data source? If not, does the form do it, or is it the data module? Is it the data link that forms part of a data aware control that handles it?

 

The way that I have done this in the past (it works, but it could be considered rather ugly) is to remove Delphi's automatically defined instance variable for the form and data module. In the form class code, I create an instance of the data module, using the form as the owner. When the data module class is created, Delphi appears to automatically handle mapping of the control data sources and I then change the data module component name. This then means that when the next data module instance is created, it can use the default component name and mapping again appears to work correctly. This appears to work well, but I would very much like to be able to say to an existing form, that it should use a specific instance of a data module, on occasion. This may be, for example, if TfA creates a form TfB, which for simplicity's sake will use the data module that TfA created. TfB may be created some time after the TdmB instance has been created (and therefore renamed), so I do not think that the mapping is automatic. I appreciate that I could create an instance of TfB before renaming the data module, or I could temporarily rename it back, but this seems very messy.

 

I am interested in how others do this. If anyone knows of a good reference source for best practices in this instance, it would be very helpful.

[Lars Fosdal 1792]

YMMV, by a LOT.  We don't use data-aware controls at all, but do all the DB work behind the scenes and pass custom classes to the UIs.  These classes have methods to retrieve themselves from the DB, save themselves, as well as various processing methods, and we use helper/proxy-classes to simplify populating the UI as well as taking input from the UI. The source of the data doesn't matter to the UI, and we found it easier to have absolute control over the UI behavior and data updates and creation. 

Our UIs are much more than CRUD interfaces, as the data types are very complex and interdependent, as well as manipulated from multiple systems.  But - in the end - we do a lot of explicit "wiring" in code in the UI proxies.

 

For other solutions, data-aware components can work perfectly, but TBH, I've only worked with one solution that used them, and they had written an entire ORM to handle their data retrieval and input from DB to UI and back,

[Martyn Spencer 0]

Thanks Lars. This is a relatively simple CRUD application and in need of some tidying, rework, but not a major rewrite. I was hoping to learn a little about the internals so that I could manage the re-mapping using some generic code, if possible. What I have currently does work, but it feels like I am abusing the way that it is "meant to work". I generally prefer to avoid this as relying upon automatic behaviour that I cannot predict or control is prone to breaking without apparent reason.

 

Your approach does sound interesting. In more recent apps, I have done something similar, in that the UI comprises basic dumb controls and I add the persistency as required. For new applications, I probably should look into some ORM frameworks, but I am a little unhappy to rely upon automated SQL generation in most instances.

 

I am still trying to find where Delphi replaces component references. I think it could well be at the streaming stage.

[Uwe Raabe 2057]

5 minutes ago, Martyn Spencer said:

I am still trying to find where Delphi replaces component references. I think it could well be at the streaming stage.

Indeed, it is! The references are resolved in TReader.FixupReferences and are based on component names. To wire the datasource on the datamodule to the controls on the form, the datamodule has to be created before the form loads. As all forms and datamodules register themselves in the Screen.Forms and Screen.Datamodules lists, they are found by simply iterating over these lists in the Vcl.Forms.FindGlobalComponent procedure, which is registered via RegisterFindGlobalComponentProc.

 

You can register your own FindGlobalComponentProc this way, which takes precedence over the internal one (System.Classes.FindGlobalComponents uses a reverse loop). The implementation might be a bit tricky, though.

[Martyn Spencer 0]

Thanks Uwe. I have found that if I create the associated data module instance on form creation, and set the owner correctly (this may be incidental and not important), the linkages work. I guess this is because the streaming of the form has not yet been initiated.

 

I think I can work with what you gave me. Appreciated.

Edited January 7, 2020 by Martyn Spencer
Clearer information.

[Uwe Raabe 2057]

11 minutes ago, Martyn Spencer said:

and set the owner correctly

That may be key here. Before looking for global components all components owned by the current one are searched (Keywords: DoFixupReferences, FindNestedComponent). So if your datamodule is owned by the form it should be found first. You may probably even omit the renaming step after loading.

[Martyn Spencer 0]

That would make sense. I definitely have a way forward. Thanks for your input, both.

[David Schwartz 426]

I'm not quite following this, and I'm curious what the crux of the problem you're trying to address seems to be.

 

AFAIK, the underlying logic uses RTTI stuff, which depends in part on strings that need to match things on the form or DM. But once everything has been created and initialized, the symbolic references are typically no needed. (They can slow things down.)

 

It's fairly common in some apps to only have a few forms and DMs get auto-created at startup, and create the rest dynamically. When this is done, you typically comment out the global form var at the bottom of the Interface section of the forms that are created dynamically. That way it's entirely possible to create as many instances of the form as desired. Assuming the form itself creates the associated DM, there's not going to be any problem. You WILL have a problem if you try to share a single DM among multiple forms, but that does not seem to be what you're doing.

 

 So what exactly is your concern? (I have one based on what's said above, but I want to see if it's the same as what you're thinking.)

[Martyn Spencer 0]

Hi David,

 

Primarily I was looking to confirm my understanding of how the process worked internally.

 

Your comment is in line with my findings and is pretty much how I handle things, in that if I create form A, with data module B, if B is created and owned by A, the data source fix-ups seem to be consistently applied and I don't observe any issues. Having said that, even though it always seemed to work, I was interested in how other developers handled the situation in case there was something I was missing. From what I read above, I think my understanding is OK.

 

In the situation that you mention in your third paragraph, where a single DM may be shared by a form and maybe a popup that is generated by that form, there are problems and it requires the data module name to be as per what is specified in the designer. Some situations really don't merit a data module specific to the popup and it is here where I wanted to streamline the process of assigning controls to data sources.

 

My reason for posing the question in the first place was that I took over maintenance of an existing program and really did not like how the original programmer worked and was trying to move more towards the methods that I currently use (and work). As part of this process, I thought it prudent to confirm my understanding of the process.

 

Hope that provides more detail for you.

 

I do actually think that my question has now been answered, and whilst I am interested in any discussion that arises from this, please don't feel the need to explain much more to me, as I think I am OK now.

 

Best wishes,

 

Martyn

 

 

[David Schwartz 426]

The thing to realize is that forms are objects -- instances of class TForm or a descendant. People don't seem to think of forms as regular objects, but they are. They also have restrictions due to potential multitasking conflicts with the VCL since it's not "re-entrant" code (how I learned it). That is, it's not "thread-safe". It has its own data. A DataModule is the same. 

If you had defined any normal class with private data members in it, you'd expose them via properties and thereby hide the private data through encapsulation. People don't do that with forms for whatever reason; rather, they reach inside of form objects and fiddle with what should be PRIVATE data members directly. But because of how the VCL streams stuff, the components on the form cannot actually be made private, which leads to people cheating and then justifying it as if that makes it OK. Then they get their knickers in a wad when things don't work properly. Well DUH!

 

If you treat a form as if it was any other kind of object, then it would work. That is, if you expected it to be used by multiple clients, you'd maintain separate data elements for each client and they would not be aware of that since you'd be accessing everything through properties. Your questions do not seem to indicate that's what you're doing because you wouldn't be aware of the Name values given to anything on the Forms or DMs via the IDE. You wouldn't even care.

 

Most likely, you'd set up properties to provide access to the data on the form or DM and have a Factory pattern to give you instances of it. The Factory would either ensure that each one had isolated private data elements or ensure that no conflicts existed when accessing things that aren't thread-safe.

 

You said you think your question has been answered, but I think you were asking the wrong question from the outset.

 

Database components like TDatabase, TDataset, and TTable are not stateless. You cannot generally share them among different forms. TQuery components can be treated as stateless if they're simple lookup tables and their queries obtain the entire result set; but if not, you're going to end up with quite a mess.

 

The client forms should not know about how things are implemented on the forms and DMs they're sharing. Your original problem statement is expressed in a way that assumes this is not the case -- that the clients DO know what's living inside of the shared objects. THAT is a fundamental flaw in the approach you laid out, and your questions arise due to this flawed model.

 

Change your approach and the questions will disappear along with the conflicts you're experiencing.

 

Forms are normal objects. If you treat them like normal objects, they'll work like normal objects. You cannot do the typical kind of "cheating" that most Delphi devs do with forms and expect these encapsulation problems to be solved by some kind of magic.

 

Here is the crux of the issue: the RTL can only stream PUBLIC data members. That doesn't give you a license to TREAT them as if they're "safe" just because they're PUBLIC! They need to be regarded as if they're PRIVATE.

 

Perhaps I'm completely off-base in terms of what you've arrived at in your thinking, but I can't tell that from what you've said here.

Edited January 15, 2020 by David Schwartz

[Martyn Spencer 0]

Thanks for the extensive explanation, David. Fortunately I am aware of all that you describe and it is a good "understanding check" for me. I appreciate you taking the time to write it. All very useful info. I am working on a system that was written by a Delphi developer who clearly did not understand encapsulation and was quite happy to write code wherever the fancy took him. I still am quite happy that my question was answered before 🙂