JSON serialization using published properties w/o attributes

[Zoë Peterson 15]

Are there any object-to-JSON serialization libraries available that work with published properties (not public, not fields) and which allow customizing the serialization externally to the class?  Dalija talked about this in her "When in Rome" series (https://dalijap.blogspot.com/2021/04/when-in-rome-do-as-romans-do-part-i.html) but didn't point to an alternative that worked that way.  Any license is fine, though open source of some sort would be preferable.

 

Specifically, I'm working with third party TComponent classes which already have most of their state exposed as published properties, and I can't add attributes to customize things.  A couple of the components do use DefineProperties for some additional state when stored as DFMs, and I need to be able to register some sort of callback to serialize those.  I was able to get close with mORMot's ObjectToJSON with RegisterCustomSerializer, but it doesn't support mixing published and custom properties in a single class.  All of the other ones I've found do public/private fields, require attributes in the original classes, or fail in the marshalling step with unhelpful error messages.

 

Edit: I figured a workaround using mORMot for my specific usage, so I don't need this anymore, though it is a bit hacky, so if there's a better approach, I certainly won't mind hearing it.

Edited August 12, 2021 by Zoë Peterson

[David Schwartz 442]

Couldn't you derive a child class, redefine the properties to reflect those in the ancestor class, then add the necessary attributes to them?

[Uwe Raabe 2162]

8 minutes ago, David Schwartz said:

Couldn't you derive a child class, redefine the properties to reflect those in the ancestor class, then add the necessary attributes to them?

Might be tricky to persuade the third party lib to create your new classes rather than its own.

[Dalija Prasnikar 1513]

15 hours ago, Zoë Peterson said:

Are there any object-to-JSON serialization libraries available that work with published properties (not public, not fields) and which allow customizing the serialization externally to the class?  Dalija talked about this in her "When in Rome" series (https://dalijap.blogspot.com/2021/04/when-in-rome-do-as-romans-do-part-i.html) but didn't point to an alternative that worked that way.  Any license is fine, though open source of some sort would be preferable.

I am working on one, but it is work in progress. Intention is to make this during follow up series of blogposts. Since I am quite busy at the moment, I am not sure when will this see the light of day.

 

I already have such framework written in Swift, so most of the mental work is finished, but there is still a lot to do beyond merely translating it to Delphi, as it does not have all bells and whistles fully in place.

 

Edited August 12, 2021 by Dalija Prasnikar

[Zoë Peterson 15]

5 hours ago, Dalija Prasnikar said:

I am working on one, but it is work in progress. Intention is to make this during follow up series of blogposts. Since I am quite busy at the moment, I am not sure when will this see the light of day.

Understandable, and thank you for the initial posts.  We have a settings framework internally that works that way and works great, but it uses XML and would have been hard to extract.  By the time I'd gotten to your posts I'd already run through half a dozen different JSON serialization libraries that all used extended RTTI and attributes, and I was beginning to think I was crazy for expecting one to take advantage of Delphi's long established persistence support.

[Zoë Peterson 15]

6 hours ago, David Schwartz said:

Couldn't you derive a child class, redefine the properties to reflect those in the ancestor class, then add the necessary attributes to them?

As Uwe said, that wouldn't have worked since the parent object was creating child objects of the appropriate type, and I don't have any control over that.  For context, it was for a VCL style to JSON converter and I want to use the VCL's existing style loading code.

 

Regardless, I still think it's weird that literally every JSON serialization library I could find relied on attributes and public/private fields.  Delphi has had TPersistent and properties with 'stored' and 'default' and different names than the backing fields for its entire existence.  They work well.  It's great that the newer extended RTTI/attribute approach is available, but I absolutely do not understand why that's the only approach anyone's used.

Edited August 12, 2021 by Zoë Peterson

[Dalija Prasnikar 1513]

35 minutes ago, Zoë Peterson said:

I absolutely do not understand why that's the only approach anyone's used.

I keep asking myself the same question over and over again. Let me know if you find the answer 😕

[Tommi Prami 148]

11 hours ago, Zoë Peterson said:

It's great that the newer extended RTTI/attribute approach is available, but I absolutely do not understand why that's the only approach anyone's used.

What is disadvantage and/or deal breaker not to use Attribute based library? Why it won't work for you? (Just genuinely curious) ?

[Uwe Raabe 2162]

4 hours ago, Tommi Prami said:

Why it won't work for you?

You can't put attributes on code not under your control (like the VCL f.i.).

[SwiftExpat 65]

TMS FNC Core has a a library that works with published properties.  It is licensed with FNC. 

TMS blog post describing it here:  https://biz.tmssoftware.com/site/blog.asp?post=646

 

Here is your code forked / converted to work with FNC : https://github.com/SwiftExpat/vsf2json .  EXE and a json sample are in the releases so you can try it out if you like.

 

I had to derive a class to include the list of style objects, but not much code to write at all.

  TExportStyleSource = class(TSeStyleSource)
  public
    function GetStyleObjects:TList;
  published
    property StyleObjects: TList read GetStyleObjects;
  end;

[Zoë Peterson 15]

6 hours ago, SwiftExpat said:

TMS FNC Core has a a library that works with published properties.  It is licensed with FNC. 

TMS blog post describing it here:  https://biz.tmssoftware.com/site/blog.asp?post=646

I did see that one in my search and should have looked at it more closely.  Ultimately I avoided it because I was trying to stick to an open source library to make it easier to share the code, and then sort of forgot about it. 🙂

6 hours ago, SwiftExpat said:

Here is your code forked / converted to work with FNC : https://github.com/SwiftExpat/vsf2json .  EXE and a json sample are in the releases so you can try it out if you like.

Nice! StyleObjects is definitely a better name for that part than "Objects" is. 🙂  The FNC version is missing a bunch of the data I had to manually construct in the mORMot version though:  Colors, SysColors, and Fonts are public sub-objects of TSeStyleSource and none of them have properties that match the name/value pairs that they represent.  That's what the "WriteColors", "WriteSysColors" and "WriteFonts" functions were doing, which were loosely based on the SaveToStream functions in the respective objects (TSeStyleColors, TSeStyleSysColors, TSeStyleFonts).  The style objects also support nesting, and your version only includes the top-level of it.  

[Zoë Peterson 15]

11 hours ago, Tommi Prami said:

What is disadvantage and/or deal breaker not to use Attribute based library? Why it won't work for you? (Just genuinely curious) ?

Like Uwe said, I'm trying to serialize objects I don't control (stock VCL components), so I can't modify the original declarations.

 

Part of is learning curve too.  This started for personal use and I wasn't expecting to spend days researching JSON serializers.  I started by just passing the base object into various framework's ObjectToJSON routines.  Delphi's built-in ones fail with a marshalling error several layers deep, with no indication of what it was faulting on.  Neon, I think, failed because it decided that the object should be stored as an array since it had a "Count" property, and there wasn't a way to force it to do otherwise.  I looked at a few other frameworks too and don't remember exactly how they all came up short, but my starting point was "This is a TPersistent descendent that already has published properties; it should be easy to get at least a minimal export without doing anything fancy", and aside from mORMOT and TMS's FNC framework, that wasn't true.

 

Other than that though, I largely agree with the arguments Dalija covered in her blog: Part 1, Part 2, Part 3, Part 4

[SwiftExpat 65]

On 8/13/2021 at 5:37 PM, Zoë Peterson said:

The style objects also support nesting, and your version only includes the top-level of it.  

I played a little more with this, but honestly it is no less code than what you already wrote, basically a recursive loop writing a JSON.

On 8/12/2021 at 12:54 PM, Uwe Raabe said:

Might be tricky to persuade the third party lib to create your new classes rather than its own.

This statement form Uwe helped me understand the complexity better.  FNC gets the object type using RTTI, so I am not sure it is worth trying to trick / persuade it.

 

StyleAPI is an example of how to seal an API.  Since the reference is via an include, I would be tempted to copy it and modify visibility.

 

Zoe, you deserve a thanks for digging through the API and making an export with it.