How to type json properties that are arrays with mixed element types?

[Lars Fosdal 1792]

Consider the following Json data:

The array elements are lists of integers, strings and objects.

{
       "message": [
              [ 0, "a text" ],
              [ 1  ],
              [ 1, { "switch": true } ],
              [ 2,  "text one",  "text line two" ]
       ]
}

Assume the JsonData const below is filled with the Json above. Using the Json tools from unit REST.Json, I would typically do like this.

 

const
  JsonData = // see Json data above
var
  Message: TJsonMessage;
begin
  Message := TJson.JsonToObject<TJsonMessage>(JsonData);

 

But - how should TMessageArray be declared to accept the above structure? Is it actually possible? 

 

TJsonMessage = class
private
  Fmessage: TMessageArray;
public 
  property message: TMessageArray read FMessage write FMessage;
end;

 

[Lars Fosdal 1792]

Crickets....

[Kryvich 165]

May be TMessageArray = array of array of Variant; ?

program TestJson;
{$APPTYPE CONSOLE}
{$R *.res}
uses
  System.SysUtils, REST.Json;
const
  JsonData = '{'
    + '  "message": ['
    + '    [ 0, "a text" ],'
    + '    [ 1  ],'
    + '    [ 1, { "switch": true } ],'
    + '    [ 2,  "text one",  "text line two" ]'
    + '  ]'
    + '}';
type
  TMessageArray = array of array of Variant;
  TJsonMessage = class
  private
    Fmessage: TMessageArray;
  public
    property message: TMessageArray read FMessage write FMessage;
  end;
var
  Message: TJsonMessage;
begin
  try
    if TypeInfo(TJsonMessage) = nil then
      Writeln('No RTTI for ', TJsonMessage.ClassName);
    Message := TJson.JsonToObject<TJsonMessage>(JsonData);
    Writeln('Length = ', Length(Message.message));
    Message.Free;
  except
    on E: Exception do begin
      Writeln(E.ClassName, ': ', E.Message);
      Readln;
    end;
  end;
end.

I tried this option, but my test application gives the exception: "Exception class EConversionError with message 'Internal: Cannot instantiate type TestJson.TJsonMessage'.".
I still do not understand why.

[Guest]

18 minutes ago, Kryvich said:

I still do not understand why.

Declare TJsonMessage  in a unit

[Kryvich 165]

@Schokohase It worked, thanks! Can you explain why?

[Guest]

Did you notice an interface or implementation section in a program (*.dpr) file as you do have in a unit file?

 

So everything you wrote in a program file it will behave as in the implementation section of a unit file.

[Lars Fosdal 1792]

So - what happens to 

{ "switch": true }

in an array of array of variant?

 

[Kryvich 165]

@Schokohase In fact RTTI is generated for types declared in a DPR file. But it is not findable by TRttiContext.FindType(AQualifiedName: string). In the Delphi help for the function FindType it is written:

Quote

FindType searches for the type in all packages and works only for public types that have qualified names.

But there is no information what types have qualified names. And only in System.Rtti.pas I found this comment:

Quote

Types not declared in interface section of unit have no qualified name and can't be looked up by name.

@Lars Fosdal Unfortunately, this will not work for the Variant type without additional processing. You can try to declare a custom reverter for an element of the inner array.

[Sherlock 663]

OT: JSON is cool and all, but it also promotes dropping all kinds of junk in it without thinking. Well maybe a little: "Let the next guy take care of this"

 

Sherlock

[Lars Fosdal 1792]

29 minutes ago, Sherlock said:

OT: JSON is cool and all, but it also promotes dropping all kinds of junk in it without thinking. Well maybe a little: "Let the next guy take care of this"

Indeed.  This structure was probably designed by someone that is not loading objects, but who just walks the structures with a weakly typed language.

[Rollo62 536]

If so many people urgently (mis)use it that way, this should be a sign that there is a strong need 🙂

[Lars Fosdal 1792]

1 hour ago, Kryvich said:

@Schokohase

@Lars Fosdal Unfortunately, this will not work for the Variant type without additional processing. You can try to declare a custom reverter for an element of the inner array.

@Kryvich Do you know of any good examples of custom reverters?

I am only going to consume these structures, not produce them - so that simplifies it a bit.  Basically, I'd love to be able to transform the inner array elements to a single object, as there is only a limited number of permutations.

[Lars Fosdal 1792]

1 minute ago, Rollo62 said:

If so many people urgently (mis)use it that way, this should be a sign that there is a strong need 🙂

Or, a lack of established conventions - or - as for xml, an endless series of abuse 😛

[Rollo62 536]

I've made some tests a while ago, and put them into an empty, commented region (for later unbury, if I need them).

Maybe it helps somehow.

class function TJson_Marshal_Base<T>.Internal_FromObject(AValue: TJSONObject ) : TJson_Marshal_Base<T>;

var
  LMar: TJSONUnMarshal;

begin
    Result := nil;

    if not Assigned( AValue ) then
        Exit;

    LMar := TJSONUnMarshal.Create();  // This sets DateTimeIsUTC internally by default

    try
        LMar.DateTimeIsUTC := True;  // !! Important to ensure same Coding/Decoding

{$REGION 'OptionalReverter'}

  //        LMar.RegisterReverter(TToolPanel_Parcel,     //TODO remove, since it never gets here
  //             function (Data: TListOfStrings) : TObject
  //             var
  //                 LObj : TToolPanel_Parcel;
  //             begin
  //
  //                 if Length(Data) >= 9 then
  //                 begin
  //                     LObj := TToolPanel_Parcel.Create;
  //
  //                     LObj.FValueType      := StrToIntDef(Data[0], -1);
  //                     LObj.FValueNum       :=             Data[1];
  //                     LObj.FValueUnit      :=             Data[2];
  //                     LObj.FTimestamp      :=             StrToDateDef( Data[3], 0);
  //                     LObj.FTimestampColor :=             StrToIntDef(  Data[4], Integer( TAlphaColorRec.Red ));
  //                     LObj.FLine1          :=             Data[5];
  //                     LObj.FLine2          :=             Data[6];
  //
  //                     LObj.FCoord.FromString_Location(    Data[7] );
  //
  //                     LObj.FGeoCode        :=             Data[8];
  //
  //                     if Length(Data) >= 10 then
  //                         LObj.FMultiLine  :=             Data[9]
  //                     else
  //                         LObj.FMultiLine  :=             '';
  //
  //                     Result := LObj;
  //                 end
  //                 else
  //                 begin
  //                     Result := nil;
  //                 end;
  //
  //             end
  //
  //                            );


{$ENDREGION}
        //UnMarshal will attempt to create a TTestObject from the TJSONObject data
        //using RTTI lookup - for that to function, the type MUST be defined in a unit

        Result := TJson_Marshal_Base<T>.Create; //( AData );
        Result := LMar.CreateObject(TJson_Marshal_Base<T>, AValue, Result ) as TJson_Marshal_Base<T>;

    finally
        LMar.Free;
    end;
end;

 

[Kryvich 165]

@Lars Fosdal Sorry I cannot help you with it. I tried the attribute [JsonReflectAttribute(ctObject???,rtObject???,TArrayElementInterceptor)] but there is no a converter/reverter type that accepts values of any type: strings, numbers, and objects. Perhaps you may need to create a descendant of TJSONUnMarshal to handle all the options.

 

It would be great if Embarcadero improves JSON support and allow to convert other types to a custom record if this record can take values of other types:

type
  TSwitch = class
  private
    FSwitch: Boolean;
  public
    property Switch: Boolean read FSwitch write FSwitch;
  end;

  TArrayElement = record
  private
    FValue: Variant;
  public
    class operator Implicit(const Value: Integer): TArrayElement;
    class operator Implicit(const Value: string): TArrayElement;
    class operator Implicit(const Value: TSwitch): TArrayElement;
  end;

 

[Lars Fosdal 1792]

Gotta love the level of detail in the help for the interceptors 😞 
http://docwiki.embarcadero.com/Libraries/Rio/en/REST.Json.Interceptors
 

Quote

Embarcadero Technologies does not currently have any additional information.

 

Nudging @David Millington...

 

[Lars Fosdal 1792]

Reported a doc error: https://quality.embarcadero.com/browse/RSP-23024

 

Doh! Wrong issue!

Edited January 4, 2019 by Lars Fosdal
wrong link

[Lars Fosdal 1792]

Correct link:
REST.Json.Interceptors types/methods are undocumented
https://quality.embarcadero.com/browse/RSP-23026

 

Unabashedly mentions @Marco Cantu