JSON benchmarks

[pyscripter 750]

hydrobyte/TestJSON: A simple project to test JSON libraries with Delphi and C++Builder. presents JSON library benchmarks comparing a large number of alternatives.

 

One thing that strikes me, is that the System.JSON rtl library is doing relatively well compared to the competition, both in terms of performance and in terms of JSON validation.   With the exception of Find, is very competitive in all other areas.

 

I have seen many claims that System.JSON is very slow and that the xyz library is so many times faster.   Do these benchmarks suck (like most benchmarks)?   Or is it the case that System.JSON is not that bad?  What is your experience?

[Tommi Prami 147]

13 hours ago, pyscripter said:

hydrobyte/TestJSON: A simple project to test JSON libraries with Delphi and C++Builder. presents JSON library benchmarks comparing a large number of alternatives.

Way cool! 

Is there test/comparison is Objects and/or Record serialization supported and speed etc?? Did not check code, because bit busy...

I've used Grijjy in one of my hobby projects for serializing objects and/or records. Can't remember which they all are, poossibly both)

Edited Friday at 06:03 AM by Tommi Prami

[pyscripter 750]

1 hour ago, Tommi Prami said:

Is there test/comparison is Objects and/or Record serialization supported and speed etc??

No. Most of the alternatives do not support serialization.  Grizzy and Superobject do.   Delphi offers a couple of ways.   But serialization is not  necessarily dependent on JSON parsing.   For example NEON is using System.JSON.   

Edited Friday at 07:06 AM by pyscripter

[Stefan Glienke 2138]

The tests ignore some subtle differences in the implementations. While some implementations ensure that you don't create multiple pairs with the same name the System.JSON AddPair method simply ignores that.

 

You can test that by adding a fLib.Add(SKey, SVal); after the loop in TTestSpeedRun.DoSubTestGen - some libraries will report 50000 items, while some report 50001 - System.JSON for example.

The fact that JDO uses a linear search here is a bit disappointing and the reason it completely falls off the cliff on the generate test.

[Vincent Gsell 14]

16 hours ago, pyscripter said:

I have seen many claims that System.JSON is very slow and that the xyz library is so many times faster.   Do these benchmarks suck (like most benchmarks)?   Or is it the case that System.JSON is not that bad?  What is your experience?

Hi, 

For me, certains of the fastest JSON  libraries that I practiced is just not present (among others, Mormot, pasjson, etc.).
 

41 minutes ago, Stefan Glienke said:

The tests ignore some subtle differences in the implementations. While some implementations ensure that you don't create multiple pairs with the same name the System.JSON AddPair method simply ignores that.

Indeed ! And validation test could be usefull in this kind of bench ! 

 

It is astonishing that many libs does not pass certain basic "proper Json format" validation ! 

• An empty object can be represented by {}
• Arrays are encapsulated within opening and closing square brackets
• An empty array can be represented by []
• Each member should have a unique key within an object structure

[Der schöne Günther 336]

As far as I know, nowhere does it specify that keys must be unique.

[Stefan Glienke 2138]

5 minutes ago, Der schöne Günther said:

As far as I know, nowhere does it specify that keys must be unique.

No, but according to RFC 8259, they *should* be unique.

 

I just pointed out that by supporting that feature, there needs to be done more than simply adding pairs without any lookup.

In fact, it looks like some implementations don't even allow adding duplicate names.

Edited Friday at 09:37 AM by Stefan Glienke

[Lars Fosdal 1851]

IMO, if you need multiple values for a field, you should use a list element.

Duplicate key/value pairs in JSON just doesn't make sense.
https://JsonLint.com does not like them either.

[pyscripter 750]

Since you asked for Serializer benchmakrs:

 

paolo-rossi/delphi-neon: JSON Serialization library for Delphi includes a benchmark against the Rest.Json serializer and it beats it hand down:

 

 

I have replaced the Rest.Json serializer with the System.Json.Serializers TJSONSerializer.  Here are the results:

 

 

So now TJsonSerializer beats Neon hands down.

 

TJsonSerializer looks good but it has some rough edges.   To run the benchmarks I had to add a converter that handles Enumerated values as strings instead of the default integers:

type
  TEnumStringConverter = class(TJsonConverter)
  public
    function CanConvert(ATypeInf: PTypeInfo): Boolean; override;
    function ReadJson(const AReader: TJsonReader; ATypeInf: PTypeInfo; const AExistingValue: TValue;
      const ASerializer: TJsonSerializer): TValue; override;
    procedure WriteJson(const AWriter: TJsonWriter; const AValue: TValue;
      const ASerializer: TJsonSerializer); override;
  end;

{ TEnumStringConverter }

function TEnumStringConverter.CanConvert(ATypeInf: PTypeInfo): Boolean;
begin
  // This converter can handle any type that is an enumeration
  Result := (ATypeInf.Kind = TTypeKind.tkEnumeration) and
    (ATypeInf <> TypeInfo(Boolean));
end;

function TEnumStringConverter.ReadJson(const AReader: TJsonReader; ATypeInf:
    PTypeInfo; const AExistingValue: TValue; const ASerializer:
    TJsonSerializer): TValue;
var
  LIntValue: Integer;
begin
  LIntValue := System.TypInfo.GetEnumValue(ATypeInf, AReader.Value.AsString);

  if LIntValue = -1 then // GetEnumValue returns -1 if the name is not found
    raise EJsonSerializationException.CreateFmt('Invalid string value "%s" for enumeration "%s".', [AExistingValue.AsString, ATypeInf.Name]);

  // Create a TValue of the specific enum type using its ordinal value.
  Result := TValue.FromOrdinal(ATypeInf, LIntValue);
end;

procedure TEnumStringConverter.WriteJson(const AWriter: TJsonWriter; const
    AValue: TValue; const ASerializer: TJsonSerializer);
begin
    AWriter.WriteValue(System.TypInfo.GetEnumName(AValue.TypeInfo, AValue.AsOrdinal));
end;

See also:  

 

Bummer: System.Json.Converters already includes TJsonEnumNameConverter that does the job.

 

 

Edited Friday at 03:29 PM by pyscripter

[Stefan Glienke 2138]

FWIW, such benchmarks should also run in parallel because especially string-related parsing is usually full of heap allocations that can suffer highly in multithreading.

 

I am also sure that @Arnaud Bouchez could join the game and stomp all these libraries into the ground performancewise with mormot 

Edited Friday at 10:45 PM by Stefan Glienke

[Arnaud Bouchez 408]

To be fair, the tests do not seem very realistic to me.

 

And using any non-standard library for a core process like JSON should be not only with Delphi, but also with FPC.
We are on the 21st century, and modern object pascal is not just Delphi. ;)

[Anders Melander 1988]

14 minutes ago, Arnaud Bouchez said:

We are on the 21st century, and modern object pascal is not just Delphi. 😉