Arnaud Bouchez

This was time for a new "stable" release of the Open Source mORMot 2 framework! You can find it on our GitHub repository: https://github.com/synopse/mORMot2/releases/tag/2.3.stable 🙂 This release will be a Long Term Support Security Branch, as we will maintain it for the next years for main bugs and security fixes. Added - Swagger/OpenAPI Client Generator - IDocList/IDocDict Containers - SID/DACL/SACL/SDDL/ACE Security Objects - async web server: IOCP support on Windows, metrics gathering and standard logging - TSynMustache can work on plain data via RTTI, in addition to TDocVariant - introducing TRttiMap for DTO process. Changed - Upgraded SQLite3 to 3.46.1 - Enhancements to the LDAP client, HTTP/HTTPS client, Kerberos auth, Peer Cache, ORM. - Lots other bug fixes, optimisations and enhancements. More information on our blog: https://blog.synopse.info/?post/2024/10/16/Release-of-mORMot-2.3-Stable

Personal biais: the mORMot 2 Open Source framework has a very efficient JSON library, and several ways to use it: - from RTTI, using classes, records, collections, dynamic arrays, mORMot generics... - from variants, and a custom "document" variant type to store JSON objects or arrays... - from high-level IDocList / IDocDict holders. See https://blog.synopse.info/?post/2024/02/01/Easy-JSON-with-Delphi-and-FPC It is perhaps the fastest library available, working on Delphi and FPC, with unique features, like: list := DocList('[{"a":10,"b":20},{"a":1,"b":21},{"a":11,"b":20}]'); // sort a list/array by the nested objects field(s) list.SortByKeyValue(['b', 'a']); assert(list.Json = '[{"a":10,"b":20},{"a":11,"b":20},{"a":1,"b":21}]'); // create a dictionary from key:value pairs supplied from code dict := DocDict(['one', 1, 'two', 2, 'three', _Arr([5, 6, 7, 'huit'])]); assert(dict.Len = 3); // one dictionary with 3 elements assert(dict.Json = '{"one":1,"two":2,"three":[5,6,7,"huit"]}'); // convert to JSON with nice formatting (line feeds and spaces) Memo1.Caption := dic.ToString(jsonHumanReadable); // integrated search / filter assert(DocList('[{ab:1,cd:{ef:"two"}}]').First('ab<>0').cd.ef = 'two');

@David Schwartz Command line tool is now available: https://github.com/synopse/mORMot2/tree/master/src/tools/mopenapi (this is a good showcase how cross-platform and cross-compiler command line switches parsing could be done with mORMot) I will release a Win32 binary somewhere in the next days. Thanks for the feedback!

This is not yet clean OOP for sure, since it would break the Liskov principle. IMHO there is no way to make it "elegant". My guess is that the "elegant" way is to use composition. That is, compose a "shared object", available to your function, which would have a circle and a rectangle properties, then additional properties. OOP should be as natural as possible. If you are fighting against your types, some refactoring may be needed. Multiple inheritance is IMHO never mandatory, for a clean OOP model. And always refer the the SOLID principles!

I have edited the blog post to make it clear:

Indeed. You need some basic units of mORMot for the client code, because it uses its HTTP, RTTI and JSON cross-platform and cross-compiler abilities. But are not tied to use the other mORMot ORM or SOA features, which are uncoupled from this. To use the client methods, you just use the generated class methods, as with regular Delphi code. No fancy mORMot usage for the end user: the rodent stays hidden in his hole. You have plenty of output sample in the blog article. For instance: https://gist.github.com/synopse/e0dacfcc870db67013de55e43276f07b unit auth.api; {$I mormot.defines.inc} interface { -------------------------------------------------------------------- AUTHENTIQ API client as TAuthClient class Generated 6 Sep 2024 by ab via mormot2tests - DO NOT MODIFY BY HAND! -------------------------------------------------------------------- } uses classes, sysutils, mormot.core.base, mormot.core.unicode, mormot.core.text, mormot.core.buffers, mormot.core.datetime, mormot.core.rtti, mormot.core.json, mormot.core.variants, mormot.net.client;

Since it seems to be a trending subject, a new OpenAPI client generator is now available for Delphi: https://blog.synopse.info/?post/2024/09/06/Swagger/OpenAPI-Client-Generator-for-Delphi-and-FPC ;D Here are some features of our OpenAPI code generator for Delphi and FPC: - Fully OpenSource, both generator and generated source code - Use high-level pascal records and dynamic arrays for "object" DTOs - Use high-level pascal enumerations and sets for "enum" values - Translate HTTP status error codes into high-level pascal Exceptions - Recognize similar "properties" or "enum" to reuse the same pascal type - Support of nested "$ref" for objects, parameters or types - Support "allOf" attribute, with proper properties inheritance/overloading - Support "oneOf" attribute, for strings or alternate record types - Support of "in":"header" and "in":"cookie" parameter attributes - Fallback to variant pascal type for "oneOf" or "anyOf" JSON values - Generated source code units are very small and easy to use, read and debug - Can generate very detailed comment documentation in the unit source code - Tunable engine, with plenty of generation options (e.g. about verbosity) - Tested with several Swagger 2 and OpenAPI 3 reference content But still not fully compliant to all existing files: feedback is welcome! Here is a snippet of a generated method signature, with high-level Data Transfer Objects as records (TUserShort/TDBAccount), and complete documentation, with proper Exceptions generation for the HTTP results error codes: // post_account_res_add_grant_auth [post] /accounts/{uuid}/add-grant-auth/ // // Summary: Gives a user permissions to grant authorization on the account // Description: // Roles: vm_admin for vm object, templates otherwise // // Params: // - [path] Uuid (required): Hypervisor uuid // - [body] Payload (required) // // Responses: // - 200 (main): Success // - 400 [EValidationErrorResponse]: Parameters have invalid format or type // - 401 [EUnauthorizedResponse]: User is not authenticated // - 403 [EForbiddenResponse]: User has insufficient permissions // - 404 [EResourceNotFoundError]: Hypervisor not found // - 422 [EIntegrityErrorResponse]: Parameters have valid format but are not compatible // with the server state function PostAccountResAddGrantAuth(const Uuid: string; const Payload: TUserShort): TDbAccount; The generated code is compatible even with oldest Delphi (7-2009) - nice to add native REST/HTTP client abilities to an existing monolithic application. It is even fully compatible with FPC, because we should not be tied to a single compiler. The source code of the generator is a single unit which leverages the mORMot RTTI and JSON kernel for its internal plumbing. And you will find several generated sample code units in the blog article, to make yourself your idea about the level of this little unit. Feedback is welcome!

Just to inform anyone looking for an OpenAPI client code generator, that I just made a new one, for both Delphi and FPC. It seems to be more advanced than Wagner's generator (e.g. it converts errors, enums and allOf/oneOf attributes), and is fully Open Source. https://blog.synopse.info/?post/2024/09/06/Swagger/OpenAPI-Client-Generator-for-Delphi-and-FPC Here are the top features of this OpenAPI client code generator for Delphi and FPC: Use high-level pascal records and dynamic arrays for "object" DTOs Use high-level pascal enumerations and sets for "enum" values Translate HTTP status error codes into high-level pascal Exceptions Recognize similar "properties" or "enum" to reuse the same pascal type Support of nested "$ref" for objects, parameters or types Support "allOf" attribute, with proper properties inheritance/overloading Support "oneOf" attribute, for strings or alternate record types Support of "in":"header" and "in":"cookie" parameter attributes Fallback to variant pascal type for "oneOf" or "anyOf" JSON values Each method execution is thread-safe and blocking, for safety Generated source code units are very small and easy to use, read and debug Can generate very detailed comment documentation in the unit source code Tunable engine, with plenty of generation options (e.g. about verbosity) Leverage the mORMot RTTI and JSON kernel for its internal plumbing Compatible with FPC and oldest Delphi (7-2009) Tested with several Swagger 2 and OpenAPI 3 reference content, but your input is welcome, because it is not fully compliant! Hoping you may find it interesting. The blog article has several example of the actual output of the generator, from several sources. It is the kind of information I would have wanted to see with other libraries. Use the source, Luke!

Two last weeks of the challenge... Still time to post your own entry!

And don't forget the command line parser available in mORMot 2. - Works on all OS; - Works on Delphi and FPC; - Can auto-generate the help content with proper formatting; - Minimal code writing. In practice, resulting code is short and efficient: https://github.com/synopse/1brc-ObjectPascal/blob/main/entries/abouchez/src/brcmormot.lpr#L446

This is a "classical" problem with Delphi, overloaded functions, and floating points values. IMHO using currency is an awful workaround here, and should not be kept in any serious codebase. It works "by chance". Another workaround, still using the Math unit, is to make the computation in two steps: function RoundExDouble(x: double): Double; begin x := x * 10; Result := Ceil(x) / 10; end; function RoundExInteger(x: double): Integer; begin x := x * 10; Result := Ceil(x); end; So the expected Ceil(double) overload is clearly picked up by the compiler. Or define our own unique Ceil() function, which does not depend on Math.pas and ensure we use the right value type: function ceil(x: double): integer; begin result := trunc(x) + ord(frac(x) > 0); end; This is this solution I used in my entry of the challenge: stay away from Math.pas. 😉

I missed the info, sorry. For a SQL solution, it is very good. Out of curiosity, how much memory does it need for the 16.5 GB file? Does it use SQLite3 and its virtual tables internally for its SQL dialect (something like https://www.sqlite.org/csv.html)?

Easy, and wrong. You are reading the station weathers reference data with one row per station. And making a min/max/average of a single data per station. The challenge is to read a 1 billion (1,000,000,000) rows of CSV data for all those 41343 stations, and compute it. There is a generator of a 16GB CSV file to inject and process. So 0.33s for 41343 rows would make around 8000 seconds, i.e. 5.5 days.

Since years, our Open Source mORMot framework offers several ways to work with any kind of runtime arrays/objects documents defined, e.g. via JSON, with a lot of features, and very high performance. Our TDocVariant custom variant type is a powerful way of working with such schema-less data, but it was found confusing by some users. So we developed a new set of interface definitions around it, to ease its usage, without sacrificing its power. We modelized them around Python Lists and Dictionaries, which is proven ground - with some extensions of course. Two one-liners may show how our mORMot library is quite unique in the forest/jungle of JSON libraries for Delphi (and FPC): +] assert(DocList('[{ab:1,cd:{ef:"two"}}]')[0].cd.ef = 'two'); assert(DocList('[{ab:1,cd:{ef:"two"}}]').First('ab<>0').cd.ef = 'two'); Yes, this code compiles and run on Delphi - even the antique Delphi 7. 😆 If you compare e.g. to how the standard JSON library works in the RTL, with all its per-node classes, you may find quite a difference! And there is no secret in regard to performance, that our JSON processing is fast. More info: https://blog.synopse.info/?post/2024/02/01/Easy-JSON-with-Delphi-and-FPC

OpenSource unit https://github.com/synopse/mORMot2/blob/master/src/core/mormot.core.zip.pas supports deletion - just use TZipWrite.CreateFrom() with a list of files to ignore when reading the existing zip.