Arnaud Bouchez

I don't see any reason why it shouldn't.

FindFirst is definitively the slowest. It needs at least two calls (+ FindClose), and maintain some state in-between.
 

GetFileAtributes() is the fastest under Windows, and fpaccess/euidaccess on POSIX systems.
Note that a fallback to FileAge() is needed in case of sharing violation:
 

function FileExists(const FileName: string): Boolean;
{$IFDEF MSWINDOWS}
// use GetFileAttributes: much faster than original, which uses FileAge=FindFirst
var Attr: Integer;
    LastError: Cardinal;
begin
    Attr := Integer(GetFileAttributesW(pointer(FileName)));
  if Attr <> -1 then
    Result := Attr and FILE_ATTRIBUTE_DIRECTORY = 0 else begin
    LastError := GetLastError;
    Result := (LastError <> ERROR_FILE_NOT_FOUND) and
              (LastError <> ERROR_PATH_NOT_FOUND) and
              (LastError <> ERROR_INVALID_NAME) and
              // (use FileAge to test SHARE_EXCLUSIVE files)
              ((LastError = ERROR_SHARING_VIOLATION) or (FileAge(FileName)<>-1));
  end;
end;
{$ENDIF}
{$IFDEF LINUX}
begin
  Result := euidaccess(PChar(FileName), F_OK) = 0;
end;
{$ENDIF}

(extracted from our Enhanced RTL source code)

But the version currently included in Delphi 10.3 Rio SysUtils.pas is just as fast on Windows.
On POSIX, it uses stats, which is slower than euidaccess().

1 hour ago, Kryvich said:

So MySQL database + FireDAC- mORMot-Windows service could be compiled and run even with Community Edition, if the database and the service are located on the same machine.

If you use the mORMot layer with direct MySQL ODBC provider - or even better https://sourceforge.net/projects/zeoslib/ - you have a 100% Open Source access with the Community Edition, also for remote servers.

The mORMot framework has its own database access layer, which by-pass DB.pas and the TDataSet, and work directly from/to JSON on the database provider, for best efficiency.
You can use FireDAC as DB access layer with mORMot, but this is not mandatory - and even slower in practice.

See https://synopse.info/forum/viewtopic.php?pid=28311#p28311 for actual benchmark numbers.
From this test, ZDBC direct access to MySQL is more than 3 times faster than FireDAC, to retrieve individual objects (7138/s vs 2171/s). This includes the ORM layer, and the JSON serialization/unserialization.
Of course, a local SQLite3 database is the fastest (by far), and fits very well the MicroServices paradigm.

@dummzeuch The feature matrix (page 18ff) states that FireDAC is not in CE - see "Windows/Mac enterprise database support, including MySQL, MariaDB, Microsoft SQL Server, Oracle Database, InterBase, PostgreSQL, Informix, Sybase SQL Anywhere, Microsoft Access, IBM DB2 Server, Firebird, Advantage Database, generic ODBC driver" is not checked, and also MongoDB, most/some WebBroker and DBExpress features, DataSnap and RadServer. 

@Andrea Magni Technically, non-ARC Linux may be supported, but we have some caveats - see https://synopse.info/forum/viewtopic.php?pid=28534#p28534

Now that Delphi 10.3 Rio is out, we had to ensure that our little mORMot would run with this revision.
Since we don't make any commercial software with Delphi any more (we switched to FPC), I downloaded the Community Edition.
We disabled the Error Insight feature, which seems not very stable especially with our mORMot complex units - as with previous Delphi versions.

In respect to the previous "Starter" editions, this CE version includes everything you expect for good work: Win32 + Win64 + OSX + mobile compilers, and even the source code of the RTL and enabled command-line compilers!
The IDE seems really refreshed with its new layout. You feel at home and with some modern and fresh air!
Great work Embarcadero!

Of course, the "Pro" and "Architect" features are missing (like FireDAC, DataSnap or RADServer).
But mORMot is self-contained even for the database access, so with the CE you can write full-features REST/SOA client-servers and MVC web applications, with a simple ORM and high-performance OleDB/ODBC database access to SQlite3, MSSQL, Oracle, Firebird, MySQL and others (by-passing the TDataSet class for better performance) and even NoSQL like MongoDB.
Ensure you follow the Embarcadero license terms when you use mORMot with the Community Edition.

With Rio, the main breaking change was that the PCRE API switched from an UTF-8 to UTF-16, and did hide the UTF-8 function calls.
But after a small patch, now everything works as expected - at least for our regression tests.

See http://blog.synopse.info/post/2018/11/24/mORMot-running-on-Delphi-10.3-Rio

I had to disable ErrorInsight with Delphi 10.3 too, in order to be able to compile anything with mORMot.

No progress since the previous versions about ErrorInsight.

what do you mean by "a couple of IFs" ? 🙂 that it is not reliable in practice / in all cases ?
(I have seen Oracle databases instances down and having crunched data so I don't expect MySQL to be more reliable)

Range.AsInt(0, 11, 2) is IMHO less readable than a regular while loop.
Why not Range.AsInt(0, 2, 11) or Range.AsInt(2, 0, 11) ?

Someone not able to read and understand the following loop would have to learn it ASAP before coding in Delphi any further:

var
  i: integer;
begin
  i := 1;
  repeat
    Writeln(i);
    inc(i, 2);
  until i > 10;  
end.

@Dany Marmur SQLite3 is really fully ACID - see https://www.sqlite.org/lockingv3.html - and it is especially safe and efficient with its WAL mode https://www.sqlite.org/wal.html

I think MySQL is ACID if you use the InnoDB engine.

Perhaps fun for you, but certainly complicated and slower than a simple "while" loop, especially when the process within the loop is small and the range is huge.
The duplicated SetLength() is just so "funny".

Just use a while loop!
This is the clear and efficient way of writing such code, without messing with the "for" variable.

var
  i: integer;
begin
  i := 1;
  while i <= 10 do begin
    Writeln(i);
    inc(i, 2);
  end;
  Readln;
end.

Or a repeat .. until of course:

var
  i: integer;
begin
  i := 1;
  repeat
    Writeln(i);
    inc(i, 2);
  until i > 10;  
  Readln;
end.

I think the compiler is lost since you didn't put any parameter to your external function definition.

It is just the same for the result parameter in RAX as for regular parameters.

Try with the exact function definition including all parameters, and I hope (crossed fingers!) it will be fine.

The "stack frames" parameter in the Project Options is pointless on Win64 IIRC.

Only with an `asm .noframe` directive you may have no stack frame.

IMHO the redirection call won't be noticeable in performance, especially if you write:

asm
    .noframe
    jmp AVXGENERATE_TSLIVERHELPER_NS
end;

I wouldn't play with inline + assembly with Delphi, which has a long history of problems when inlining code...

The SQLite3 documentation you quoted didn't take into account that a client/server layer like mORMot.
And for this use case (mORMot + SQlite3), it states the contrary in https://www.sqlite.org/whentouse.html

Quote

Server-side database

Systems designers report success using SQLite as a data store on server applications running in the datacenter, or in other words, using SQLite as the underlying storage engine for an application-specific database server.

With this pattern, the overall system is still client/server: clients send requests to the server and get back replies over the network. But instead of sending generic SQL and getting back raw table content, the client requests and server responses are high-level and application-specific. The server translates requests into multiple SQL queries, gathers the results, does post-processing, filtering, and analysis, then constructs a high-level reply containing only the essential information.

Developers report that SQLite is often faster than a client/server SQL database engine in this scenario. Database requests are serialized by the server, so concurrency is not an issue. Concurrency is also improved by "database sharding": using separate database files for different subdomains. For example, the server might have a separate SQLite database for each user, so that the server can handle hundreds or thousands of simultaneous connections, but each SQLite database is only used by one connection.

@Hans J. Ellingsgaard You are making assumption, I am afraid.

On the server side, a single SQlite3 insert takes a few microseconds.
A batched SQlite3 from our ORM (i.e. several inserts per requests) writes at 200,000 inserts per second on a laptop.
See https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITL_59
No remote client/server DB I know is so fast, even running on localhost.

And from concurrent writers over HTTP, you got more than 6000 inserts per second on a laptop with 200 concurrent clients (one insert per request).
See https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITL_4

This is faster than regular client/server databases, and we tested it even with 50,000 clients - try to make it with a regular RBMS.

We setup backup and replication in our framework for SQLite3 backup.
As you stated, using an ORM like the one in mORMot would allow to switch to another DB, even a NoSQL one like MongoDB, with no code modification.
But from all our tests, SQLite3 was the fastest when building MicroServices - i.e. Services which require a local storage.

It is a perfect match for mORMot + SQLite3.

But I am biased for sure! 😄

You may even use easily one database per account, if you wish too... this ease maintenability and scaling a lot!

And eventually you would be able to use FPC to compile your server application for Linux, if you need to.

Performance will be much higher than with FireBird or other remote DB, and SQlite3 is perhaps the most well tested DB ever.  https://www.sqlite.org/testing.html
We use it on production with DB of dozen of GB, with no problem - and it usually uses much less disk space than FireBird for instance.

As backend, consider SQlite3 as a local database on the server side. It is perfect for such service storage.
There are several SQLite3 libraries for Delphi, from FireDAC to Zeos/ZDBC or mORMot.

In your code, if you don't know anything about SQL, consider using an ORM like Aurelius or mORMot to work with objects, and not SQL.

@Stefan Glienke

Seems indeed like it was not part of it. 🙂

See https://www.cs.utexas.edu/users/novak/grammar.html

Perhaps the other way around: let strong typing be OFF by default (backward compatible) but enable it to ON for new code requiring it. 🙂

About Unicode/AnsiString it was a requirement due to another breaking change.

@Kryvich Changing the compiler requirements on such would break backward compatibility. This is why I doubt it would happen.

Another problem is about the overloaded functions. The compiler will complain about ambiguous usage, and will abort compilation.
And another is about type helpers: it is for a given type, so you can't use double.ToString as TDistanceMeters.ToString IIRC. I complained against that when they introduced type helpers - I guess noone at EMB uses user defined simple types (whereas I use it everywhere).

@David Heffernan Defining records is indeed the proper way of ensuring very strong type checking. Especially with the implicit conversion introduced with Delphi 10.3. And you would gain methods, e.g. for making conversions explicit (which is even better).

But in practice, I like the fact that "type double" gives at least some information in the IDE (e.g. code completion) which is not needed to be put as parameter or function name.

2 hours ago, Kryvich said:

Regarding the naming of units. It makes sense to name a unit like this: ProjectName.UnitName.pas. ProjectName can be the abbreviated name of your project. And if the project is really big (like Delphi) - add the name of the subproject instead, for ex.: System.Types, Vcl.Forms, FMX.Forms etc. An even Subproject.Subproject.UnitName.pas: FMX.Forms.Border.pas, FMX.ListView.Appearances.pas... 

Yes, camel-casing or using commas are options.
Since some of the code is expected to run on Delphi 7 (for several convenient reasons, no technical ones) we didn't use comma.
But the point of the proposal was that we won't use the project name, but the *folder* name within the unit name, to ensure 1. that the file is easy to locate in the SCM or at file level 2. the folder logic is not about projects, but to ensure data/ui/projects/etc... are uncoupled. Folder layout, therefore unit name layout, therefore the system architecture, should not be project-oriented, but domain-oriented - see https://martinfowler.com/tags/application architecture.html

I just published a blog article about how to write clean code, and also high-performance code.

http://blog.synopse.info/post/2018/11/12/EKON-22-Slides-and-Code

These were sessions at latest EKON 22 conference in Germany: we were more than 140 Delphi developers!

Hope this helps!