Arnaud Bouchez

We use FPC for Linux server-side development, cross-compiling via Lazarus from Windows via https://github.com/LongDirtyAnimAlf/fpcupdeluxe/releases

Then we use https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux to run the executable as console with no VM, and validate the regression tests quickly and efficiently.

Even if FPC/Lazarus is Open Source, its compiler is very good.

IIRC the Delphi for Linux compiler uses ARC, which could become a real PITA to debug for memory leaks for a server application.
At least, FPC has a single memory model on all targets.
See http://blog.marcocantu.com/blog/2017-february-delphi-linux-compiler.html for this platform limitations/requirements.

Look into our OpenSource https://github.com/synopse/SynPDF project.

It is free, fast, creates small PDF, and you can draw your content using a regular VCL TCanvas: the same code could be used on screen and for the PDF.

Also note that there is no "resolution" in PDF, which is uses floating-point coordinates for its vectorial drawings.
If you generate from a TCanvas, your code will use integers, so here you may need to tune the resolution - just like with any VCL code.

1. It uses less disk, but more memory than a plain exe.

2. I have observed is that the combination of Delphi exe  + UPX is often falsely detected as a potential virus threat by some antiviruses.

For me, point 2 is a showstopper, since it is frightening for your users.
 

In fact, it is not allocation-free, it is heap-allocation free, since it needs to allocate its memory from the stack.

And note that the stack size can be pretty limited...
For instance, if the process is run within IIS, stack size is only 256KB (on 32-bit), which could be too small for using allocation-free collections everywhere.
Under Linux, the default thread stack size can be pretty limited in comparison to Windows, too.

What we do in practice in our servers code when we need such temporary storage, is to use a 4KB stack-allocated buffer, with optional heap allocation.

See https://synopse.info/files/html/api-1.18/SynCommons.html#TSYNTEMPBUFFER
There are some methods to pre-initialize the buffer with common patterns (zero bytes, increasing integers, random numbers)....
But you need to use pointer arithmetic, whereas this article uses generics so may a little safer.
4KB is OS memory page size, so should already be there, and avoid to reach 256KB numbers.

Use a Delphi version < 2010 (where the enhanced RTTI has been introduced) or FPC/Lazarus, and to something even smaller, https://github.com/rofl0r/KOL or our https://github.com/synopse/LVCL

As for an embedded database, I would rather use SQlite3 or even FireBird, which has an embedded version.

They are OpenSource, cross-platform, well maintained and have high feature set.

On server side, we log a lot of stuff... almost everything possible information. It is needed for post-mortem or post-bug forensic.
Of course, our mORMot framework logs automatically all SOA calls, and all ORM and/or SQL process.
And it also logs all exceptions.
See https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITL_73

This was for low-level text logs.
At higher level, we also log all high-level SOA calls, in form of local Sqlite3 databases.
This is very efficient to make any kind of statistics, via some simple SQL SELECT.
https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITLE_445

Here is what we usually do:

1. Store logs in a dedicated folder, with proper read policy.
   -> especially under Unix, all background daemons run with a dedicated user, and the log files have chmod 640 to restrain read access.
2. Always run the logs in Verbose mode, even on production, unless the execution environment is really constrained.
   (you never know what will happen, and detailed logs is a need for proper investigation, especially on production)
   -> you need a logging solution which is fast, and can log dozen MB/s with no performance penalty (e.g. our SynLog.pas unit writes in a background thread)
3. Rotate logs to keep history as long as possible
   -> SynLog.pas has built-in SynLZ-compression of the logs for very fast and efficient rotation
4. Never log passwords, or such sensitive information.
   -> password are always stored and persisted in salted + hashed form - but when transmitted as parameters, or as settings, they should be encrypted - even a naive encryption may be enough like with a TSynPersistentWithPassword class from SynCommons.pas
   -> more generally, sensitive information should be handled in memory with strong cryptography, e.g. via our cross-platform CryptDataForCurrentUser() from SynCrypto.pas
5. Have a way to hide unexpected content, when log is automated.
   -> in mORMot.pas, you can register some sensitive types which parameters will be logged as "***" during SOA automated logging of calls.
6. Ensure all those policies are always setup by default, and on all supported platforms
   -> today, we should ensure that Linux execution should be not weaker than Windows as a target (and it is easy to make executable safer under Linux)
7. Optionally use a third-party SaaS solution to store your logs in the long term, with proper encryption and search abilities
   -> like https://livemon.com - server-side written in object pascal 🙂

Most of it is available out-of-the-box in http://mormot.net

Check https://www.delphitools.info/2013/10/30/efficient-string-building-in-delphi/

TStringBuilder has been enhanced in latest Delphi 10.3 IIRC.

Our OpenSource `TTextWriter` class, which works on UTF-8 (which is very likely your eventual CSV file encoding), is very fast (it avoids most memory allocation), and has a `TTextWriter.CancelLastComma` method, as you expect.

See https://synopse.info/files/html/api-1.18/SynCommons.html#TTEXTWRITER_CANCELLASTCOMMA

5 hours ago, Alexander Brazda said:

SetLength would allocate a new string and assign this to AText, so reference counting would decrease inside the funcion instead outside.

This is wrong if the refcount is 1: in that case (which happens when you just appended some texts to a string variable), SetLength() won't copy the string, just call ReallocMem() which is a no-op with FastMM4 over a minus one byte buffer, and put a #0 at the end.

See how _UStrSetLength() is implemented in System.pas.

or even try 

procedure DeleteLastChar(var aText: string; aLastChar: char); inline;
var len: {$ifdef FPC}PSizeInt{$else}PInteger{$endif};
begin
  len := pointer(aText);
  if len <> nil then begin
    dec(len);
    if PChar(pointer(aText))[len^ - 1] = aLastChar then begin
      PChar(pointer(aText))[len^ - 1] := #0;   
      dec(len^);
     end;
  end;
end;

which has also the advantage of working with both Delphi and FPC.

But all these sounds a bit like premature optimization to me... 😞

If you are SURE that the string has a reference count of 1, e.g. if it just has been allocated and not assigned/copied anywhere else, you can simply try:

procedure DeleteLastChar(var aText: string; aLastChar: char); inline;
begin
  if (aText <> '') and (aText[length(aText)] = aLastChar) then
    dec(PInteger(PAnsiChar(pointer(aText)) - SizeOf(integer))^);
end;

This will avoid a call the heap manager.

Then do proper benchmark.

On 12/14/2018 at 3:11 PM, Kryvich said:

You're right. We can compare the squares of distances:

  checkDistance2 := Sqr(checkDistance);
  ...
  dist2 := Sqr(X1-X2) + Sqr(Y1-Y2);
  if dist2 < checkDistance2 then .....

 

Square of distances is indeed the way I would have done it: faster (no branch, no call), and no complex math.

FastMM4, in FullDebugMode is able to track such pointer problems.

I always use it, and also the FPC's Heaptrc unit (compile with -gl) which can find some other problems.

I use pointers only when the code needs it, mainly for performance reasons.
Otherwise, high-level structures like classes or dynamic arrays are just enough.

On 12/8/2018 at 9:19 AM, Primož Gabrijelčič said:

I would really like Embarcadero to enhance `with` block in that direction, so I could write:

Why not directly make it `using` and replace the `begin` `end` with `{  }` ?
:)

16 hours ago, jbg said:

There are changes in the 64 bit compiler (e.g. some loop unrolling)

Where this information came from?
If it is the case, I didn't see any performance enhancement at execution, at least from our mORMot regression tests, which run a lot of performance measurements.
In fact, most loop unrolling could even be slower on modern CPUs - for instance, AES unrolled loop which was faster on P4 is slower than a tuned rolled loop.
I doubt EMB did make proper compiler optimization in loop unrolling, and let the inlining performance problems remain.

I don't see any reason why it shouldn't.

FindFirst is definitively the slowest. It needs at least two calls (+ FindClose), and maintain some state in-between.
 

GetFileAtributes() is the fastest under Windows, and fpaccess/euidaccess on POSIX systems.
Note that a fallback to FileAge() is needed in case of sharing violation:
 

function FileExists(const FileName: string): Boolean;
{$IFDEF MSWINDOWS}
// use GetFileAttributes: much faster than original, which uses FileAge=FindFirst
var Attr: Integer;
    LastError: Cardinal;
begin
    Attr := Integer(GetFileAttributesW(pointer(FileName)));
  if Attr <> -1 then
    Result := Attr and FILE_ATTRIBUTE_DIRECTORY = 0 else begin
    LastError := GetLastError;
    Result := (LastError <> ERROR_FILE_NOT_FOUND) and
              (LastError <> ERROR_PATH_NOT_FOUND) and
              (LastError <> ERROR_INVALID_NAME) and
              // (use FileAge to test SHARE_EXCLUSIVE files)
              ((LastError = ERROR_SHARING_VIOLATION) or (FileAge(FileName)<>-1));
  end;
end;
{$ENDIF}
{$IFDEF LINUX}
begin
  Result := euidaccess(PChar(FileName), F_OK) = 0;
end;
{$ENDIF}

(extracted from our Enhanced RTL source code)

But the version currently included in Delphi 10.3 Rio SysUtils.pas is just as fast on Windows.
On POSIX, it uses stats, which is slower than euidaccess().

1 hour ago, Kryvich said:

So MySQL database + FireDAC- mORMot-Windows service could be compiled and run even with Community Edition, if the database and the service are located on the same machine.

If you use the mORMot layer with direct MySQL ODBC provider - or even better https://sourceforge.net/projects/zeoslib/ - you have a 100% Open Source access with the Community Edition, also for remote servers.

The mORMot framework has its own database access layer, which by-pass DB.pas and the TDataSet, and work directly from/to JSON on the database provider, for best efficiency.
You can use FireDAC as DB access layer with mORMot, but this is not mandatory - and even slower in practice.

See https://synopse.info/forum/viewtopic.php?pid=28311#p28311 for actual benchmark numbers.
From this test, ZDBC direct access to MySQL is more than 3 times faster than FireDAC, to retrieve individual objects (7138/s vs 2171/s). This includes the ORM layer, and the JSON serialization/unserialization.
Of course, a local SQLite3 database is the fastest (by far), and fits very well the MicroServices paradigm.

@dummzeuch The feature matrix (page 18ff) states that FireDAC is not in CE - see "Windows/Mac enterprise database support, including MySQL, MariaDB, Microsoft SQL Server, Oracle Database, InterBase, PostgreSQL, Informix, Sybase SQL Anywhere, Microsoft Access, IBM DB2 Server, Firebird, Advantage Database, generic ODBC driver" is not checked, and also MongoDB, most/some WebBroker and DBExpress features, DataSnap and RadServer. 

@Andrea Magni Technically, non-ARC Linux may be supported, but we have some caveats - see https://synopse.info/forum/viewtopic.php?pid=28534#p28534

Now that Delphi 10.3 Rio is out, we had to ensure that our little mORMot would run with this revision.
Since we don't make any commercial software with Delphi any more (we switched to FPC), I downloaded the Community Edition.
We disabled the Error Insight feature, which seems not very stable especially with our mORMot complex units - as with previous Delphi versions.

In respect to the previous "Starter" editions, this CE version includes everything you expect for good work: Win32 + Win64 + OSX + mobile compilers, and even the source code of the RTL and enabled command-line compilers!
The IDE seems really refreshed with its new layout. You feel at home and with some modern and fresh air!
Great work Embarcadero!

Of course, the "Pro" and "Architect" features are missing (like FireDAC, DataSnap or RADServer).
But mORMot is self-contained even for the database access, so with the CE you can write full-features REST/SOA client-servers and MVC web applications, with a simple ORM and high-performance OleDB/ODBC database access to SQlite3, MSSQL, Oracle, Firebird, MySQL and others (by-passing the TDataSet class for better performance) and even NoSQL like MongoDB.
Ensure you follow the Embarcadero license terms when you use mORMot with the Community Edition.

With Rio, the main breaking change was that the PCRE API switched from an UTF-8 to UTF-16, and did hide the UTF-8 function calls.
But after a small patch, now everything works as expected - at least for our regression tests.

See http://blog.synopse.info/post/2018/11/24/mORMot-running-on-Delphi-10.3-Rio

I had to disable ErrorInsight with Delphi 10.3 too, in order to be able to compile anything with mORMot.

No progress since the previous versions about ErrorInsight.

what do you mean by "a couple of IFs" ? 🙂 that it is not reliable in practice / in all cases ?
(I have seen Oracle databases instances down and having crunched data so I don't expect MySQL to be more reliable)

Range.AsInt(0, 11, 2) is IMHO less readable than a regular while loop.
Why not Range.AsInt(0, 2, 11) or Range.AsInt(2, 0, 11) ?

Someone not able to read and understand the following loop would have to learn it ASAP before coding in Delphi any further:

var
  i: integer;
begin
  i := 1;
  repeat
    Writeln(i);
    inc(i, 2);
  until i > 10;  
end.