Arnaud Bouchez

Your xoroshiro128starstar implementation is not thread-safe, by the way.

Why not just use (Int64(Random(maxInt)) shl 32) or Random(maxInt) which is fine and cross-compiler?

Or on modern CPU:

function RdRand32: cardinal; {$ifdef CPU64}
{$ifdef FPC}nostackframe; assembler; asm{$else} asm .noframe {$endif FPC} {$else}
{$ifdef FPC}nostackframe; assembler;{$endif} asm {$endif}
  // rdrand eax: same opcodes for x86 and x64
  db $0f, $c7, $f0
  // returns in eax, ignore carry flag (eax=0 won't hurt)
end;

function RdRand64: QWord;
begin
  result := (QWord(RdRand32) shl 32) or RdRand32;
end;

Note that RDRAND opcode is not available on old processor, not very fast, and sometimes returns 0 of -1 on AMD... but it is a good seed for entropy.

See also the Random32 function in https://github.com/synopse/mORMot/blob/master/SynCommons.pas which is based on Lecuyer, and use good entropy seed.

and the TAESPRNG class in https://github.com/synopse/mORMot/blob/master/SynCrypto.pas which is a true cryptographic random generator, using AES-NI very fast asm.
Both are thread-safe and use the NIST SP 800-90A compliant RDRAND Intel x86/x64 opcode as entropy source, if available.

I would not put the behavior change regarding of $R+ setting.
Your library should better be consistent and never need a re-compilation if you need to change e.g. from DEBUG to RELEASE mode.

My idea would be to raise an exception in case of out of bounds error - just like TStringList/TList.
It is what most users expect, and will be confident and pleased with.

In mORMot, the trick is that I also give direct access (as a field, not as a getter - even inlined) to the raw pointer of array storage. So the user can by-pass the range check if needed, e.g. by using a local pointer variable, and proper pointer arithmetic. I document the property as low-level and raw access to the values, to be used at your own risk.
Then advanced users, and other parts of your own library which is safely written and tested, could be able to bypass the range check, by accessing directly the pointer values.

Try to use the WinHttp API instead, for a simple HTTPS client.

You may use Automation, or try an old version of https://github.com/synopse/mORMot/blob/master/SynCrtSock.pas which should work with Delphi 5.

SQLite3 supports IIF() so it should work even on virtual tables.

I don't know why FireDac Local SQL rejects it.

https://www.trichview.com/features/trichview.html is a nice component if you find the default TRichEdit too limited.
It is very customizable, and easy to work with in your use-case, if I understand correctly.
It has plenty of samples, and good support. But it is Windows only.

Did you try with another SQLite3 library?
That was my point. 

Now I am confused.

Is you SQL statement correct outside of FireDAC?
You need to try it to be sure that the problem is in FireDAC configuration, not in your SQL itself.

Your usage seems coherent with https://sqlite.org/lang_corefunc.html#iif

Does your SQL statement returns the data on an external SQlite3 explorer program?

If it does, then the problem is in FireDAC.

Why not just a TRichEdit component?

It is easy to store the information in a simple wiki-like format - a MarkDown subset for instance - then convert it into RTF and display in the TRichEdit component.

I would rather use RawByteString for several reasons:

1. Proper reference counting;

2. (Slightly) faster allocation.
3. Better debugging experience in your case.

Some details:
1. TBytes = array of byte has a less strict reference counting. If you modify a TBytes item, all its copied instances will be modified. Whereas with RawByteString, the string will be made unique before modification (it is called Copy on Write - aka COW).

2. SetLength(TBytes) will allocate the memory and fill it with zeros, whereas SetLength(RawByteString) will just allocate the memory. If you use a memory buffer which will immediately be filled with some data, no need to fill it with zeros first.

3. Even if your RawByteString has some binary, the ASCII characters will be easier to read e.g. #2#0#7'Some Text'#0'Some other text'#49.

I don't understand why I would be involved in this discussion..

My only knowledge is that Delphi 2007 dcc32.exe could work with the .bpl not in c:\windows\system32. I only use .dcu anyway for my command-line D2007 compilation script, modifying the dcc32.cfg file and putting the dcu in a small sub folder:

c:\progs\Delphi2007>dir /s /w
 Le volume dans le lecteur C n'a pas de nom.
 Le numéro de série du volume est C006-BF3C

 Répertoire de c:\progs\Delphi2007

[.]   [..]  [bin] [lib]
               0 fichier(s)                0 octets

 Répertoire de c:\progs\Delphi2007\bin

[.]           [..]          dcc32.cfg     DCC32.EXE     rlink32.dll
               3 fichier(s)        1 087 592 octets

 Répertoire de c:\progs\Delphi2007\lib

[.]             [..]            ActiveX.dcu     Classes.dcu     ComConst.dcu
CommCtrl.dcu    ComObj.dcu      Consts.dcu      Contnrs.dcu     Graphics.dcu
ImageHlp.dcu    IniFiles.dcu    Math.dcu        Messages.dcu    Registry.dcu
RTLConsts.dcu   SyncObjs.dcu    SysConst.dcu    SysInit.dcu     System.dcu
SysUtils.dcu    Types.dcu       TypInfo.dcu     Variants.dcu    VarUtils.dcu
Windows.dcu     WinInet.dcu     WinSpool.dcu
              26 fichier(s)        2 346 725 octets

     Total des fichiers listés :
              29 fichier(s)        3 434 317 octets
               8 Rép(s)   6 376 325 120 octets libres

c:\progs\Delphi2007>

My remark was a general one. It was not about this explicit code.

Writing `TPopupMenu(_Sender)` is a hard-cast which should never appear in most code, unless you have a very vague/legacy event like a TNotifyEvent. The VCL requires it.
 

For UI code, I would write:

 if _Sender is TPopupMenu then begin
    // sometimes Delphi 10.4.1 passes a TMenuItem here, no idea how this can happen, but it caused a runtime error
    AppendMenuItem(_Sender as TPopupMenu);
  end;

The code is somewhat redundant, because "is" is actually doing the class hierarchy check that "as" is doing: calling internally InheritsFrom() twice
But it is only slightly slower - unnoticeable on a VCL app, since passing a Windows message will be slower than this "as" execution.
IMHO it is safer and easier to read and to maintain.
Having used "as" at first hand did indeed give a hint about the initial problem.

Hardcasting is like premature optimization on client side. 

Unconditional cast should never be used on UI/Client side. Always uses "is" or "as".
It may be used only when performance really matters, e.g. on core server code, with a large test coverage, and only on proven bottlenecks - which are very rare.

The first is to write "as" to raise an exception. Then - maybe - use direct class casting. Only if really needed.

IMHO such direct class casting - or cascaded "if .. is .. then ... else if .. is .. then ..." may be a smell for potential breaking some of the SOLID principles.
In VCL events, we use Sender: TObject most of the time - it could be seen as a design flaw / legacy debt. But it is a reality.
We should be able to avoid direct class casting in most of our modern code.

Could you put a reproducible sample somewhere, e.g. as gist?

21 hours ago, borni69 said:

Thanks looks like  

TCharacter.IsValid(s[i]))

is not suported in 10.4

I don't get what TCharacter.IsValid() was supposed to mean. Sounds like a big confusion from the Embarcadero RTL people.
In UTF-16 you may need two WideChar to encode an Unicode glyph - it is called a surrogate pair.
So if you want to check the UTF-16 encoding validity, you have to work at the string level, or at least test two WideChars at once when needed.

I guess this may be the reason why it disappeared. Confused and confusing.

I don't know where #127..#160 comes from. It is valid set of chars, e.g. in Europe for accentuated characters like é à â.

You are making a confusing in encoding. A Delphi string is UTF-16 encoded, so #127..#160 are some valid UTF-16 characters.

What you call "character" is confusing. #11 is a valid character, in terms of both UTF-8 and UTF-16 as David wrote.

1 hour ago, Anders Melander said:

 

Funny name you say? How about he optimize it with Gnome sort instead:

It is also known as "stupid sort".

Best algorithm name ever.

22 minutes ago, Hans J. Ellingsgaard said:

A count(*) query, on the other hand,  has a very limited impact on the server. 

I don't agree. This is highly depending on the DB itself.
A count(*) could take a lot of time for a huge database, or with a complex query with no proper indexes.

You could have a count(*) of a few rows (e.g. one or two rows) which takes a lot of time on big tables with a SQL request bypassing the indexes.
In this case, it is better to retrieve the few result rows instead of making a select count(*) following by a select *

You may either:

• Refine your SQL query, e.g. by adding some proper indexes to the DB, to make it faster on the server;
• Actually retrieve all data at once (maybe into an in-memory list/array), but use Application.ProcessMessages within the loop to avoid UI freezing;
• Don't care about exact record count: just fetch the first 1000 for instance, and write '>1000' on the UI if there are more rows.

As I wrote it never stores a double - SQLite3 doesn't support TDateTime double which is Ole/Windows specific.

In your code, text will be stored.

It is because that SQliteStudio display what is stored, i.e. text, trying several date/time layouts.

It seems that FireDac expects ISO-8601 encoding - just as SQLite3. And don't forget to set the seconds - even :00.

IIRC, SQLite3 has no native date/time format, but it can understand ISO 8601 text fields and Unix Epoch integer fields.

Thanks for the detailed feedback... from the asm sounds like a compiler issue.

It may be worth a ticket, since it may affect not only our code, but a lot of it!

From a Github issue description for our SynPDF Open Source project: Generating a PDF via VLCCanvas and TPdfDocumentGDI causes access violation when compiled with Delphi 10.4.1 with record field alignment compiler option set to "byte" or "off". When this option is set to either of "word", "double word" or "quad word", the PDF gets created without errors. The same exact code works fine when compiled with Delphi 10.4 (patch 3), regardless of the field alignment flag.

We added {$A+} and it seemed to fix the problem.
https://blog.synopse.info/?post/2020/09/09/Record-Alignement-and-Delphi-10.4.1

Sadly, I don't have access to Delphi 10.4.1 since I don't have any commercial licence, and I am waiting for the Community Edition - which is still 10.3 IIRC. So I couldn't debug the root cause and fill a JIRA ticket to EMB.
Perhaps some people from Delphi-Praxis may have encountered this issue, and found the root cause...

Maybe it is was a real fix introduced in 10.4.1, and the previous behavior was incorrect: perhaps an explicit {$A+} is required when working with records... but at least, it breaks existing code, so spreading the info may help...

I wonder if the WebSockets encapsulating in HTTP/2 makes a huge performance difference in respect to plain HTTP upgrade.

Speaking about WebSockets frames communications, not HTTP requests.

10 hours ago, David Heffernan said:

If you care about performance, measure it. 

This is the main idea.

No premature optimization. This is not because a single line ("case  ... of") is slightly faster than your work will be faster.

AnsiString with the system code page is a wrong idea - it is not able to store all Unicode content.
UTF-8 is a good idea if you use it from one end to the other in your project.
For instance, if your database layer uses "string" then using AnsiString won't help. On the contrary, conversion and memory allocation has a cost, so it may be actually slower.

Only if you have UTF-8 from end to end, e.g. in our Open Source framework, we use UTF-8 everwhere, e.g. from DB to JSON, so no UTF-16 conversion is done. It is perfect for server side. But if you write a VCL/FMX RAD app, using plain string makes more sense.