Jump to content

Arnaud Bouchez

Members
 View Profile  See their activity

  • Content Count

    325

  • Joined

  • Last visited

  • Days Won

    25

Posts posted by Arnaud Bouchez

  1. Posted · Edited by Arnaud Bouchez

    Take  look at Zeos/ZDBC.

    https://sourceforge.net/projects/zeoslib/

     

    They are Open Source, and with very good performance.

    If you bypass its TDataSet descendants, and use directly the ZDBC layer, you will get even better performance than alternatives, e.g. when you retrieve only one row of data (e.g. to fill a single class properties).

    And it has very good FPC support, too.

     

    But I guess you may be able to use your current version of FireDac with the latest Delphi, with minor changes to the source code...

  4. Posted · Edited by Arnaud Bouchez

    1 hour ago, Kas Ob. said:

    2) You are dividing and taking only the lower part, this can be done only when you are using modulo Mersenne number (m), only then you can truncated the values at the length of m+1, EDIT: don't confuse Mersenne number with Mersenne prime, like i did.

    What counts is that in x86_64 asm, MUL and DIV are 128-bit.

    https://www.felixcloutier.com/x86/mul states:

    REX.W + F7 /4    MUL r/m64    M    Valid    N.E.    Unsigned multiply (RDX:RAX ← RAX ∗ r/m64).

    and https://www.felixcloutier.com/x86/div reports:

    REX.W + F7 /6    DIV r/m64    M    Valid    N.E.    Unsigned divide RDX:RAX by r/m64, with result stored in RAX ← Quotient, RDX ← Remainder

     

    So my guess is that this code is correct, and the fastest possible on the target.
    Only if m is known in advance (as a constant), you may be able to make the division faster by using a reciprocal multiplication or a bits shift.

     

    From https://www.agner.org/optimize/instruction_tables.pdf on a modern CPU, 64-bit MUL takes around 3 cycles, and 64-bit DIV 10 cycles.
    Difficult to make it faster with any other algorithm.

     

    As a small optimization, you can omit the stack frame:

     

    function MulModASM(a, b, m: UInt64): UInt64;
    {$ifdef FPC}nostackframe; assembler; asm {$else} asm .noframe {$endif FPC}
      MOV rax, a
      MOV rcx, m
      MUL b
      DIV rcx
      MOV rax, RDX
    end;

     

    https://ideone.com/KUsgMi

     

    About register conventions, for windows/posix: rcx/rdi=a rdx/rsi=b r8/rdx=m so it should be fine.
    Just to be tested with a lot of values and high 64-bit limits.

    • Like 2

  5. Posted · Edited by Arnaud Bouchez

    1 hour ago, Mahdi Safsafi said:

    If you permit me, I’d like to give some suggestions:

    - Change the implementation by checking for CF and doing 10 time attempt when CF=0. I believe this will cost nothing compared to the additional security you get. BTW, that’s what Intel recommends:

    - Add another (optional) way to initialize the ES : e.g : CSPRNG, OS random data.

    You are right. The code I posted in this thread is indeed useless. You can't trust RDRAND to return some random values. It was a bad idea.

     

    It is never used as unique source of entropy in mORMot. Just as part of a lot of entropy gathering.
    Here is the documentation of mORMot about the initialization of our cryprographic AES-based PRNG:

     

        /// retrieve some entropy bytes from the Operating System
        // - entropy comes from CryptGenRandom API on Windows, and /dev/urandom or
        // /dev/random on Linux/POSIX
        // - this system-supplied entropy is then XORed with the output of a SHA-3
        // cryptographic SHAKE-256 generator in XOF mode, of several entropy sources
        // (timestamp, thread and system information, SynCommons.Random32 function)
        // unless SystemOnly is TRUE
        // - depending on the system, entropy may not be true randomness: if you need
        // some truly random values, use TAESPRNG.Main.FillRandom() or TAESPRNG.Fill()
        // methods, NOT this class function (which will be much slower, BTW)
        class function GetEntropy(Len: integer; SystemOnly: boolean=false): RawByteString; virtual;
     

    https://github.com/synopse/mORMot/blob/ecc375adc96e5b78d63dd58a88418874a0f622d8/SynCrypto.pas#L1114

     

    And about RDRAND, when mORMot checks the CPUID, it also runs RDRAND and if it fails to return random values, it unset its internal flag, and it will never be used, and not used as entropy.

    It is even worse on AMD, which can have CF=1 but always return 0 or -1 !!!

     

    So in practice, mORMot seems to follow your wise suggestions.
    My answer in this thread, and my RDRAND use was confusing, for sure.
    🙂

    • Like 2

  6. Posted · Edited by Arnaud Bouchez

    On 10/23/2020 at 8:03 PM, Kas Ob. said:

    don't use the full block in getting a random number

    This is a very good advice.
    Of course, since AES is a permutation algorithm, only using a part of the output is as secure as XORin it.

     

    But since we use AES-CTR, I don't see how it may be unsecure in our case:
    - we can't get the Random output feed from attackers (if they can, then we have more trouble than our PRNG)

    - AES-CTR is crypto secure by itself so even if we get the output feed, we won't be able to guess the key, even in the context of a PRNG.

     

    On 10/23/2020 at 8:03 PM, Kas Ob. said:

    use the other half to generate the next state

    Here I am confused.
    If you don't strictly use AES-CTR, but reinject the other half into the generator, you are changing the block chaining mode for something else.

    AES-CTR is proven, I don't know if some tweaked algorithm may be safe.
    By changing the CTR increment using some bits of the previous AES result? If it is odd, add 1, otherwise add 2?

     

    I will read the pdf, anyway.
    And change our source code don't avoid the XOR.

    Thanks again for the feedback.

  7. Posted · Edited by Arnaud Bouchez

    On 10/23/2020 at 4:50 PM, Mahdi Safsafi said:

    Not so ! 

    Short answer : 0 is not granted to be a random value on all architectures.  

    Long answer : https://software.intel.com/content/www/us/en/develop/blogs/rdrand-do-i-need-to-check-the-carry-flag-or-can-i-just-check-for-zero.html

    I know this.
    The comment is correct in the context of our code base, of course, not absolutely.
    Returning 0 and returning a not random value (i.e. checking the carry flag) won't hurt in our mORMot code base, since RDRAND is never used directly as random source, just as entropy source.

     

    Of course, in the context of the OP question, RDRAND is not a good idea, since it will work only one a set of machines.
    I clearly stated that in my answer.

  8. Posted

    I don't see how enabling basic authentication would make the web site more secure.

    The password is sent as plain in the headers, just base-64 encoded, so there is no benefit.
    If just adding a password would make something more secure... it would have been used everywhere.

     

    The best security advice, which is not on your blog post, is to maintain your WP installation up-to-date, with all the security fixes.

    • Like 1

  10. Posted · Edited by Arnaud Bouchez

    Your xoroshiro128starstar implementation is not thread-safe, by the way.

     

    Why not just use (Int64(Random(maxInt)) shl 32) or Random(maxInt) which is fine and cross-compiler?

     

    Or on modern CPU: 

    function RdRand32: cardinal; {$ifdef CPU64}
{$ifdef FPC}nostackframe; assembler; asm{$else} asm .noframe {$endif FPC} {$else}
{$ifdef FPC}nostackframe; assembler;{$endif} asm {$endif}
  // rdrand eax: same opcodes for x86 and x64
  db $0f, $c7, $f0
  // returns in eax, ignore carry flag (eax=0 won't hurt)
end;

function RdRand64: QWord;
begin
  result := (QWord(RdRand32) shl 32) or RdRand32;
end;

    Note that RDRAND opcode is not available on old processor, not very fast, and sometimes returns 0 of -1 on AMD... but it is a good seed for entropy.

     

    See also the Random32 function in https://github.com/synopse/mORMot/blob/master/SynCommons.pas which is based on Lecuyer, and use good entropy seed.

    and the TAESPRNG class in https://github.com/synopse/mORMot/blob/master/SynCrypto.pas which is a true cryptographic random generator, using AES-NI very fast asm.
    Both are thread-safe and use the NIST SP 800-90A compliant RDRAND Intel x86/x64 opcode as entropy source, if available.

  11. Posted · Edited by Arnaud Bouchez

    I would not put the behavior change regarding of $R+ setting.
    Your library should better be consistent and never need a re-compilation if you need to change e.g. from DEBUG to RELEASE mode.

     

    My idea would be to raise an exception in case of out of bounds error - just like TStringList/TList.
    It is what most users expect, and will be confident and pleased with.


    In mORMot, the trick is that I also give direct access (as a field, not as a getter - even inlined) to the raw pointer of array storage. So the user can by-pass the range check if needed, e.g. by using a local pointer variable, and proper pointer arithmetic. I document the property as low-level and raw access to the values, to be used at your own risk.
    Then advanced users, and other parts of your own library which is safely written and tested, could be able to bypass the range check, by accessing directly the pointer values.

    • Like 4

  19. Posted · Edited by Arnaud Bouchez

    I would rather use RawByteString for several reasons:

    1. Proper reference counting;

    2. (Slightly) faster allocation.
    3. Better debugging experience in your case.

     

    Some details:
    1. TBytes = array of byte has a less strict reference counting. If you modify a TBytes item, all its copied instances will be modified. Whereas with RawByteString, the string will be made unique before modification (it is called Copy on Write - aka COW).

    2. SetLength(TBytes) will allocate the memory and fill it with zeros, whereas SetLength(RawByteString) will just allocate the memory. If you use a memory buffer which will immediately be filled with some data, no need to fill it with zeros first.

    3. Even if your RawByteString has some binary, the ASCII characters will be easier to read e.g. #2#0#7'Some Text'#0'Some other text'#49.

  20. Posted · Edited by Arnaud Bouchez

    I don't understand why I would be involved in this discussion..

    My only knowledge is that Delphi 2007 dcc32.exe could work with the .bpl not in c:\windows\system32. I only use .dcu anyway for my command-line D2007 compilation script, modifying the dcc32.cfg file and putting the dcu in a small sub folder: 

    c:\progs\Delphi2007>dir /s /w
 Le volume dans le lecteur C n'a pas de nom.
 Le numéro de série du volume est C006-BF3C

 Répertoire de c:\progs\Delphi2007

[.]   [..]  [bin] [lib]
               0 fichier(s)                0 octets

 Répertoire de c:\progs\Delphi2007\bin

[.]           [..]          dcc32.cfg     DCC32.EXE     rlink32.dll
               3 fichier(s)        1 087 592 octets

 Répertoire de c:\progs\Delphi2007\lib

[.]             [..]            ActiveX.dcu     Classes.dcu     ComConst.dcu
CommCtrl.dcu    ComObj.dcu      Consts.dcu      Contnrs.dcu     Graphics.dcu
ImageHlp.dcu    IniFiles.dcu    Math.dcu        Messages.dcu    Registry.dcu
RTLConsts.dcu   SyncObjs.dcu    SysConst.dcu    SysInit.dcu     System.dcu
SysUtils.dcu    Types.dcu       TypInfo.dcu     Variants.dcu    VarUtils.dcu
Windows.dcu     WinInet.dcu     WinSpool.dcu
              26 fichier(s)        2 346 725 octets

     Total des fichiers listés :
              29 fichier(s)        3 434 317 octets
               8 Rép(s)   6 376 325 120 octets libres

c:\progs\Delphi2007>

     

  21. Posted · Edited by Arnaud Bouchez

    My remark was a general one. It was not about this explicit code.

    Writing `TPopupMenu(_Sender)` is a hard-cast which should never appear in most code, unless you have a very vague/legacy event like a TNotifyEvent. The VCL requires it.
     

    For UI code, I would write: 

     if _Sender is TPopupMenu then begin
    // sometimes Delphi 10.4.1 passes a TMenuItem here, no idea how this can happen, but it caused a runtime error
    AppendMenuItem(_Sender as TPopupMenu);
  end;

    The code is somewhat redundant, because "is" is actually doing the class hierarchy check that "as" is doing: calling internally InheritsFrom() twice
    But it is only slightly slower - unnoticeable on a VCL app, since passing a Windows message will be slower than this "as" execution.
    IMHO it is safer and easier to read and to maintain.
    Having used "as" at first hand did indeed give a hint about the initial problem.

     

    Hardcasting is like premature optimization on client side. 

  22. Posted · Edited by Arnaud Bouchez

    Unconditional cast should never be used on UI/Client side. Always uses "is" or "as".
    It may be used only when performance really matters, e.g. on core server code, with a large test coverage, and only on proven bottlenecks - which are very rare.

    The first is to write "as" to raise an exception. Then - maybe - use direct class casting. Only if really needed.

     

    IMHO such direct class casting - or cascaded "if .. is .. then ... else if .. is .. then ..." may be a smell for potential breaking some of the SOLID principles.
    In VCL events, we use Sender: TObject most of the time - it could be seen as a design flaw / legacy debt. But it is a reality.
    We should be able to avoid direct class casting in most of our modern code.

  24. Posted · Edited by Arnaud Bouchez

    21 hours ago, borni69 said:

    Thanks looks like   

    
TCharacter.IsValid(s[i]))

    is not suported in 10.4

    I don't get what TCharacter.IsValid() was supposed to mean. Sounds like a big confusion from the Embarcadero RTL people.
    In UTF-16 you may need two WideChar to encode an Unicode glyph - it is called a surrogate pair.
    So if you want to check the UTF-16 encoding validity, you have to work at the string level, or at least test two WideChars at once when needed.

    I guess this may be the reason why it disappeared. Confused and confusing.

×