Jump to content

Arnaud Bouchez

Members
 View Profile  See their activity

  • Content Count

    316

  • Joined

  • Last visited

  • Days Won

    22

Posts posted by Arnaud Bouchez

  2. Posted · Edited by Arnaud Bouchez

    On 11/24/2020 at 12:10 PM, Wagner Landgraf said:

    This has nothing to do with executing arbitrary SQL statements. It's regular REST endpoints.

    This is not TMS DB Remote for sure.

    But a request like GET /tms/xdata/Customer?$filter=Country/Name eq 'USA'&$orderby=Name&$top=10 seems very close to a SQL SELECT.

     

    Security seems pretty basic: is there something more than the CRUD permissions of https://download.tmssoftware.com/business/xdata/doc/web/entity_sets_permissions.html ?
    I guess the https://download.tmssoftware.com/business/xdata/doc/web/onentityget_event.html callback seems a bit manual to handle.

    Even if the client is authenticated, how to you go deeper into the authorization? For instance, if you have a DB with all customers for all salers, how to prevent one saler to get the customers of other salers from the same table?

    So not any arbitrary SQL statement could be executed, for sure. But the client side is still deciding what it queries, unless the server is bloated with authorization code.

     

    What I would like to emphase is that it may be a 3-tier physical architecture, but it is still a 2-tier logical Architecture. The logic is still in the client.
    A much cleaner n-Tier architecture would be to create a real business layer (logical 3-tier) - or even better an application layer (logicial 4-tier as in DDD), and expose only safe and efficient REST endpoints to the clients.
    Then the server will query the data, and expose it as DTOs to the clients, depending on each bounded context (business layer) or each use-case (app layer).
    The client side is still much too tied to the underlying database. In a clean SOA architecture, you don't start from the database, but from the client use-cases.

  3. Posted · Edited by Arnaud Bouchez

    Some hints about performance on REAL bottlenecks:

     

    It covers, among others, the tip of a sub-function if you have some temporary managed variables (like string).

     

    The associated code, proving the slide assumptions, is available at https://synopse.info/files/slides/EKON22_2_High_Performance_Pascal_Code_On_Servers.zip
    Worth I look to understand how it works in practice.

     

    But remember:
    "Premature Optimization if the Root of All Evil !" (DK)

  4. Posted

    It is clearly a wrong good idea.

    I don't see the benefit in respect to executing the SQL and return some JSON - which is a bad idea per se.

     

    I just answered on SO why direct exposition of the database to a REST/HTTP/JSON client is not a good idea.

    https://stackoverflow.com/a/64936643/458259

     

    It is still a 2-Tier architecture, from the logical point of view.

    Of course, it is a 3-Tier architecture from the physical point of view, but what matters is the logical point of view, and adding a physical layer is not a benefit here.

     

    • Like 1

  6. Posted · Edited by Arnaud Bouchez

    6 minutes ago, FPiette said:

    @Arnaud Bouchez This report is 10 years old. Things have changed, drivers are OK now, DirectX and Direct2D have changed.

    Do you speak about Direct 2D 1.1 or Direct 2D 1.0? 😉

     

    At least with GDI (GDI+) or Graphics32 we have consistent performance.

  9. Posted

    I had a similar problem decades ago with a French payment processor called "Yaskifo". Small startup which offered best prices at that time. But with wrong management of their internal costs, and which were sued by French banks at that time...
    Yaskifo owned me thousands of Euros, which I never saw back... Once their company was bankrupted, all customers could just weep.
    I don't know anything about FastSpring, but in doubt, my advice would be to switch to a bigger and safer alternative - at least joined to some well known bank or company. Even if their fee is higher.

    • Like 1
    • Thanks 1

  10. Posted

    You would need two memory managers, one per CPU socket....
    Which is not possible with the Delphi RTL yet.

    You may try to use a per-thread memory manager, not FastMM4, which uses a per-thread arena for small blocks. There are some around for Delphi - just google for it.

  12. Posted · Edited by Arnaud Bouchez

    Not possible, as François wrote, and also not useful.

    What would be the purpose of adding new properties at runtime? How to you access them?

     

    • If what you expect is to make public something private, then you can trick the class by inheriting it locally, publishing the properties, then hard-casting to your type.
    • If what you expect is to add some fields to an existing class, it is not possible because the class instance size if hardcoded during compilation.
    • If what you expect is to have some additional information linked to an existing class, maintain a dictionary/map with the instances, and create a co-object associated with the main object lifetime, in which you put the needed data.
    • If what you expect is to have some additional information linked to an existing class (as a variation to the previous item), inherit from this class or even better nest this class into a main owner class/TComponenet, which will have your expected behavior.

  15. Posted · Edited by Arnaud Bouchez

    On 11/6/2020 at 3:59 PM, Mike Torrettinni said:

    I read this explanation why Everything tool is so fast in indexing and searching filenames: "Searches are compiled into byte code and executed."

    They just rewrite a regexp-like engine, which are usually using byte codes.
    Note that when a regexp uses JIT, it is faster, but not in all cases.
    For pascal source code of a very fast regexp engine, check https://github.com/BeRo1985/flre - you will see it is complex to make such things fast!

     

    But to be fair, my guess is that Everything as a tool is fast not because its pattern search is fast, but because all the file names are already cached in memory. So it doesn't have to call the Windows API to get all the file names.

    The real bottleneck is not the search itself, but the OS API calls.

    Byte-code search in the list may result in the search being a bit faster, but for only a few percent.

    So this performance statement is more like a marketing statement - not false, but not the real performance trick.

     

    As  another reference code, perhaps easier to read than FLRE, check our Open Source TMatch object in https://github.com/synopse/mORMot2/blob/master/src/core/mormot.core.search.pas

    It is very fast e.g. to find e.g. '*.pas' or 'toto*.pas' or '*inside*'. When I mean fast, it is fast (using algorithms like Holub and Durian (2005) SBNDM2 algorithm).

    And if you want to see some code about an expression search engine, using an AST and some kind of byte codes, check the TExprParserMatch class in the same unit.

     

    • Like 1
    • Thanks 1

  17. Posted · Edited by Arnaud Bouchez

    Take  look at Zeos/ZDBC.

    https://sourceforge.net/projects/zeoslib/

     

    They are Open Source, and with very good performance.

    If you bypass its TDataSet descendants, and use directly the ZDBC layer, you will get even better performance than alternatives, e.g. when you retrieve only one row of data (e.g. to fill a single class properties).

    And it has very good FPC support, too.

     

    But I guess you may be able to use your current version of FireDac with the latest Delphi, with minor changes to the source code...

  20. Posted · Edited by Arnaud Bouchez

    1 hour ago, Kas Ob. said:

    2) You are dividing and taking only the lower part, this can be done only when you are using modulo Mersenne number (m), only then you can truncated the values at the length of m+1, EDIT: don't confuse Mersenne number with Mersenne prime, like i did.

    What counts is that in x86_64 asm, MUL and DIV are 128-bit.

    https://www.felixcloutier.com/x86/mul states:

    REX.W + F7 /4    MUL r/m64    M    Valid    N.E.    Unsigned multiply (RDX:RAX ← RAX ∗ r/m64).

    and https://www.felixcloutier.com/x86/div reports:

    REX.W + F7 /6    DIV r/m64    M    Valid    N.E.    Unsigned divide RDX:RAX by r/m64, with result stored in RAX ← Quotient, RDX ← Remainder

     

    So my guess is that this code is correct, and the fastest possible on the target.
    Only if m is known in advance (as a constant), you may be able to make the division faster by using a reciprocal multiplication or a bits shift.

     

    From https://www.agner.org/optimize/instruction_tables.pdf on a modern CPU, 64-bit MUL takes around 3 cycles, and 64-bit DIV 10 cycles.
    Difficult to make it faster with any other algorithm.

     

    As a small optimization, you can omit the stack frame:

     

    function MulModASM(a, b, m: UInt64): UInt64;
    {$ifdef FPC}nostackframe; assembler; asm {$else} asm .noframe {$endif FPC}
      MOV rax, a
      MOV rcx, m
      MUL b
      DIV rcx
      MOV rax, RDX
    end;

     

    https://ideone.com/KUsgMi

     

    About register conventions, for windows/posix: rcx/rdi=a rdx/rsi=b r8/rdx=m so it should be fine.
    Just to be tested with a lot of values and high 64-bit limits.

    • Like 2

  21. Posted · Edited by Arnaud Bouchez

    1 hour ago, Mahdi Safsafi said:

    If you permit me, I’d like to give some suggestions:

    - Change the implementation by checking for CF and doing 10 time attempt when CF=0. I believe this will cost nothing compared to the additional security you get. BTW, that’s what Intel recommends:

    - Add another (optional) way to initialize the ES : e.g : CSPRNG, OS random data.

    You are right. The code I posted in this thread is indeed useless. You can't trust RDRAND to return some random values. It was a bad idea.

     

    It is never used as unique source of entropy in mORMot. Just as part of a lot of entropy gathering.
    Here is the documentation of mORMot about the initialization of our cryprographic AES-based PRNG:

     

        /// retrieve some entropy bytes from the Operating System
        // - entropy comes from CryptGenRandom API on Windows, and /dev/urandom or
        // /dev/random on Linux/POSIX
        // - this system-supplied entropy is then XORed with the output of a SHA-3
        // cryptographic SHAKE-256 generator in XOF mode, of several entropy sources
        // (timestamp, thread and system information, SynCommons.Random32 function)
        // unless SystemOnly is TRUE
        // - depending on the system, entropy may not be true randomness: if you need
        // some truly random values, use TAESPRNG.Main.FillRandom() or TAESPRNG.Fill()
        // methods, NOT this class function (which will be much slower, BTW)
        class function GetEntropy(Len: integer; SystemOnly: boolean=false): RawByteString; virtual;
     

    https://github.com/synopse/mORMot/blob/ecc375adc96e5b78d63dd58a88418874a0f622d8/SynCrypto.pas#L1114

     

    And about RDRAND, when mORMot checks the CPUID, it also runs RDRAND and if it fails to return random values, it unset its internal flag, and it will never be used, and not used as entropy.

    It is even worse on AMD, which can have CF=1 but always return 0 or -1 !!!

     

    So in practice, mORMot seems to follow your wise suggestions.
    My answer in this thread, and my RDRAND use was confusing, for sure.
    🙂

    • Like 2

  22. Posted · Edited by Arnaud Bouchez

    On 10/23/2020 at 8:03 PM, Kas Ob. said:

    don't use the full block in getting a random number

    This is a very good advice.
    Of course, since AES is a permutation algorithm, only using a part of the output is as secure as XORin it.

     

    But since we use AES-CTR, I don't see how it may be unsecure in our case:
    - we can't get the Random output feed from attackers (if they can, then we have more trouble than our PRNG)

    - AES-CTR is crypto secure by itself so even if we get the output feed, we won't be able to guess the key, even in the context of a PRNG.

     

    On 10/23/2020 at 8:03 PM, Kas Ob. said:

    use the other half to generate the next state

    Here I am confused.
    If you don't strictly use AES-CTR, but reinject the other half into the generator, you are changing the block chaining mode for something else.

    AES-CTR is proven, I don't know if some tweaked algorithm may be safe.
    By changing the CTR increment using some bits of the previous AES result? If it is odd, add 1, otherwise add 2?

     

    I will read the pdf, anyway.
    And change our source code don't avoid the XOR.

    Thanks again for the feedback.

  23. Posted · Edited by Arnaud Bouchez

    On 10/23/2020 at 4:50 PM, Mahdi Safsafi said:

    Not so ! 

    Short answer : 0 is not granted to be a random value on all architectures.  

    Long answer : https://software.intel.com/content/www/us/en/develop/blogs/rdrand-do-i-need-to-check-the-carry-flag-or-can-i-just-check-for-zero.html

    I know this.
    The comment is correct in the context of our code base, of course, not absolutely.
    Returning 0 and returning a not random value (i.e. checking the carry flag) won't hurt in our mORMot code base, since RDRAND is never used directly as random source, just as entropy source.

     

    Of course, in the context of the OP question, RDRAND is not a good idea, since it will work only one a set of machines.
    I clearly stated that in my answer.

  24. Posted

    I don't see how enabling basic authentication would make the web site more secure.

    The password is sent as plain in the headers, just base-64 encoded, so there is no benefit.
    If just adding a password would make something more secure... it would have been used everywhere.

     

    The best security advice, which is not on your blog post, is to maintain your WP installation up-to-date, with all the security fixes.

    • Like 1
×