Jump to content

Arnaud Bouchez

  • Content Count

  • Joined

  • Last visited

  • Days Won


Everything posted by Arnaud Bouchez

  1. Arnaud Bouchez

    Delphi 64bit compiler RTL speedup

    No, it was not just "reserved", there was a lot more of dirty pages with Intel TBB. We tried it on production on Linux, on high-end servers with heavy multi-thread process, and the resident size (RES) was much bigger - not only the virtual/shared memory (VIRT/SHR). Also the guys from https://unitybase.info - which have very high demanding services - evaluated and rejected the Intel TBB use. Either the glibc MM https://sourceware.org/glibc/wiki/MallocInternals or our https://github.com/synopse/mORMot2/blob/master/src/core/mormot.core.fpcx64mm.pas give good results on Linux, with low memory consumption. Anyway, I wouldn't use Windows to host demanding services. So if you have a Windows server with a lot of memory, you are free to use Intel TBB if you prefer.
  2. Arnaud Bouchez

    Delphi 64bit compiler RTL speedup

    The 3rd party dll are Intel TBB if I am correct. So you should at least mention it, with the proper licence terms, and provide a link. About memory management, from my tests the Intel TBB MM is indeed fast, but eats all memory, so it is not usable for any serious server-side software, running a long time. Some numbers, tested on FPC/Linux, but you got the idea: - FPC default heap 500000 interning 8 KB in 77.34ms i.e. 6,464,959/s, aver. 0us, 98.6 MB/s 500000 direct 7.6 MB in 100.73ms i.e. 4,963,518/s, aver. 0us, 75.7 MB/s - glibc 2.23 500000 interning 8 KB in 76.06ms i.e. 6,573,152/s, aver. 0us, 100.2 MB/s 500000 direct 7.6 MB in 36.64ms i.e. 13,645,915/s, aver. 0us, 208.2 MB/s - jemalloc 3.6 500000 interning 8 KB in 78.60ms i.e. 6,361,323/s, aver. 0us, 97 MB/s 500000 direct 7.6 MB in 58.08ms i.e. 8,608,667/s, aver. 0us, 131.3 MB/s - Intel TBB 4.4 500000 interning 8 KB in 61.96ms i.e. 8,068,810/s, aver. 0us, 123.1 MB/s 500000 direct 7.6 MB in 36.46ms i.e. 13,711,402/s, aver. 0us, 209.2 MB/s for multi-threaded process, we observed best scaling with TBB on this system BUT memory consumption raised to 60 more space (gblic=2.6GB vs TBB=170GB)! -> so for serious server work, glibc (FPC_SYNCMEM) sounds the best candidate
  3. If a method does two diverse actions, define two methods. If a method performs an action which is something on/off or enabled/disabled then you can use a boolean, if the false/true statement is clearly defined by the naming of the method. If a method performs something, but with a custom behavior, don't use boolean (or booleans) but an enumeration or even better a set. It will be much more easy to understand what it does, without looking into the parameter names, and it will be more open to new options/behaviors. function TMyObject.SaveTo(json: boolean): string; // what is the behavior with json=false? function TMyObject.SaveToJson(expanded: boolean): string; // what does SaveToJson(true/false) mean without knowing the parameter name? function TMyObject.SaveToJson(expanded, usecache: boolean): string; // what does SaveToJson(true/false, true/false) mean without knowing the parameters names? type TMyObjectSaveToJsonOptions = set of (sjoExpanded, sjoUseCache); function TMyObject.SaveToJson(options: TMyObjectSaveToJsonOptions): string; // you understand what does SaveToJson([]) or SaveToJson([sjoExpanded]) or SaveToJson[sjoExpanded, sjoUserCache]) mean
  4. Arnaud Bouchez

    The Case of Delphi Const String Parameters

    All this is a void discussion. This code is just broken and should be fixed. It has nothing to do with const or whatever. The compiler is doing what it should, but the code is plain wrong. I fully agree with @David Heffernan here. About the "address", it should be pointer(Value) not @Value. pointer(value) returns the actual pointer of the string content in heap, so will change. It is a faster alternative to @value[1] which works also with value='' -> pointer(value)=nil. @Value returns the memory adress of the local Value variable on the stack, so won't change.
  5. Binary is not text, so it is pointless for your problem. You need the integers to be written as text, not as 4 bytes binary. You could directly write to the TWriteCachedFileStream, too, without any temporary string. And append the integer values using a shortstring and old str() instead of IntToStr() which uses the heap.
  6. Signing the executable is the key here. And also make a minimal security audit. A password should be hashed, and never stored in the executable itself. It has nothing to do with Delphi. It was a poor security design of the application. About logic security, and reverse engeniering, Java or C# are much worse than Delphi. You can easily decompile Java or C# executable.... unless it has been obfuscated explicitly. I can tell you that I "hacked" so many C# dlls which we lost the source... 😉 Whereas a Delphi exe is compiled, and lack a lot of RTTI, so it is much more difficult to get something about it.
  7. Note that you don't store the content, you re-assign each new line. So you are testing something non reallistic, which is worthless trying to optimize it. What is slow is not data moving, but memory allocation. One performance problem is the temporary allocation of strings, if you call Integer.ToString. For our mORMot TTextWriter, we don't use any temporary allocation, and even have pre-computed values for the smallest integers. Note that Delphi TStringBuilder will actually be slower on Win32 than naive concatenation. It also allocate a temporary string for integer appending... 😞 https://www.delphitools.info/2013/10/30/efficient-string-building-in-delphi/3/ I would stick with naive string concatenation, and I guess it will be fast enough in practice. It would be premature optimization otherwise.
  8. Arnaud Bouchez

    git and Delphi tooling?

    Git for Desktop is just a bloated Electron app.. I would not recommend it. I don't use any gui tool for git. For simple git process, I use - on Linux, I use some simple scripts: https://github.com/synopse/mORMot2/blob/master/commit.sh and https://github.com/synopse/mORMot2/blob/master/kompare.sh on Linux - for mORMot, I made a simple VCL app which calls a source comparison tool, then call some scripts. https://github.com/synopse/mORMot/tree/master/SQLite3/Documentation/SourceCodeRep (which also update a fossil repository altogether with github - for standalone/private repositories, https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki is awesome, Windows native, with much more features than git and it has a build-in web ui, and the ability to mirror to git)
  9. Asking the very same question here and in SO is not very fair.
  10. Arnaud Bouchez

    How create a website whit Delhpi

    Our Open Source mORMot has a powerful MVC Web development model, based on Mustache templates for HTML/CSS views, and interfaces/classes to write the Controller and the Model. Check https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITLE_498 and
  11. Arnaud Bouchez

    Multiple Instances of Python Engine

    I guess this is a python runtime limitation. Due to the https://en.wikipedia.org/wiki/Global_interpreter_lock then the engines are not thread-safe. Whereas for instance, we use SpiderMonkey (for running JavaScript) with a per-thread engine, with no problem.
  12. Arnaud Bouchez

    is it possible to undeclare an identifier?

    I guess the only way is to use a "free" name like "ErrorMsg" instead of "Error"...
  13. Arnaud Bouchez

    auto-generated REST APIs?

    This is not TMS DB Remote for sure. But a request like GET /tms/xdata/Customer?$filter=Country/Name eq 'USA'&$orderby=Name&$top=10 seems very close to a SQL SELECT. Security seems pretty basic: is there something more than the CRUD permissions of https://download.tmssoftware.com/business/xdata/doc/web/entity_sets_permissions.html ? I guess the https://download.tmssoftware.com/business/xdata/doc/web/onentityget_event.html callback seems a bit manual to handle. Even if the client is authenticated, how to you go deeper into the authorization? For instance, if you have a DB with all customers for all salers, how to prevent one saler to get the customers of other salers from the same table? So not any arbitrary SQL statement could be executed, for sure. But the client side is still deciding what it queries, unless the server is bloated with authorization code. What I would like to emphase is that it may be a 3-tier physical architecture, but it is still a 2-tier logical Architecture. The logic is still in the client. A much cleaner n-Tier architecture would be to create a real business layer (logical 3-tier) - or even better an application layer (logicial 4-tier as in DDD), and expose only safe and efficient REST endpoints to the clients. Then the server will query the data, and expose it as DTOs to the clients, depending on each bounded context (business layer) or each use-case (app layer). The client side is still much too tied to the underlying database. In a clean SOA architecture, you don't start from the database, but from the client use-cases.
  14. Some hints about performance on REAL bottlenecks: It covers, among others, the tip of a sub-function if you have some temporary managed variables (like string). The associated code, proving the slide assumptions, is available at https://synopse.info/files/slides/EKON22_2_High_Performance_Pascal_Code_On_Servers.zip Worth I look to understand how it works in practice. But remember: "Premature Optimization if the Root of All Evil !" (DK)
  15. Arnaud Bouchez

    auto-generated REST APIs?

    It is clearly a wrong good idea. I don't see the benefit in respect to executing the SQL and return some JSON - which is a bad idea per se. I just answered on SO why direct exposition of the database to a REST/HTTP/JSON client is not a good idea. https://stackoverflow.com/a/64936643/458259 It is still a 2-Tier architecture, from the logical point of view. Of course, it is a 3-Tier architecture from the physical point of view, but what matters is the logical point of view, and adding a physical layer is not a benefit here.
  16. Arnaud Bouchez

    Speed of Graphics32

    Did you try with FMX? Then you can switch the rendering engine behind it.
  17. Arnaud Bouchez

    Speed of Graphics32

    Do you speak about Direct 2D 1.1 or Direct 2D 1.0? 😉 At least with GDI (GDI+) or Graphics32 we have consistent performance.
  18. Arnaud Bouchez

    Speed of Graphics32

    @FPiette Direct2D is no silver bullet either. It has been reported to be slower than GDI, due to wrongly implemented drivers, and is somewhat deprecated/unoptimized/unused. @Mike James Don't forget about GDI+ which is part of Windows, and has good performance, with nice features like Alpha channel and good antialiaising. Which kind of drawing do you need?
  19. Arnaud Bouchez

    NERO5 chess engine

    So many "goto" in the source code. 🙂
  20. I had a similar problem decades ago with a French payment processor called "Yaskifo". Small startup which offered best prices at that time. But with wrong management of their internal costs, and which were sued by French banks at that time... Yaskifo owned me thousands of Euros, which I never saw back... Once their company was bankrupted, all customers could just weep. I don't know anything about FastSpring, but in doubt, my advice would be to switch to a bigger and safer alternative - at least joined to some well known bank or company. Even if their fee is higher.
  21. Arnaud Bouchez

    Threads on dual-Xeon system

    You would need two memory managers, one per CPU socket.... Which is not possible with the Delphi RTL yet. You may try to use a per-thread memory manager, not FastMM4, which uses a per-thread arena for small blocks. There are some around for Delphi - just google for it.
  22. Arnaud Bouchez

    Feature Request for String Paste As

    Some tools - like cnpack IIRC - allow to format the text into source code string after being pasted.
  23. Not possible, as François wrote, and also not useful. What would be the purpose of adding new properties at runtime? How to you access them? If what you expect is to make public something private, then you can trick the class by inheriting it locally, publishing the properties, then hard-casting to your type. If what you expect is to add some fields to an existing class, it is not possible because the class instance size if hardcoded during compilation. If what you expect is to have some additional information linked to an existing class, maintain a dictionary/map with the instances, and create a co-object associated with the main object lifetime, in which you put the needed data. If what you expect is to have some additional information linked to an existing class (as a variation to the previous item), inherit from this class or even better nest this class into a main owner class/TComponenet, which will have your expected behavior.
  24. @David Heffernan , come out of @Stefan Glienke body! Now both Delphi reference guys have enabled their internal Technical English Compiler (tm) in paranoid mode! 😄
  25. Exactly. This is one of the reasons why enabling extended RTTI did increase the exe size by a big margin, and why I usually disable it unless I know I will actually need it.