Jump to content


Popular Content

Showing content with the highest reputation since 04/28/20 in Posts

  1. Vandrovnik

    Function with just underscore _() seems to be valid

    With this info in mind, for bad customers you can start programming like that: const _ = 1; _____ = 2; x = _____ + (_-_-_) + 0 + _+0-0+_ ;
  2. Arnaud Bouchez

    Experience/opinions on FastMM5

    You are right: FastMM5 challenged me... and since no one responded to my offer about helping it run on FPC/Linux, and also since I wanted something Open Source but not so restrictive, I created https://github.com/synopse/mORMot2/blob/master/src/core/mormot.core.fpcx64mm.pas which is GPL/LGPL and MPL. So you can use it with closed software. It uses the same core algorithms than FastMM4. I like it so much, and missed it so much in FPC... 🙂 I was involved in ScaleMM2, and a per-thread arena for small blocks didn't convince me: it tends to consume too much RAM when you have a lot of threads in your process. Note that a threadvar is what the FPC standard MM uses. I wanted to take the best of FastMM4 (which is very proven, stable and efficient), but drive it a little further in terms of multi-threading and code quality. FastMM4 asm is 32-bit oriented, its x86_64 version was sometimes not very optimized for this target - just see its abuse of globals, not knowledge of micro-op fusion or CPU cache lines and locks, and sparse use of registers. Also focusing on a single compiler and a single CPU, with not all the features of FastMM4 in pascal mode, helped fpcx64mm appear in two days only. Last but not least, I spent a lot of time this last year in x86_64 assembly, so I know which patterns are expected to be faster. The huge regression test suite of mORMot helps having a proven benchmark - much more aggressive and realistic than microbenchmarks (like string concatenation in threads, or even the FastCode benchmark) on which most other MM relies for measurement. When the regression tests are more than twice faster than with the FPC standard MM on Linux - as @ttomas reported - then we are talking. It runs a lot of different scenarios, with more than 43,000,000 individual tests, and several kind of HTTP/TCP servers on the loopback, running in-memory or SQLite databases, processing JSON everywhere, with multiple client threads stressing it. When I run the test on my Linux machine, I have only a few (less than a dozen) system Linux nanosleeps (better than Windows sleep) , and less than 2 ms waiting during a 1 minute of heavy tests - and only for Freemem. I really don't like the microbenchmarks used for testing MM. Like the one published in this forum. For instance IntelTBB is very fast for such benchmarks, but it doesn't release its memory as it should, and it is unusable in practice. I guess that some user code, not written with performance in mind, and e.g. abusing of str := str+'something' patterns would also be more than twice faster. And if your code has to reallocate huge buffers (>256KB) in a loop, using mremap on Linux may make a huge performance boost since no data would be copied at all - Linux mremap() is much better than what Windows or BSD offer! Yes, huge memory blocks are resized by the Linux Kernel by reaffecting its TLB redirection tables, without copying any memory. No need to use AVX512 if you don't copy anything! And plain SSE2 (with non-volatile mov for big buffers) is good enough to saturate the HW memory bandwidth - and faster than ERMS in practice. IMHO there was no need to change the data structures like FastMM5 did - I just tuned/fixed most of its predecessor FastMM4 asm, reserved some additional slots for the smaller blocks (<=80 bytes are now triplets), implemented a safe and efficient spinning, implement some internal instrumentation to catch multi-threading bottlenecks, and then Getmem didn't suffer from contention any more! I knew than FastMM4 plus some tweaks could be faster than anything else - perhaps even FastMM5.
  3. Günther Schoch

    Experience/opinions on FastMM5

    Hello David I see your concerns and Peirre le Riche and I discussed a lot on the licensing. I tried to explain the background in https://en.delphipraxis.net/topic/2751-fastmm5-now-released-by-pierre-le-riche-small-background-story/ we see 3 groups of "users" a) the vast majority is fine with FastMM4 as the applications do not suffer under any multi-threading related memory manager problem. Means: nobody is forced to switch. b) the developer having heavy multi-threaded applications consuming a lot of rather expensive CPU. There FastMM5 really helps and the small amount of money that Pierre is asking for in form of a dual license (starting with 99$) is nothing compared with other expenses. c) and there is Embarcadero: As explained in my intro story a modern memory manager would actually be part of the scope of Delphi (in theory). Pierre solved this problem already once (with FastMM4) for free. This story will not be repeated by FastMM5 as Pierre needs obviously some financial payback to maintain the product. BTW: When my company started to sponsor the development of FastMM5, I whould never had thought that it pays back that fast. We got beginning of the year our first 2 AMD Epyc 64/128 based servers for hosting your Delphi WebServices. Scaling up our services for such platforms was really only possible with FastMM5.
  4. Newly released FastMM5: https://github.com/pleriche/FastMM5 FastMM is a fast replacement memory manager for Embarcadero Delphi applications that scales well across multiple threads and CPU cores, is not prone to memory fragmentation, and supports shared memory without the use of external .DLL files. Version 5 is a complete rewrite of FastMM. It is designed from the ground up to simultaneously keep the strengths and address the shortcomings of version 4.992: Multithreaded scaling across multiple CPU cores is massively improved, without memory usage blowout. It can be configured to scale close to linearly for any number of CPU cores. In the Fastcode memory manager benchmark tool FastMM 5 scores 15% higher than FastMM 4.992 on the single threaded benchmarks, and 30% higher on the multithreaded benchmarks. (I7-8700K CPU, EnableMMX and AssumeMultithreaded options enabled.) It is fully configurable runtime. There is no need to change conditional defines and recompile to change options. (It is however backward compatible with many of the version 4 conditional defines.) Debug mode uses the same debug support library as version 4 (FastMM_FullDebugMode.dll) by default, but custom stack trace routines are also supported. Call FastMM_EnterDebugMode to switch to debug mode ("FullDebugMode") and call FastMM_ExitDebugMode to return to performance mode. Calls may be nested, in which case debug mode will be exited after the last FastMM_ExitDebugMode call. Supports 8, 16, 32 or 64 byte alignment of all blocks. Call FastMM_EnterMinimumAddressAlignment to request a minimum block alignment, and FastMM_ExitMinimumAddressAlignment to rescind a prior request. Calls may be nested, in which case the coarsest alignment request will be in effect. All event notifications (errors, memory leak messages, etc.) may be routed to the debugger (via OutputDebugString), a log file, the screen or any combination of the three. Messages are built using templates containing mail-merge tokens. Templates may be changed runtime to facilitate different layouts and/or translation into any language. Templates fully support Unicode, and the log file may be configured to be written in UTF-8 or UTF-16 format, with or without a BOM. It may be configured runtime to favour speed, memory usage efficiency or a blend of the two via the FastMM_SetOptimizationStrategy call. Experience/opinions welcome ...
  5. Quick-And-Dirty ( complete package attached ) unit HideControls; interface procedure Register; implementation uses Vcl.Controls, DesignIntf, DesignEditors; type THideControlsEditor = class(TComponentEditor) private procedure SetDesignVisible(Value: Boolean); public procedure ExecuteVerb(Index: Integer); override; function GetVerb(Index: Integer): string; override; function GetVerbCount: Integer; override; end; procedure Register; begin RegisterComponentEditor(TControl, THideControlsEditor); end; procedure THideControlsEditor.ExecuteVerb(Index: Integer); begin case Index of 0: SetDesignVisible(False); 1: SetDesignVisible(True); end; end; function THideControlsEditor.GetVerb(Index: Integer): string; begin case Index of 0: Result := 'Hide Control(s)'; 1: Result := 'Show Control(s)'; else Result := ''; end; end; function THideControlsEditor.GetVerbCount: Integer; begin Result := 2; end; procedure THideControlsEditor.SetDesignVisible(Value: Boolean); var I: Integer; list: IDesignerSelections; begin list := CreateSelectionList; Designer.GetSelections(list); for I := 0 to list.Count - 1 do if list.Items[I] is TControl then TControl(list.Items[I]).SetDesignVisible(Value); end; end. HideControlsAtDesignTimeDemo.zip
  6. Pierre le Riche

    FastMM5 now released by Pierre le Riche (small background story)

    Rep movs is only used for large blocks, where it is the fastest mechanism on modern processors. For smaller blocks SSE2 is used, if available. As an interesting aside, apparently the Linux kernel uses rep movs for all moves. The rationale behind that is to force the CPU manufacturers to optimize rep movs even for small moves. Historically rep movs has been hit and miss from one CPU generation to the next. I would like to add more platform support, but there has to be sufficient demand to offset the development and maintenance burden. I don't want v5 to become an "ifdef hell" like, to a large extent, v4. v4 supports so many different configurations that it is littered with ifdefs. It makes it really hard to expand functionality because there are so many different combinations that need to be tested. That said, it is on my "to-do" list to add support for most or all of the platforms supported by the latest Delphi versions. I think it would be particularly useful to have the debugging features available on other platforms as well.
  7. Stefan Glienke

    RTTI info for TBytes using runtime packages

    The issue is that TBytes is an alias for TArray<Byte> - generic types being used across different modules have several implications such as that they manifest into each module with their own typeinfo being generated. Try this code for example: uses Rtti, SysUtils; var ctx: TRttiContext; begin Writeln(ctx.GetType(TypeInfo(TBytes)).QualifiedName); end. In a regular console application it will print: System.TArray<System.Byte> While you would assume that it should print: System.SysUtils.TBytes Now if you enable runtime packages for this console application and let it link with rtl you will get an ENonPublicType here. Every module that is using TBytes is not referencing some type that is declared in the RTL runtime package but emiting its own version of TArray<Byte> into its binary. Personally I think this is an issue with the compiler treating the declaration of TBytes = TArray<Byte> different from TBytes = array of Byte in terms of the typeinfo generated - I was sure there is a QP entry somewhere about it but I cannot find it right now. To further provide the proof that aforementioned assumptions of other posters are wrong use the following code with a console application linking with the rtl package: uses Rtti, TypInfo, Classes, SysUtils; var ctx: TRttiContext; begin Writeln(ctx.GetType(TBytesStream).GetField('FBytes').FieldType.Handle = TypeInfo(TBytes)); end. This will print FALSE as the typeinfo of the FBytes field which is of type System.SysUtils.TBytes in fact differs from the typeinfo generated for the usage of TBytes in the console application binary. In fact there does not even exist a type called TBytes but only System.TArray<System.Byte> as TBytes it just an alias as I said and the declaration TBytes = type TArray<Byte> is not legit and raises E2574 Instantiated type can not be used for TYPE'd type declaration
  8. Hi, when you read the title, you probably already had the standard answer in your head: "Does not work with Indy, supports only OpenSSL 1.0.2 at most and thus no TLS 1.3". I can reassure you, that is not my point. Or actually even more precisely: That is exactly my point 😉 I've spent "a little" time writing the Indy support for OpenSSL 1.1.1 and TLS 1.3, and added a push request to Indy: #299. At the same time I have fixed a few issues that have been posted to GitHub (see PR). I wrote 2 new IO handlers ( one for server and one for client), the old ones are unchanged to avoid conflicts. Everything was written and tested in Delphi Berlin 10.1.2 on Win32 and Win64. I have neither macOS nor iOS nor Linux nor Android, nor FreePascal, nor older (or newer) Delphi versions. I have tried to keep older Delphi versions in mind to ensure that it will run on them, but there have been no tests on my part. I have tested it extensively in small test applications with other servers/clients. In addition, I built it into a large Real World program with TCP server/client, SMTP/IMAP/POP clients, FTP client, HTTP client, and it also ran there without problems. Unfortunately the nice man, who has provided new binary files of OpenSSL under indy.fulgan.com has said that he does not offer versions > 1.0.2 anymore. So I used the versions of slWebPro in the beginning (they even still work on WinXP), later I used the versions of Overbyte, because they do not have any external dependencies (and are digitally signed, but no XP anymore^^). But both worked without problems. All files are located in the subfolder "Lib/Protocols/OpenSSL". There are also subfolders "static" and "dynamic" which offers quite extensive imports of the OpenSSL API, once for static linking, once for dynamic loading/unloading. For dynamic loading there are also possibilities in the "IdOpenSSLLoader.pas" to trigger the loading/unloading itself, if you need the API outside of the IO handler (e.g. for your own x509 generation). To save me the double writing of the imports, I wrote a kind of intermediate code in the folder "Intermediate", which I let generate with "GenerateCode" to the two variants. The tool "GenerateCode" is only a simple string customization and actually only designed for Berlin, so I didn't bother about downward compatibility. As a normal user of the IO handlers you don't need them, only if you make changes to the API implementation. So and now comes your. It would be nice if one or the other would test this, so that it is not only WOMM certified, but can withstand more real situations. For me it also works with the Indy, which comes with Delphi Berlin, when I create another unit, which provides some new Indy types and functions. Of course some units have to be adapted locally to use the new unit.
  9. David Heffernan

    Experience/opinions on FastMM5

    That's not really the point. The developers of any code own it. They get to choose how they licence it. They have many options. Basic decency says we respect their choices. I'm confident that every person who is critical of somebody else's choice of licence would flat out reject it if anybody told them how to licence their own software.
  10. Anders Melander

    Experience/opinions on FastMM5

    So you're complaining that Pierre has enabled us to use FastMM 5 for free and that there's conditions for this use? I think "thank you" would be more appropriate.
  11. That sums it up. Use the xml doc feature for methods and properties, and do it straight away. If it is postponed, it never happens. A colleague asked me on how to use a specific feature yesterday... So... I opened a couple of related units I wrote some time ago, and they had NO xml doc, so yeah, that never happened. The interface sections alone are 1070 and 350 lines, so... I guess I really should work more on my own documentation rather than offer advice To my defense, they are widely used in our projects, so there are plenty of usage examples
  12. dummzeuch

    Delphi Closedown Error

    BTW: Yesterday I finally found and fixed a problem in GExperts which caused an access violation on IDE shutdown (an OTAPI interface reference that wasn't released after use). But it was caught and logged by an exception handler, so nobody using a release version of GExperts should have seen it. The fix was exactly one line of code, but it took me hours to track it down. The fun with working on IDE plugins: You can reproduce the problem, you see the call stack, you can even single step to the last line of your own code before it happens, but then you only get lots of assembler code of the IDE and other plugins which tells you nothing about what happens.
  13. David Heffernan

    Experience/opinions on FastMM5

    Don't use the hammer then. Make your own. Your choice.
  14. Pierre le Riche

    Experience/opinions on FastMM5

    Hi all, I've added experimental support for NUMA in a branch (numa_support). The idea is to link both the arenas and threads (for which performance matters) to a "NUMA mask". When scanning the arenas for available blocks it will perform a bitwise "and" between the mask for the arena and the mask for the thread, and if the result is non-zero then the arena is allowed to serve blocks to that thread. In this way you can completely separate the memory pools between threads or groups of threads. I have not tested how well this works in practice (I don't have a NUMA system on hand), but I believe VirtualAlloc is smart enough to provide memory from the NUMA node closest to the CPU the thread is running on. I have made it so you can specify a mask by block size. Version 4 is susceptible to cache thrashing when adjacent small blocks share the same cache line and are written to by different CPUs. By using this mechanism that can also be avoided. Pierre
  15. Günther Schoch

    Experience/opinions on FastMM5

    Well, during the design phase of FastMM5 this feature was discussed but not (yet) implemented. The background was: a) a lot of the software is now running on large AWS nodes or similar virtual severs. There the optimization via NUMA is rather a special case b) modern processors as the AMD EPYC https://www.nextplatform.com/2019/08/15/a-deep-dive-into-amds-rome-epyc-architecture/ have internal optimization strategies But we are open to everything that makes the FastMM5 performance significantly better. regards Günther (Günther Schoch, gs-soft AG = we sponsored FastMM5)
  16. Fred Ahrens

    Fonts & ligatures

    My brain would detect this as an error while reading the code. I always expect to see in the source code the characters I did type and not the characters the IDE "thinks" might look better.
  17. Uwe Raabe

    When computers try to be clever

    IIRC, that functionality exists since Delphi 1. After 25 years someone finally managed to hit it. Congratulations.
  18. The first 64-bit iphones were released 7 years ago (iPhone 5s), and today they are the vast majority (more than 98%), it is no wonder that Apple ended support for 32-bit iphones a long time ago. Apple has not even updated iOS 32-bit for more than 4 years, and Apple has not accepted 32-bit apps in its store for some years. I personally have not compiled anything for ios 32 bits for some years. In the beginning, the 64-bit iOS kept backwards compatibility with 32-bit apps, but that ended when Apple released iOS 11. A good way to find out how many users are running only 64-bit apps is to look at the graphics of the most used versions of iOS (11, 12 and 13 have support only for 64-bit apps).
  19. Stefan Glienke

    RTTI info for TBytes using runtime packages

    Regardless the current typeinfo issue I would always check for typeKind = tkDynArray (and possibly even tkArray) and ElemType = Byte because then you can also handle any custom declared TMyBytes = array of Byte type.
  20. I'm gaining experience, with the mistakes and failures on the way. Rome wasn't built in a day. It shouldn't, but it gives me a little comfort knowing not everybody is perfect and I'm not the only one having odd 'relationship' with the documentation.
  21. Dave Nottage

    Thread programming without sleep or WaitFor events

    Then you should not be using UDP, since order of packets is not guaranteed. With UDP, every packet should be considered data independent of all others.
  22. David Heffernan

    Capturing Console Output

  23. Lars Fosdal

    XML to SVG

    So... I went full tilt A**hole Admin and removed the noise. 1. Don't assume everything is an insult 2. Don't abuse the report system
  24. Mark-

    XML to SVG

    Hello, I was VERY interested until I read "(no source code)" and "(includes source code repository access)". Not sure what the later means. Never purchase components without source code. My2c. Mark