Jump to content


Popular Content

Showing content with the highest reputation on 08/27/20 in all areas

  1. Lars Fosdal

    a pair of MM test

    A man in a hurry is always late.
  2. Arnaud Bouchez

    a pair of MM test

    TBB is fast in benchmarks, but from our experiment not usable on production on a server. TBB consumes A LOT of memory, much more than FM4/FM5 and alternatives. Numbers for a real multi-threaded Linux server are a show stopper for using TBB. On production on a huge Multi Xeon server, RAM consumption after a few hours stabilisation is gblic=2.6GB vs TBB=170GB - 60 times more memory ! With almost no actual performance boost. This mORMot service handles TB of incoming data, sent by block every second, with thousands of simultaneous HTTPS connections. See https://github.com/synopse/mORMot/blob/master/SynFPCCMemAligned.pas#L55 So never trust any benchmark. Try with your real workload. What we found out with https://github.com/synopse/mORMot/blob/master/SynFPCx64MM.pas may be interesting for the discussion. Using AVX for medium blocks moves/realloc doesn't change in practice in respect to an inlined SSE2 move (tiny/small/medium blocks), or a non-temporal move (using movntdq opcode instead of plain mov - for large blocks). For large blocks, using mremap/VirtualAlloc in-place reallocation is a better approach: relying on the OS and performing no move is faster than AVX/AVX2/AVX512. SynFPCx64MM is currently only for FPC. Used on production with heavily loaded servers. It is based on FastMM4 design, fully optimized in x86_64 asm, but with a lockless round-robin algorithm for tiny blocks (<=256 bytes), and an optional lockless list for FreeMem - which are the bottleneck for most actual servers. It has several spinning alternatives in case of contention. And it is really Open Source - not like FastMM5. We may publish a Delphi-compatible version in the next weeks.
  3. Fr0sT.Brutal

    a pair of MM test

    Dude probably you should start a blog instead
  4. Hello, just a small note that I released a generic circular buffer library under open source license (Apache 2.0) here: https://github.com/MHumm/CircularBuffer Feel free to use it or to contribute to it. Cheers TubroMagic
  5. Arnaud Bouchez

    a pair of MM test

    On Windows, we use http.sys kernel mode which scales better than anything on this platform. It is faster than IOCP since it runs in the kernel. On Linux, we use our own thread-pool of socket server, with a nginx frontend as reverse proxy on the unix socket loopback, handling HTTPS and HTTP/2. This is very safe and scalable. And don't trust micro benchmarks. Even worse, don't write your own benchmark. They won't be as good as measuring of a real application. As I wrote, Intel TBB is a no-go for real server work due to huge memory consumption. If you have to run some specific API calls to release the memory, this is a big design flow - may be considered as a bug (we don't want to have the application stale as it would have with a GC) - and we would never do it. To be more precise, we use long-living threads from thread pools. So in practice, the threads are never released, and the memory allocation and the memory release are done in diverse threads: one thread pool handles the socket communication, then other thread pool will consume the data and release the memory. This is a scenario typical from most event-driven servers, running on multi-core CPUs, with a proven ring-oriented architecture. Perhaps Intel TBB is not very good at releasing memory with such pattern - whereas our SynFPCx64MM is very efficient in this case. And we almost never realloc - just alloc/free using the stack as working buffer if necessary.
  6. rvk

    Could not load OpenSSL library.

    If your application is 32bit you also should use the 32 bit dll's. Not the x64 bit ones. Putting them in your exe directory and not using IdOpenSSLSetLibPath should also work.
  7. Angus Robertson

    Twsocket Tcp Client miss some packets

    If you look at any ICS components or samples that receive data, you see that you need to loop within the event to receive data with that function, you may get six bytes first time, 29K next time, loop until nothing more is returned. If you receive a string, it may be a partial line, so you need to buffer it until the a full line is available. ICS also has line mode that makes sure you receive full lines, but you may lose data if the last line does include CRLF. Also, you can not read UTF8 data reliably like this, because you may get a partial character with the first read. You need to wait until you have a line, then convert from ANSI to Unicode. If you used the OverbyteIcsIpStmLogTst.dpr sample I suggested, none of this would be an issue, it's all done for you. Angus
  8. Darian Miller

    how to run git commands from Delphi app

    You could research how the Delphi IDE handles GIT commands for its integration. https://sourceforge.net/p/radstudioverins/code/HEAD/tree/branches/git-hg/gitide/
  9. Dalija Prasnikar

    Boolean short-circuit with function calls

    The only mention for nullables was in context of custom managed records that are prerequisite for implementing nullables. From that perspective we kind of have nullable support right now in 10.4. But that is far from having full compiler support for nullable types. Nullable types include way more than just defining common RTL type (we don't even have that in 10.4) and one of the things that such support also implies (assumes) is ternary operator. Additionally ternary operator has more broader usage scope than nullables themselves and thus is more interesting, worthwhile feature.
  10. Ternary is the correct term for computer language operators with three parameters. https://en.m.wikipedia.org/wiki/Ternary_operation
  11. Lars Fosdal

    Get started writing Linux apps

    I wrote a small cookbook on getting Ubuntu up and running and setting up Rio and Sydney for compiling Linux applications.
  12. Sherlock

    how to run git commands from Delphi app

    Best practice has always been to just build the command line and execute that. No hassle with DLL changes, no issues with 32Bit vs. 64Bit and last but by far not least: easy testing by just copying the generated command line into a terminal and checking the output.
  13. It makes no sense, especially if a compiler directive instructs otherwise. My guess is that there is some badly written structure behind the scenes and it would be too much effort to "fix" it now. This is typically when we start to name our bugs, celebrate their birthday, call them a feature and see where they evolve to 🙂
  14. Kas Ob.

    RansomWare blues

    ICMP, this is a debatable subject but you can read about it but here an example and it is nice https://blog.securityevaluators.com/icmp-the-good-the-bad-and-the-ugly-130413e56030 Notice in that article it all the advantage from blocking ICMP is useless when the same attacks initiated using different protocol, so what is the point of blocking ICMP when the attacker can achieve the same with TCP,UDP... ? I don't block ICMP, on contrary ICMP attacks ( flood , redirect ..) had the smallest impact in comparison to any other, and on other hand , i maintain a mechanism to detect servers sees each others and their problem, so TCP,UDP and ICMP is used between servers to check online status and report, TCP has Nagle algorithm which i do not disable for this particular usage, and UDP is used to compare with TCP ping time, but both are level 7 so if things went south, then something happening on OS or my software, then ICMP is been used to detect if something level 3 is went wrong. lower than that is not recommended as it might flag it for the host company security software and block it, (i learned my own lesson with that) For more information, as you look interested, i will give you few keywords to search and research. "Kali Linux" this one is very useful and i recommend to be familiar with, there is many resources about what it does take an example this https://www.youtube.com/c/DavidBombal/search?query=kali , that channel is nice and the guy does explain things slow and in details ( also not in much details as he have paid course) but it is fun to know using Kali and fun with friends in the same time. "Osi layers X attacks " It is imperative to understand the communications layers https://en.wikipedia.org/wiki/OSI_model most attacks happen on layer 7, 5 ,2 and the lower the worse. for a list of the protocols by layer https://en.wikipedia.org/wiki/List_of_network_protocols_(OSI_model) Why i use Ping to detect attacks and their origin, because most what call them hackers, are in fact just using scripts and tools built by real hackers, and many of those real hackers leave hints, like fingerprints and patterns deliberately in that script, those scripts and tools are not easy to modify or understand by non-expert, they in most time need to see if the target is still online, the easiest way to decide is ping it. "nmap scan host" Even smaller/lighter tools like https://nmap.org/ has many usages, can scan a completely blocked host by its firewall and report the OS type and may be even the version of the system without one open port, (try it for your self), most likely will detect the system with no open ports at all. https://nmap.org/book/man-os-detection.html The real deal is the following two, but those are depends mostly on what has been already discovered from security holes on almost every security hole on public domain, but still it is nice to be familiar with, but this is not easy to setup and run, used them in the past for long time, currently i don't use them, i do things my way. https://www.metasploit.com/ https://www.rapid7.com/products/nexpose/ So read and get familiar with the above stuff, stop when you thing that is enough knowledge to have, and if you feel that something is still unclear then read more, As for ICMP, i don't recommend to block it, but it is up to you.
  15. I can’t see past // must call all functions, defeat short circuit evaluation aVal := A(); bVal := B(); cVal := C(); Result := aVal or bVal or cVal; I don’t care for temporarily disabling short circuit evaluation. I’d rather have the predictability of one rule for expression evaluation. Mix and match adds an impedance to understanding for the reader.