Leaderboard

It took me a bit longer than expected to get here but I believe I've finally reached the goal. The following shows VTune profiling a Delphi application, with symbol, line number and source code resolution: Download Get the source here: https://bitbucket.org/anders_melander/map2pdb/ And a precompiled exe here: https://bitbucket.org/anders_melander/map2pdb/downloads/ The source has only been tested with Delphi 10.3 - uses inline vars so it will not compile with older versions. Usage map2pdb - Copyright (c) 2021 Anders Melander Version 2.0 Parses the map file produced by Delphi and writes a PDB file. Usage: map2pdb [options] <map-filename> Options: -v Verbose output -pdb[:<output-filename>] Writes a PDB (default) -yaml[:<output-filename>] Writes an YAML file that can be used with llvm-pdbutil -bind[:<exe-filename>] Patches a Delphi compiled exe file to include a reference to the pdb file -test Works on test data. Ignores the input file Example: Configure your project linker options to output a Detailed map file. Compile the project. Execute map2pdb <map-filename> -bind Profile the application with VTune (or whatever) Known issues The -bind switch must occur after the filename contrary to the usage instructions. PDB files larger than 16Mb are not valid. This is currently by design. 64-bit PE files are not yet supported by the -bind option. As should be evident I decided not to go the DWARF route after all. After using a few days to read the DWARF specification and examine the FPC source I decided that it would be easier to leverage the PDB knowledge I had already acquired. Not that this has been easy. Even though I've been able to use the LLVM PDB implementation and Microsoft's PDB source as a reference LLVM's implementation is incomplete and buggy and the LLVM source is "modern C++" which means that it's close to unreadable in places. Microsoft's source, while written in clean C and guaranteed to be correct, doesn't compile and is poorly commented. Luckily it was nothing a few all-nighters with a disassembler and a hex editor couldn't solve. Enjoy!

Not really, but we did have multi-byte and multi-codepoint character sequences, which MSDN claims CharNextA() and CharNextW() do handle. But not well enough, in this case. As that article describes, you need .NET 5, which was just released 4 1/2 months ago, to handle Grapheme clusters correctly in things like enumeration, etc.

No - when the already allocated block is large enough it doesn't do much. And when it has to actually allocate it just does a lock cmpxchg on a bool field of the block types - when its already locked by another thread it simply tries the next bigger one. Speaking of FastMM4 allocation was never the problem with many threads because of this mechanism - it could only lead to a bit memory overhead because some small blocks might be little overallocated because it had to go past the currently locked sub allocators. The issue was in deallocating because that needs of course to the allocator it the memory belongs to. This is what Pierre addressed in V5. FastMM having *a* global lock is a myth.

ICS V8.66 has been released at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio and 10.4 Sydney, and C++ Builder 2006 to XE3, 10.2 Tokyo, 10.3 Rio and 10.4 Sydney. ICS supports VCL and FMX, Win32, Win64 and MacOS 32-bit targets. The distribution zip includes the latest OpenSSL 1.1.1k win32, with other versions of OpenSSL being available from the download page. Changes in ICS V8.66 include: 1 - Added a new TIcsInetAlive component to check for IPv4 and/or IPv6 internet connectivity, using Ping and/or HTTP, defaulting to www.msftconnecttest.com run by Microsoft for Windows 10 alive checking. The online and offline check intervals may be set, and event fires when online state changes. Sample OverbyteIcsHttpRestTst has a demo for new component. 2 - OpenSSL 1.0.2 and 1.1.0 ceased security fix support over 12 months ago so ICS now only supports 1.1.1, with 3.0 support due in the next few months. This removes a lot of legacy code and functions, and several old ICS functions needed for backward compatibility. Two components only used for 1.0.2 have been removed, TSslStaticLock and TSslDynamicLock which may give an error loading forms, until they are removed. SslContext SslOptions is now ignored, use SslOptions2 instead which has more modern options. Can not remove SslOptions because it's saved on too many DFMs. Most modern applications should not be using SslConext, but IcsHosts for servers for high level client components like TSslHttpRest. 3 - Undertook a major clean-up of OpenSSL functions that may require end user application changes if low level OpenSSL functions have been used, hopefully very rarely. All OpenSSL functions have been renamed to their original names removing ICS f_ prefix for commonality with other Delphi applications. Renamed literals BIO_CTRL_PENDING_, CRYPTO_NUM_LOCKS_, CRYPTO_LOCK_, OPENSSL_VERSION_ with underscore to avoid conflicts with functions of the same case sensitive names. Removed unit OverbyteIcsLibeayEx not used for a while. Most OpenSSL type pointers to Dummy array are now simple Pointer type. Changed most OpenSSL var parameters to pointers for commonality. New TSslWSocket method SslOK is TLS/SSL negotiated OK. LoadVerifyLocations no longer lets OpenSSL use default CA folder locations which is potentially a security risk. 4 - Bring server SSL/TLS security levels up to latest Mozilla recommendations. In 2021 Mozilla now recommends TLSv1.3 as modern ciphers and TLSv1.2/1.3 as Intermediate supporting all browsers from last five years, so IcsHosts now use sslCiphersMozillaSrvTLS12 as Intermediate level, also Mozilla recommends no cipher server preference so changed that. Updated SslSrvSecurity levels: sslSrvSecInter, sslSrvSecInterFS and sslSrvSecHigh now all the same TLSv1.2 or 1.3, sslSrvSecTls12Less now TLSv1.2 only, sslSrvSecSsl3 not supported, only sslSrvSecBack supports TLSv1 and 1.1, sslSrvSecTls13Only unchanged TLSv1.3 only. 5 - Added support for YuOpenSSL which provides OpenSSL in a pre-built DCU statically linked into applications, rather than using external OpenSSL DLLs. This make application distribution more reliable since it can not fail by users deleting the DLLs or copying incompatible versions into the directory. YuOpenSSL is a commercial product from https://www.yunqa.de/ and is supplied as separate compiled DCUs for Delphi 5 to 10.4. DEFINE YuOpenSSL in Include\OverbyteIcsDefs.inc determines whether the DCU is linked or the external DLLs. Note only one version of OpenSSL can be linked with YuOpenSSL, whereas different DLLs can be supported. Apart from setting the define and adding a path to YuOpenSSL.dcu, no other application code changes are needed unless you check or report the DLL directory, when GSSLStaticLinked can be checked to see if OpenSSL is statically linked. Also worth noting that YuOpenSSL provides access to the all OpenSSL APIs, while ICS only accesses those DLL APIs needed for functionality we support. 6 - Added a better way of configuring Socks proxy and HTTP Tunnel proxy settings in TWSocket, similarly to that added for THttpCli in V8.62. The new ProxyURL property sets SOCKS or HTTP Tunnel proxy settings using a single URL, ie proto://[user:password@]host:port where proto is socks5 or http. Also added new methods SetSocks and SetHTTPTunnel to set proxy properties in a single call. Socks proxy and HTTP tunnelling again support host names, broken in V8.56. The sample OverbyteIcsSocksTst has been updated to demonstrate use of the new ProxyURL property and also now supports HTTP Tunnel proxy. 7 - TIcsRestEmail has new OAAuthType property and event for OAuth2 browser URL perhaps via email for servers. TIcsRestEmail now sets OAuth2 errors, and clears old tokens. Increased default timeout for TIcsRestEmail AOuth2 login to 120 seconds since several web page warnings may be seen while attempting to login through the web page. 8 - TIcsHttpMulti has a new ParseLevels property that follow links on a parsed web page to lower level pages, which are also parsed, looking for files to download with a specific file extension, ie .avi. It ignores parent links without a file extension, and external links. This allows the component to download files indexed using HTTP from CCTV camera memory cards. 9 - In TSslHttpRest ensure SSL is initialised for non-REST HTTP requests like GET, POST and PUT. 10 - In TSslWSocketServer, added better error handling when IcsHosts SslCert is blank, so SslContext is still created for automatically created self signed certificates. 11 - In TSslWSocketServer, TSslHttpAppSrv and TSslHttpSrv, added new onBeforeCtxInit event which if set is called once before each IcsHosts SslContext is initialised to allow the context parameters to be adjusted for special ciphers or protocols. 12 - In THttpCli, AddrResolvedStr is now still available after the connection closes to report the IP address used for the connection. When using an HTTPS proxy with the CONNECT command don't send host and port which some servers are unable to handle, probably broken with V8.64. 13 - In TIcsMailQueue, added a new method CheckOAuthLogins that should be called before the queue starts when using OAuth2 email logins, which allows interactively checking with a browser of any OAuth2 login credentials, avoids asking for login first time email sent maybe days later. 14 - In TSslX509Certs, When starting the local web server for file challenges, check the server IP address still exists or change to 0.0.0.0, may not work if multiple IPs exist. 15 - Updated to OpenSSL 1.1.1k with two high security fixes, one introduced in the last version relating to a certificate chain validation that ICS does not use, the other relating to servers using TLSv1.2 and renegotiation, and that feature is removed in V8.66.

Can you try with a tiny console application. I've used the following for test: program test; {$APPTYPE CONSOLE} {$R *.res} uses System.Classes, System.SysUtils; procedure WasteSomeTime; begin var Strings := TStringList.Create; try for var i := 1 to 100000 do Strings.Add(i.ToString); Strings.Sort; finally Strings.Free; end; end; begin try for var i := 1 to 10 do begin Writeln(i.ToString); WasteSomeTime; end; except on E: Exception do Writeln(E.ClassName, ': ', E.Message); end; end. If that doesn't work either then please zip the map file + exe produced by the above code and post it here or at the issue tracker on bitbucket.

Fixed. https://bitbucket.org/anders_melander/map2pdb/commits/8c94db07a2da1b503ce6f00c39792ee0b5479387

The problem is the MSF container format. It's a blocked format with 4096 bytes in each block and up to 4096 blocks in each "interval" (4096*4096 = 16Mb). Each interval starts with a Free Page Map (FPM, a bit like a FAT). Since I wanted to concentrate on getting the PDB layout right my MSF abstraction doesn't implement intervals so once I pass 4096 blocks the file becomes invalid. Here's LLVMs explanation of the MSF format: https://llvm.org/docs/PDB/MsfFile.html A proper implementation of intervals will require a bit of work. At present I can assume that a physical stream offset value equals the logical offset value. Once I add intervals I will have to take into the FPMs into account since the blocks are no longer contiguous. Also when I write I will have to consider that a write can start on once side of the FPMs and continue on the other side of them. It isn't rocket science but it does complicate the IO layer considerably. No. I get those too. I'm not sure what they are for but they are in segment 4 (TLS) and 5 (PDATA= exception data) so you'll not miss them. The warning just means that the address of a symbol couldn't be matched to any of the modules (units). It should be. How do you come to the conclusion that it wasn't patched? Try the -v switch. Okay. There's probably a bug in the command line interface then. I discovered the problem with the position of the -bind switch just before release so there's probably more.

DirectWrite has some methods that could be usefull. https://docs.microsoft.com/en-us/windows/win32/api/dwrite/nn-dwrite-idwritetextanalyzer It is able to display for example 🧑🏿‍🦽 correctly.

Have a look at my commits #3476 and #3477. There was some rather odd code in TUnitIdentifierList.LoadFromFile which always wrote the cache file even if nothing had changed at all. It now never writes the file but instead returns False if it was in the old format. The caller will then parse the source code and write the cache file. I haven't timed it but I think this should speed up the process significantly due to a lot less file system activity.

And attached here is a text element enumerator that handles 🤷🏽‍♀️ correctly using ICU. Note though that the Edit Box and the List Box do not display 🤷🏽‍♀️correctly (shown as two characters). The content of the EditBox is åá̂̃̄🤷🏽‍♀️ (copied and pasted here). EnumTextElements.pas

Those other libraries are likely either reading the UDP data for you and then giving you each datagram's actual data in the event, or they have peeked the socket to determine the available bytes and then telling you that size, ICS doesn't do either of those, it is just notifying you that datagrams have arrived, but is not doing anything to gleam information about them to present to you. For comparison, Indy's TIdUDPClient (which is not event-driven) requires you to provide it with a pre-allocated buffer for it to receive bytes into. But TIdUDPServer (which is event driven) will read each datagram into an internal buffer and then fire an event giving you the actual bytes that were read into that buffer. So, there is room for different mentalities, depending on your needs.

There is no such thing as a "normal graphic character" in Unicode. What you are thinking of as a "character" is officially referred to as a "grapheme", which consists of 1 or more Unicode codepoints linked together to make up 1 human-readable glyph. Individual Unicode codepoints are encoded as 1 or 2 codeunits in UTF-16, which is what each 2-byte Char represent. When a codepoint is encoded into 2 UTF-16 codeunits, that is also known as a "surrogate pair". That emoji is 1 Unicode codepoint: U+1F64F (Folded Hands) Which is encoded as 2 codeunits in UTF-16: D83D DE4F That will only allow you to determine the value of the 1st Unicode codepoint in the grapheme. But then you need to look at and decode subsequent codepoints to determine if they "combine" in sequence with that 1st codepoint. That emoji takes up 8 bytes, consisting of 4 UTF-16 codeunits: D83D DE4F DB3C DFFB Which decode as 2 Unicode codepoints: U+1F64F (Folded Hands) U+1F3FB (Emoji Modifier Type-1-2) And it gets even more complicated especially for Emoji, because 1) modern Emoji support skin tones and genders, which are handled using 1+ modifier codepoints, and 2) multiple unrelated Emoji can be grouped together with codepoint U+200D to create even new Emoji. For example: U+1F469 (Woman) U+1F3FD (Emoji Modifier Type-4) U+200D (Zero Width Joiner) U+1F4BB (Personal Computer) Which is treated as 1 single Emoji of a light-skined woman sitting behind a PC. Or, how about: U+1F468 (Man) U+200D (ZWJ) U+1F469 (Woman) U+200D (ZWJ) U+1F467 (Girl) U+200D (ZWJ) U+1F466 (Boy) Which is treated as 1 single Emoji of a Family with a dad, mom, and 2 children. See https://eng.getwisdom.io/emoji-modifiers-and-sequence-combinations/ for more details. Delphi itself only concerns itself with the encoding/decoding of UTF-16 itself (especially when converting that data to other encodings, like ANSI, UTF-8, etc). Delphi does not care what the UTF-16 data represents. Graphemes are handled only by application code that needs to be do text processing, glyph rendering, etc. Things that are outside of Delphi's scope as a general programming language. Most of the time, you should just let the OS deal with them. Unless you are writing your own engines that need to be Grapheme-aware. By using a Unicode library that understands the rules of Graphemes, Emojis, etc. Delphi has no such library built-in, but there are several 3rd party Unicode libraries that do understand those things.

That does not work at all - CharNext does not work well. And that article is from 2007 - did we even have emojis back then? 😄 Try yourself with the testcase here: https://docs.microsoft.com/en-us/dotnet/core/compatibility/globalization/5.0/uax29-compliant-grapheme-enumeration

The preferred way to specify authentication credentials in THttpClient is to use its CredentialStorage property, eg: httpClient.CredentialStorage.AddCredential(TCredential.Create(TAuthTargetType.Server, '', '', 'username', 'password')); Or its AuthEvent event: procedure TMyForm.HTTPClientAuthEvent(const Sender: TObject; AnAuthTarget: TAuthTargetType; const ARealm, AURL: string; var AUserName, APassword: string; var AbortAuth: Boolean; var Persistence: TAuthPersistenceType); begin if AnAuthTarget = TAuthTargetType.Server then begin AUserName := 'username'; APassword := 'password'; end; end; httpClient.AuthEvent := HTTPClientAuthEvent; See Using an HTTP Client: Handling Authentication and Certificates in Embarcadero's DocWiki.

That is true for TCP, where there is no 1:1 relationship between sends and reads, it is just a stream of bytes. But that is not true for UDP, where there is a 1:1 relationship, since sends/reads deal only in whole datagrams. A datagram can't span multiple read events, a read operation must read the whole datagram in one go, or else the unread portions will be lost.

Another option would be to peek the socket first to query the size of the next available datagram without reading it, then allocate the buffer to that size, then read the datagram into the buffer. But a dynamic array of raw bytes is trivial to shrink in size, so over-allocating a little is not a bad thing (unless you are running in an embedded system).

Apparently simple, but in practice nothing trivial, and with nothing on the internet about this, I decided to do a brief tutorial to master the Android system bars. https://github.com/viniciusfbb/fmx_tutorials/tree/master/delphi_android_system_bars/