Profiler for Delphi

[Bill Meyer 337]

I have had success with ProDelphi, which has been well maintained. My only complaint is that as it will instrument a maximum of 64,000 routines, on large projects, you must enter a list of folders or files to exclude. That said, I have been very pleased with it over the years. There are 32 and 64 bit versions for Delphi and for Lazarus, and the price is low. http://www.prodelphi.de/

 

[Larry Hengen 39]

On 4/27/2020 at 8:05 AM, Bill Meyer said:

... the price is low. http://www.prodelphi.de/

 

I agree the price is quite reasonable.  Not crazy about the UI, but it's usable.  I have been trying to use the ProDelphi 64 bit profiler against a large project using many run-time packages that has lots of code in the DPR. I am not getting the body code profiled.  Any tips?  I am trying to profile performance from startup to the appearance of the main form including any calls into the run-time packages.

[HeZa 3]

On 4/26/2020 at 1:39 AM, Attila Kovacs said:

As I could not find anything measuring the InitUnits, and it has its reasons, I come up with this poor mans profiler: ...

Oh, I'm only seeing it now. Thanks for sharing it.

[Anders Melander 1795]

On 4/23/2020 at 5:18 PM, David Heffernan said:

What we really really need is a way to generate PDB files. If only Emba would add that functionality.

I'm once again in need of a good 64-bit profiler so I've been looking into what would be needed in order to generate PDB files from whatever Delphi can produce.

 

First of all it seems the only good debug info source on the Delphi side is the MAP file and the format of that seems a bit unstable. However since it's a text file it should be easy to adapt to any changes in it.

 

Then there's the PDB file format. The only good and reliable documentation of that appears to be what the LLVM project has produced. Microsoft at one time published some of their source code for dealing with PDB files but since then the repository appears to have gone into limbo and they have not kept it up to date like they said they would.

Anyway, in short, a PDB file is a MSF container file consisting of a number of streams containing the different debug info in CodeView format. The MSF format is pretty simple and since I only need to produce PDBs I don't need to bother about decoding MSF files. That leaves the PDB CodeView streams. Thanks to the LLVM documentation their format are known. However, after reading the LLVM source, it has become clear that it would be a huge task to implement all the different stream formats and all the different data types. I guess one could get by with a subset but it doesn't really seems worth the effort to create something that only works sometimes.

The biggest problem however is that I would be implementing something based on a port of a third party's interpretation of Microsoft's old (and by now out of date) source. PDB is an internal VS format and Microsoft has stated that it can, and mostly will, change with each new version of VS. For this reason they provide the Debug Interface Access API for those that need to read PDB files. Unfortunately there's no (known) corresponding API to write PDB files.

PDB support is important for the LLVM project (they too would like VTune to work with their output ) so they appear to be trying to keep up with the PDB changes and since they obviously have more resources for that task than I do I've decided to aim for a solution that piggybacks on their work instead of rolling my own.

 

The solution I'm going to try is to parse the MAP file in Delphi, write out the relevant information in YAML format and then use the LLVM llvm-pdbtil tool to convert the YAML to PDB. Sounds easy and it probably would be except for the fact that the required YAML format is undocumented... The llvm-pdbutil yaml2pdb documentation does a handwave with "The YAML syntax is not described here. Instead, use llvm-pdbutil pdb2yaml and examine the output for an example starting point.". Lazy bastards

 

Anyway, this was just a dump of the information I've been able to gather on the topic of generating PDB from Delphi in case I burn out before completing anything usable.

[Stefan Glienke 2009]

22 minutes ago, Anders Melander said:

The solution I'm going to try is to parse the MAP file in Delphi, write out the relevant information in YAML format and then use the LLVM llvm-pdbtil tool to convert the YAML to PDB. Sounds easy and it probably would be except for the fact that the required YAML format is undocumented... The llvm-pdbutil yaml2pdb documentation does a handwave with "The YAML syntax is not described here. Instead, use llvm-pdbutil pdb2yaml and examine the output for an example starting point.". Lazy bastards

That's also something that went through my head when I was reading the LLVM documentation. 

[Anders Melander 1795]

Well that turned out to be a big waste of time

 

I can now produce a YAML file from the MAP file using my own tool and a PDB file from the YAML file with the LLVM tool.

 

Unfortunately VTune couldn't care less. It turns out that in order for VTune to load the PDB file the file needs to be referenced in the PE debug section of the exe file. This is something Microsoft's linker does when you specify the /DEBUG option but Delphi's linker of course doesn't.

 

I can't even patch the debug section Delphi's linker does produce since it's only a single byte. I assume that it's possible to patch the file by replacing the whole debug section and update the PE header but right now I don't know the required format of it.

The format of the debug section is documented here: http://www.debuginfo.com/articles/debuginfomatch.html and http://www.godevtool.com/Other/pdb.htm

 

I don't have time to pursue this any further right now, so I hope someone else is willing to have a go at it. I'll post my source when I have a spare moment.

Edited March 13, 2021 by Anders Melander

[Attila Kovacs 629]

@Anders Melander Maybe this could inject it, just to test how the pdb performs http://www.pe-explorer.com/peexplorer-tour-section-editor.htm

[Anders Melander 1795]

I have several tools that can edit PE files (among them PE Explorer), but I think it would be faster to just write a tool that does the job the right way. It seems there also a GUID that needs to be matched between the PDB and EXE.

[Attila Kovacs 629]

Yeah the GUID match can be done on the PDB, at least I've seen tools for do that

39 minutes ago, Anders Melander said:

but I think it would be faster to just write a tool that does the job the right way

yeah, when the pdb works yes, but untested? 

[Anders Melander 1795]

1 minute ago, Attila Kovacs said:

Yeah the GUID match can be done on the PDB, at least I've seen tools for do that

I know that VS refuses to load a PDB if the GUID doesn't match and I assume VTune does as well. They probably both use the DbgHelp API so maybe it's a feature of that.

I think the flow here would be to generate a PDB with a GUID and then update the EXE with the GUID and the name of the PDB (sans the path).

 

5 minutes ago, Attila Kovacs said:

yeah, when the pdb works yes, but untested? 

I can't see patching the EXE with a hex editor would buy me anything and it's not that easy when there isn't already room in the debug directory. It could be used to verify that the PDB is valid, but since the final tool chain will need to patch the EXE anyway one might as well get that out of the way first and only then worry about the validity of the PDB.

[Vincent Parrett 754]

@Anders Melander have you had a look at the roslyn compiler repo, there may be some useful pdb info in there too https://github.com/dotnet/roslyn

[John Kouraklis 94]

On 4/23/2020 at 6:26 PM, Tom F said:

I more recently purchased Nexus Quality Suite and was very happy with it.  Great tool, great support, great modern product: https://www.nexusdb.com/support/index.php?q=node/27156.  

As someone previously said, "You get what you pay for."  NQS was well worth what we paid for it.

I tried NQS for a while. The UI is very confusing to me. It is 80s but I don't mind that much; what I don't get is how you load the projects and profile them. Even when you launch it from inside Delphi, it doesn't work smoothly. I found it very cumbersome.

 

The only other reliable and affordable solution I found is ProDelphi. I haven't tried the x64 version but the x32 works nicely. The only downside is that it alters the source code during what they call instrumentation. But it cleans the sources after that perfectly.

 

And they are very responsive with support. 

[Anders Melander 1795]

6 hours ago, Vincent Parrett said:

have you had a look at the roslyn compiler repo, there may be some useful pdb info in there too

The content of a .NET pdb file is not the same as one produced by the native linker so that will not help. Besides I have already solved the pdb part of the problem and am able to generate a pdb file from the Delphi map file.

 

The remaining problem is that VTune will not load the pdb file unless the file is referenced inside the exe file. I have verified with process monitor that VTune doesn't even look for the pdb file when loading an application compiled with Delphi.

It's probably a fairly simple task to patch the exe with the required information but it involves altering the PE structures and that is something I know very little about - and I don't really have time to learn that stuff.

 

The following is the PE layout of an application that does contain the required debug information (the tool used is PE View) :

• An entry in the PE header points to the debug directory (an array of IMAGE_DATA_DIRECTORY).
  The number of entries in the debug directory is Size div SizeOf( IMAGE_DATA_DIRECTORY). In this case the size is $1C so the count is one.

 

• The entry in the debug directory contains a value that specifies the type of the entry (in this case CodeView) and a pointer (relative address) to the debug data (a IMAGE_DEBUG_TYPE_CODEVIEW structure):

 

• Finally the IMAGE_DEBUG_TYPE_CODEVIEW structure contains the name of the PDB file:

The IMAGE_DEBUG_TYPE_CODEVIEW structure looks like this:

struct CV_INFO_PDB70 {
  DWORD  CvSignature;
  GUID Signature;
  DWORD Age;
  BYTE PdbFileName[];
};

 

And this is the same information of a Delphi compiled application:

• Notice that the size of the debug directory is 1 (an invalid value incidentally), so the count is zero. Game over.

 

[Stefan Glienke 2009]

Does it help to put the pdb into one of the directories mentioned in 6. of this doc?

https://software.intel.com/content/www/us/en/develop/documentation/vtune-help/top/set-up-project/search-directories/search-order.html

[pyscripter 689]

Just in case you have not seen it SamplingProfiler 64 – test version - DelphiTools,

I mentioned earlier the updated instrumenting profiler ase379/gpprofile2017: Gpprof with unicode support and new features. (github.com).  

 

Between the two you can get the job done, but I admit being able to use VTune (also free) would be great.

[Stefan Glienke 2009]

I like SamplingProfiler but compared to the capabilities of VTune or μProf it's just a toy.

[Anders Melander 1795]

28 minutes ago, Stefan Glienke said:

Does it help to put the pdb into one of the directories mentioned in 6. of this doc?

No.

By default VTune will look for the file in the same folder as the executable and I have also tried to add this folder manually to the search list but, as I said, VTune doesn't look for the file anywhere.

 

4 minutes ago, pyscripter said:

Between the two you can get the job done, but I admit being able to use VTune (also free) would be great.

I'm sure those two have their uses but for me they simply does not provide the level of detail that I need. I need call graphs and asm level analysis.

[Anders Melander 1795]

1 hour ago, Anders Melander said:

And this is the same information of a Delphi compiled application:

• Notice that the size of the debug directory is 1 (an invalid value incidentally), so the count is zero. Game over.

Not quite Game Over it seems.

It appears that the Size being 1 is just because whomever wrote the linker has misunderstood the meaning of the field. 1 in this case means that there one entry, so I could replace the 1 with $1C and the entry would be valid. If I then assume that data in the debug directory is now valid then the one entry points to the .debug segment. I assume this segment contains the TDS debug data or something like (it starts with the TDS signature "FB09"). It's only present (for both 32- and 64-bit) if I link with debug info enabled.

 

Now since this debug info isn't used anyway when profiling with VTune, I can just hijack the area occupied by it and store my IMAGE_DEBUG_TYPE_CODEVIEW structure there. This means that I won't have to deal with adding new sections and updating all the various offsets in the PE header. Should be doable with what I know so far.

 

I have to some gardening to take care of now but I'll give it another go this evening. Stay (V)tuned...

[Guest]

I don't know about VTune per se, but for what i do understand and know, pdb file can be dependent from the EXE and yet debuggers/profilers should be able to recognize them, my try in the past was a fail, yet then i managed to use MAP file parser from EurekaLog and managed to (in very simplified way) add one stream to the MSF file and generate simple/skeleton pdb file, just the one and back then Process Explorer started to show the thread stack address translated into functions right, tested successfully also on a debugger, later it did fail miserable when the table was big, not small, i really can't remember the details now.

 

Anyway, @Anders Melander , from what i recall i didn't touch the exe it was the pdb who should matched the TimeStamp in the PE header of the exe, and that it is.

Also googling now, i found this http://www.debuginfo.com/articles/debuginfomatch.html which i think does show that IMAGE_OPTIONAL_HEADER is irrelevant to use external pdb files.
"DBG files use a similar approach, where the role of the unique identifier is assigned to the executable’s timestamp (which is stored in the executable’s file header, IMAGE_FILE_HEADER.TimeDateStamp). The same timestamp is stored in the header of the DBG file (IMAGE_SEPARATE_DEBUG_HEADER.TimeDateStamp). When a debugger checks whether a DBG file matches the executable, it reads the timestamp from the DBG file and compares it with the timestamp stored in the executable. If timestamps are not equal, the DBG file is considered unmatched. In addition, Visual Studio debuggers also check for presence of IMAGE_FILE_DEBUG_STRIPPED flag in Characteristics field of the executable’s file header (IMAGE_FILE_HEADER.Characteristics), and refuse to load the DBG file if the flag is not set (actually, they check the flag first and do not look for DBG file at all if the flag is not set). WinDbg debugger does not check this flag in the default configuration, and uses only timestamp to verify that the DBG file is matched.|"

And good luck.

[Anders Melander 1795]

6 hours ago, Kas Ob. said:

Anyway, @Anders Melander , from what i recall i didn't touch the exe it was the pdb who should matched the TimeStamp in the PE header of the exe, and that it is.

As I wrote, VTune doesn't even look for the pdb file so the problem must be in the exe.

 

Anyhow I have now managed to patch the exe so VTune at least logs that it is looking for a pdb (it didn't before) but can't find one. Process Monitor still shows that it isn't looking for the file though so I'm not there yet.

 

My pdb file only contains module/unit and line number information as I couldn't find a way to pass the method names on to the yaml2pdb tool at that time. I think I have found out how to do that now but I haven't tried it yet.

 

6 hours ago, Kas Ob. said:

Also googling now, i found this http://www.debuginfo.com/articles/debuginfomatch.html which i think does show that IMAGE_OPTIONAL_HEADER is irrelevant to use external pdb files.

Yes, that is one of the resources I'm using - and as far as I can see it states exactly the opposite:

Quote

When debug information for an executable is stored in PDB file, the executable’s debug directory contains an entry of type IMAGE_DEBUG_TYPE_CODEVIEW. This entry points to a small data block, which tells the debugger where to look for the PDB file.

 

So far I'm doing this:

1. Set NtHeaders32.OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DEBUG].Size = SizeOf(TImageDebugDirectory)
   Old value was 1 which is invalid.
2. Set DebugDirectory._Type := IMAGE_DEBUG_TYPE_CODEVIEW
   Old value was IMAGE_DEBUG_TYPE_UNKNOWN (zero). A bit lazy on Borland's behalf since there is a IMAGE_DEBUG_TYPE_BORLAND defined.
3. Replace the old TDS debug data with a CV_INFO_PDB70 structure containing the RSDS signature, a null GUID, 1 for the Age and the pdb filenamer as a zero terminated UTF8 string.
4. Clear the checksum to zero.

I have verified the patched exe with various PE editors and viewers and everything seems to be in place but still no cigar.

[Anders Melander 1795]

55 minutes ago, Anders Melander said:

Replace the old TDS debug data with a CV_INFO_PDB70 structure containing the RSDS signature, a null GUID, 1 for the Age and the pdb filenamer as a zero terminated UTF8 string.

It turns out the null GUID was what kept VTune from looking for the pdb file.

VTune now loads my pdb file but doesn't resolve the addresses to source lines. I will probably have to populate the pdb with method information for that to work. Makes sense.

 

I guess I'm not getting any sleep tonight

[Vincent Parrett 754]

33 minutes ago, Anders Melander said:

I guess I'm not getting any sleep tonight

Don't stop now.. we're all here waiting for the exciting news (and screenshots) 😉. I've long been envious of devs who were able to use vtune (and other tools which didn't support delphi ) - this would be massive if you pull it off!

[Fr0sT.Brutal 900]

Classic. The issue Emba couldn't resolve in decades is done in couple of days by a clever enthusiast

[Anders Melander 1795]

On 3/15/2021 at 9:00 AM, Fr0sT.Brutal said:

Classic. The issue Emba couldn't resolve in decades is done in couple of days by a clever enthusiast

Don't count your chickens before they hatch... I think you jinxed me there.

 

Here's what works so far:

• Parse a MAP file and produce an YAML file.
• Convert the YAML to PDB.
• Update the EXE with a reference to the PDB.
• Not crash VTune while it's loading the PDB

Here's what doesn't work:

• Getting VTune to use any of the information in the PDB

I know that VTune reads the PDB because I can see in Process Monitor that it looks for the source files. Unfortunately the file names it looks for are the module (i.e. unit) names and not the file names and it doesn't search the source folders I have defined:

I'm using VTune 2019 btw since that's the last version to support Windows 7.

 

Anyway if you can stomach watching the sausages getting made then the current source is available here: https://bitbucket.org/anders_melander/map2pdb/

There are two projects: map2yaml and bindpdb. In addition to that the llvm-pdbutil tool is needed. The Tools folder contains a batch file that calls all three in order. Delphi 10.3 or later is required (inline vars).

[Attila Kovacs 629]

does pdb2yaml gives the same yaml back?