MAP2PDB - Profiling with VTune

[Attila Kovacs 629]

Well, OllyDbg loads fine a 27Mb pdb and I picked an address from the end of the map and it's translated in the debugger.

Great work!

 

[Attila Kovacs 629]

There is however this message on bind: "Image does not contain a debug directory address - please link with debug info enabled" It's enabled.

edit: hmmmm .debug is not present in the exe

edit2: got it, place debug info in a separate tds file was checked

Edited April 5, 2021 by Attila Kovacs

[Attila Kovacs 629]

My VTunes stucks here:

 

[Anders Melander 1784]

8 minutes ago, Attila Kovacs said:

My VTunes stucks here:

Exactly my experience.

[Anders Melander 1784]

Hmmm. Maybe one just has to be patient. Very patient...

 

[Attila Kovacs 629]

I've seen in procmon that something happens but slow af and had nothing to do with the pdb. I'm still waiting for the confirmation email for the intel tech forum where google found some similar posts.

[Virgo 18]

Is map file format different in older delphi versions? With Delphi 5 and Delphi XP map files it always fails with  "Overlapping segments".

[Anders Melander 1784]

22 minutes ago, Virgo said:

Is map file format different in older delphi versions?

I don't know.

 

22 minutes ago, Virgo said:

With Delphi 5 and Delphi XP map files it always fails with  "Overlapping segments".

And does the segments overlap?

[Virgo 18]

Now, that I'm checking: is 0 length .xdata ovelapping .tls?

 

Start         Length     Name                   Class
 0001:00401000 00003B90H .text                   CODE
 0002:00405000 00000100H .itext                  ICODE
 0003:00406000 000007BCH .data                   DATA
 0004:00407000 00002B74H .bss                    BSS
 0005:00000000 00000008H .tls                    TLS
 000A:00000000 00000000H .xdata                  PDATA

Detailed map of segments

 0001:00000000 000036F4 C=CODE     S=.text    G=(none)   M=System   ACBP=A9
 0001:000036F4 00000110 C=CODE     S=.text    G=(none)   M=SysInit  ACBP=A9
 0001:00003804 0000038C C=CODE     S=.text    G=(none)   M=test     ACBP=A9
 0002:00000000 000000AC C=ICODE    S=.itext   G=(none)   M=System   ACBP=A9
 0002:000000AC 00000054 C=ICODE    S=.itext   G=(none)   M=test     ACBP=A9
 0003:00000000 00000794 C=DATA     S=.data    G=DGROUP   M=System   ACBP=A9
 0003:00000794 00000024 C=DATA     S=.data    G=DGROUP   M=SysInit  ACBP=A9
 0004:00000000 00002B68 C=BSS      S=.bss     G=DGROUP   M=System   ACBP=A9
 0004:00002B68 0000000C C=BSS      S=.bss     G=DGROUP   M=SysInit  ACBP=A9
 0005:00000000 00000008 C=TLS      S=.tls     G=(none)   M=System   ACBP=A9

 

[Anders Melander 1784]

1 hour ago, Virgo said:

Now, that I'm checking: is 0 length .xdata ovelapping .tls?

Yes, but it's a bug on my part.

I previously ignored empty segments (i.e. the PDATA segment) but that caused other problems so now I add them to the segment list. I just forgot that I still need to ignore them when checking for overlaps.

 

You can probably just delete the line from you map file. If there are any references to the segment in the map file ten you will have to remove those as well (search for 000A:).

[Virgo 18]

Warning: [   41] Failed to resolve symbol to module: [0005:00000008] SysInit.TlsLast

But otherwise ok.

 

But Delphi 5 is weird and probably hopeless...

 Start         Length     Name                   Class
 0001:00000000 00002B14H .text                   CODE
 0002:00000000 000000BCH .data                   DATA
 0002:000000BC 000004D1H .bss                    BSS

 

[Anders Melander 1784]

Fixed the problem where some symbols (in particular generics) either resolved to the calling unit or to a wrong line number in the implementing unit.

Thanks to @Stefan Glienke for pestering me about this until it got solved.

 

Source committed and binary uploaded. And now... Pizza! 🍕

[Attila Kovacs 629]

Just tested with a D2007 project and no problems at all.

[Attila Kovacs 629]

1 hour ago, Virgo said:

But Delphi 5 is weird and probably hopeless...

 Start         Length     Name                   Class
 0001:00000000 00002B14H .text                   CODE
 0002:00000000 000000BCH .data                   DATA

you could read those offsets from the PE (optionally), you are already iterating through the sections, should not be a big problem

[Wagner Landgraf 43]

Works great here as well, just tested!

Thank you @Anders Melander, for this awesome work. There is a big chance I can will ditch AQTime now!

[Vincent Parrett 750]

With @Stefan Glienke changes merged it's blazingly fast👍. I added a postbuild event to every project in my project group (131 projects) and barely notice it running now! 

 

This is what I have for the postbuild event

 

%MAP2PDB% $(OUTPUTDIR)\$(OUTPUTNAME).map -bind:$(OUTPUTDIR)\$(OUTPUTFILENAME)

 

Note that I did needed the outputfilename part for the bind option as it only looks for exe (so for bpl, dll you need it).  I created an environment variable for the path to map2pdb to avoid hard coding the path in my dproj files. Sadly if you use libsuffix anywhere you need to manually add that in to the map file arg as $(OUTPUTNAME) doesn't include it, and there isn't a macro for it. 

 

[Stefan Glienke 2006]

6 minutes ago, Vincent Parrett said:

it's blazingly fast👍.

Profiling map2pdb with VTune using a pdb built with the map file from map2pdb 🤯

 

6 minutes ago, Vincent Parrett said:

Note that I did needed the outputfilename part for the bind option as it only looks for exe (so for bpl, dll you need it). 

I guess that is fixable.

Edited April 7, 2021 by Stefan Glienke

[Wagner Landgraf 43]

On 4/6/2021 at 5:53 AM, Attila Kovacs said:

I've seen in procmon that something happens but slow af and had nothing to do with the pdb. I'm still waiting for the confirmation email for the intel tech forum where google found some similar posts.

Are you referring to the slowness of the "Resolving information" step? Any news about this?

I have the same problem, it's taking more than 30 minutes and still going on...

[Edwin Yip 154]

On 4/5/2021 at 5:44 AM, Wagner Landgraf said:

Very interesting tool, I believe it opens lots of possibilities. I use AQTime and Sampling Profiler, is anyone able to tell the differences and advantages of VTune compared to those two tools?

Maybe price? ;)

[Edwin Yip 154]

Another great job done by @Anders Melander (apart from the drag-and-drop component suite)!

 

Another has a download link to the standalone VTune Profiler? I want to avoid downloading the 3.3GB oneAPI toolkit....

[Vincent Parrett 750]

55 minutes ago, Edwin Yip said:

I want to avoid downloading the 3.3GB oneAPI toolkit....

Use the web installer, you can then choose which parts to install.

[Edwin Yip 154]

18 minutes ago, Vincent Parrett said:

Use the web installer, you can then choose which parts to install.

Thanks! And realized it needs Win 10 and above ;)

[ConstantGardener 31]

1 hour ago, Edwin Yip said:

Another great job done by @Anders Melander (apart from the drag-and-drop component suite)!

 

Another has a download link to the standalone VTune Profiler? I want to avoid downloading the 3.3GB oneAPI toolkit....

....and "Better Translationmanager"!!!

[Anders Melander 1784]

5 hours ago, Wagner Landgraf said:

Are you referring to the slowness of the "Resolving information" step? Any news about this?

I have the same problem, it's taking more than 30 minutes and still going on...

I too have that problem. So far I haven't been able to profile our main project because the symbol resolution step takes longer (hours) than I'm willing to wait.

 

It's probably caused by a bug in map2pdb (e.g. some hash table is incorrect) but for now my plan is to add a white/black list option to include and exclude units from the pdb. So for example if I specify the switch -exclude:dx* then any unit that starts with "dx" (i.e. DevExpress) will be excluded from the pdb.

 

4 hours ago, Edwin Yip said:

Another has a download link to the standalone VTune Profiler?

It's hard to spot but there's one at the bottom of the download page:

https://software.intel.com/content/www/us/en/develop/articles/oneapi-standalone-components.html#vtune

 

3 hours ago, Edwin Yip said:

And realized it needs Win 10 and above 😉

I'm on Windows 7 so I had to find an older version of VTune. VTune 2019 was the last one to support Windows 7. Unfortunately it's a trial and it expires in a week...

[Anders Melander 1784]

New version with @Stefan Glienke's improvements uploaded.

https://bitbucket.org/anders_melander/map2pdb/downloads/

 

Notable changes:

• VTune now attributes samples to the correct source lines within a block.
• Removing RTTI from the exe has reduced the size by 15%.
• The pdb is now being built in a temporary memory stream which is significantly faster due to reduced I/O.