Jump to content
Anders Melander

MAP2PDB - Profiling with VTune

Recommended Posts

Well, OllyDbg loads fine a 27Mb pdb and I picked an address from the end of the map and it's translated in the debugger.

Great work!

 

Share this post


Link to post

There is however this message on bind: "Image does not contain a debug directory address - please link with debug info enabled" It's enabled.

edit: hmmmm .debug is not present in the exe

edit2: got it, place debug info in a separate tds file was checked

Edited by Attila Kovacs

Share this post


Link to post

I've seen in procmon that something happens but slow af and had nothing to do with the pdb. I'm still waiting for the confirmation email for the intel tech forum where google found some similar posts.

Share this post


Link to post

Is map file format different in older delphi versions? With Delphi 5 and Delphi XP map files it always fails with  "Overlapping segments".

Share this post


Link to post
22 minutes ago, Virgo said:

Is map file format different in older delphi versions?

I don't know.

 

22 minutes ago, Virgo said:

With Delphi 5 and Delphi XP map files it always fails with  "Overlapping segments".

And does the segments overlap?

Share this post


Link to post

Now, that I'm checking: is 0 length .xdata ovelapping .tls?

 

Start         Length     Name                   Class
 0001:00401000 00003B90H .text                   CODE
 0002:00405000 00000100H .itext                  ICODE
 0003:00406000 000007BCH .data                   DATA
 0004:00407000 00002B74H .bss                    BSS
 0005:00000000 00000008H .tls                    TLS
 000A:00000000 00000000H .xdata                  PDATA


Detailed map of segments

 0001:00000000 000036F4 C=CODE     S=.text    G=(none)   M=System   ACBP=A9
 0001:000036F4 00000110 C=CODE     S=.text    G=(none)   M=SysInit  ACBP=A9
 0001:00003804 0000038C C=CODE     S=.text    G=(none)   M=test     ACBP=A9
 0002:00000000 000000AC C=ICODE    S=.itext   G=(none)   M=System   ACBP=A9
 0002:000000AC 00000054 C=ICODE    S=.itext   G=(none)   M=test     ACBP=A9
 0003:00000000 00000794 C=DATA     S=.data    G=DGROUP   M=System   ACBP=A9
 0003:00000794 00000024 C=DATA     S=.data    G=DGROUP   M=SysInit  ACBP=A9
 0004:00000000 00002B68 C=BSS      S=.bss     G=DGROUP   M=System   ACBP=A9
 0004:00002B68 0000000C C=BSS      S=.bss     G=DGROUP   M=SysInit  ACBP=A9
 0005:00000000 00000008 C=TLS      S=.tls     G=(none)   M=System   ACBP=A9

 

Share this post


Link to post
1 hour ago, Virgo said:

Now, that I'm checking: is 0 length .xdata ovelapping .tls?

Yes, but it's a bug on my part.

I previously ignored empty segments (i.e. the PDATA segment) but that caused other problems so now I add them to the segment list. I just forgot that I still need to ignore them when checking for overlaps.

 

You can probably just delete the line from you map file. If there are any references to the segment in the map file ten you will have to remove those as well (search for 000A:).

Share this post


Link to post

Warning: [   41] Failed to resolve symbol to module: [0005:00000008] SysInit.TlsLast

But otherwise ok.

 

But Delphi 5 is weird and probably hopeless...

 Start         Length     Name                   Class
 0001:00000000 00002B14H .text                   CODE
 0002:00000000 000000BCH .data                   DATA
 0002:000000BC 000004D1H .bss                    BSS

 

Share this post


Link to post

Fixed the problem where some symbols (in particular generics) either resolved to the calling unit or to a wrong line number in the implementing unit.

Thanks to @Stefan Glienke for pestering me about this until it got solved.

 

Source committed and binary uploaded. And now... Pizza! 🍕

  • Like 4
  • Thanks 5

Share this post


Link to post
1 hour ago, Virgo said:

But Delphi 5 is weird and probably hopeless...

 Start         Length     Name                   Class
 0001:00000000 00002B14H .text                   CODE
 0002:00000000 000000BCH .data                   DATA

you could read those offsets from the PE (optionally), you are already iterating through the sections, should not be a big problem

Share this post


Link to post

With @Stefan Glienke changes merged it's blazingly fast👍. I added a postbuild event to every project in my project group (131 projects) and barely notice it running now! 

 

This is what I have for the postbuild event

 

%MAP2PDB% $(OUTPUTDIR)\$(OUTPUTNAME).map -bind:$(OUTPUTDIR)\$(OUTPUTFILENAME)

 

Note that I did needed the outputfilename part for the bind option as it only looks for exe (so for bpl, dll you need it).  I created an environment variable for the path to map2pdb to avoid hard coding the path in my dproj files. Sadly if you use libsuffix anywhere you need to manually add that in to the map file arg as $(OUTPUTNAME) doesn't include it, and there isn't a macro for it. 

 

Share this post


Link to post
6 minutes ago, Vincent Parrett said:

it's blazingly fast👍.

Profiling map2pdb with VTune using a pdb built with the map file from map2pdb 🤯

 

6 minutes ago, Vincent Parrett said:

Note that I did needed the outputfilename part for the bind option as it only looks for exe (so for bpl, dll you need it). 

I guess that is fixable.

Edited by Stefan Glienke
  • Like 2

Share this post


Link to post
On 4/6/2021 at 5:53 AM, Attila Kovacs said:

I've seen in procmon that something happens but slow af and had nothing to do with the pdb. I'm still waiting for the confirmation email for the intel tech forum where google found some similar posts.

Are you referring to the slowness of the "Resolving information" step? Any news about this?

I have the same problem, it's taking more than 30 minutes and still going on...

Share this post


Link to post
On 4/5/2021 at 5:44 AM, Wagner Landgraf said:

Very interesting tool, I believe it opens lots of possibilities. I use AQTime and Sampling Profiler, is anyone able to tell the differences and advantages of VTune compared to those two tools?

Maybe price? ;)

Share this post


Link to post

Another great job done by @Anders Melander (apart from the drag-and-drop component suite)!

 

Another has a download link to the standalone VTune Profiler? I want to avoid downloading the 3.3GB oneAPI toolkit....

Share this post


Link to post
55 minutes ago, Edwin Yip said:

I want to avoid downloading the 3.3GB oneAPI toolkit....

Use the web installer, you can then choose which parts to install.

  • Like 1

Share this post


Link to post
18 minutes ago, Vincent Parrett said:

Use the web installer, you can then choose which parts to install.

Thanks! And realized it needs Win 10 and above ;)

Share this post


Link to post
1 hour ago, Edwin Yip said:

Another great job done by @Anders Melander (apart from the drag-and-drop component suite)!

 

Another has a download link to the standalone VTune Profiler? I want to avoid downloading the 3.3GB oneAPI toolkit....

....and "Better Translationmanager"!!!

  • Like 1

Share this post


Link to post
5 hours ago, Wagner Landgraf said:

Are you referring to the slowness of the "Resolving information" step? Any news about this?

I have the same problem, it's taking more than 30 minutes and still going on...

I too have that problem. So far I haven't been able to profile our main project because the symbol resolution step takes longer (hours) than I'm willing to wait.

 

It's probably caused by a bug in map2pdb (e.g. some hash table is incorrect) but for now my plan is to add a white/black list option to include and exclude units from the pdb. So for example if I specify the switch -exclude:dx* then any unit that starts with "dx" (i.e. DevExpress) will be excluded from the pdb.

 

4 hours ago, Edwin Yip said:

Another has a download link to the standalone VTune Profiler?

It's hard to spot but there's one at the bottom of the download page:

https://software.intel.com/content/www/us/en/develop/articles/oneapi-standalone-components.html#vtune

 

3 hours ago, Edwin Yip said:

And realized it needs Win 10 and above 😉

I'm on Windows 7 so I had to find an older version of VTune. VTune 2019 was the last one to support Windows 7. Unfortunately it's a trial and it expires in a week...

  • Like 2

Share this post


Link to post

New version with @Stefan Glienke's improvements uploaded.

https://bitbucket.org/anders_melander/map2pdb/downloads/

 

Notable changes:

  • VTune now attributes samples to the correct source lines within a block.
  • Removing RTTI from the exe has reduced the size by 15%.
  • The pdb is now being built in a temporary memory stream which is significantly faster due to reduced I/O.
  • Like 3
  • Thanks 1

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×