MAP2PDB - Profiling with VTune

[Edwin Yip 154]

I guess map2pdb enabled the birth of the following new open source stack tracing library ;)

https://github.com/thatlr/Delphi-StackTrace

[Anders Melander 1784]

2 hours ago, Edwin Yip said:

I guess map2pdb enabled the birth of the following new open source stack tracing library 😉

https://github.com/thatlr/Delphi-StackTrace

Nice!

Any idea about who the author is?

[Edwin Yip 154]

2 hours ago, Anders Melander said:

Nice!

Any idea about who the author is?

I know nothing more than his github profile page ;)

[Attila Kovacs 629]

Hey @Anders Melander! This still rocks! Thx! 😉

[Anders Melander 1784]

7 minutes ago, Attila Kovacs said:

This still rocks! Thx! 😉

Thanks. I'm not using it that much myself at the moment because it takes VTune about an hour (no kidding!) to resolve the symbols of the application I'm working on 😕

 

[Attila Kovacs 629]

Yeah, I'm using it mostly with OllyDbg for debugging.

[Jan Rysavy 8]

I would like to implement support for reading of JEDI .jdbg files: https://bitbucket.org/anders_melander/map2pdb/issues/2/parser-for-jdbg-files

 

Does anyone know if .jdbg parser code exists? Otherwise I can probably write it based on TJclBinDebugGenerator.Create: https://github.com/andremussche/asmprofiler/blob/b785168f7caec8ff91d59d912dfb0e9e4519f803/Source/EXT/jcl/JclDebug.pas#L2358

 

Edit: nevermind, found .jdbg parser - TJclBinDebugScanner.

Edited June 15, 2022 by Jan Rysavy

[Mahdi Safsafi 225]

Excellent work ... Thanks !

I was using the tool with some map file and I got an AV error. I started investigating and I found a bug in the map parser : 

// debug.info.reader.map.pas (line 349):
var SegmentID: Cardinal := HexToInt16(Reader.LineBuffer, n);

The SegmentID should be read as a DECIMAL value and not as a HEX ! if the map file contains more than 9 segment, the next segment is emitted like (0010:xxxxx) but you're reading it like a hex (so its ID becomes 16 !) and then there is this line 

FSegments[ListIndex] := Result; // FSegments[10 .. ListIndex - 1] = nil => AV

 A simple fix : 

  function DecToInt32(const s: string; var Offset: integer): integer;
  var
    P: PChar;
  begin
    Result := 0;
    P := PChar(s);
    while Ord(P^) in [$30 .. $39] do
    begin
      Result := (Result * 10) + (Ord(P^) - $30);
      Inc(P);
      Inc(Offset);
    end;
  end;
// ...
var SegmentID: Cardinal := DecToInt32(reader.LineBuffer, n);

 

[Anders Melander 1784]

2 hours ago, Mahdi Safsafi said:

I found a bug in the map parser

Thanks. I'll get that fixed.

https://bitbucket.org/anders_melander/map2pdb/issues/4/segment-index-incorrectly-parsed-as-hex

[Attila Kovacs 629]

this bitbucket is a piece of shit

[Anders Melander 1784]

Yup. That pretty much sums it up.

Like everything else Atlassian touches it only gets worse with time.

 

My plan is to switch everything to Gitlab at some time but other stuff keeps getting in the way.

[Vincent Parrett 750]

22 minutes ago, Anders Melander said:

My plan is to switch everything to Gitlab at some time but other stuff keeps getting in the way

Much simpler to switch to github, for gitlab you have to apply for free hosting of open source. 

[Anders Melander 1784]

8 minutes ago, Vincent Parrett said:

Much simpler to switch to github

Yes, it is - but I like the spirit of Gitlab.

I need to get some real-life experience with it anyway because I also plan to switch the company from BitBucket to Gitlab at some point. Github is way too expensive for enterprise use.

 

8 minutes ago, Vincent Parrett said:

for gitlab you have to apply for free hosting of open source. 

I've had a Gitlab account for quite some time. AFAIR I didn't need to "apply". I think I just checked a box somewhere.

Edited January 12, 2023 by Anders Melander

[Vincent Parrett 750]

27 minutes ago, Anders Melander said:

I've had a Gitlab account for quite some time. AFAIR I didn't need to "apply". I think I just checked a box somewhere.

Ah ok, I misread their site - was looking at this page - https://about.gitlab.com/solutions/open-source/join/

 

Didn't realise that was for the full version of gitlab. 

[David Champion 48]

@Anders Melander I am using Code Commit for my latest project, during lock down I needed to swap out the On Premise version of BitBucket.

It has the feature of locating the service at a particular data center and encrypting the source code at rest.

 

It's worth a look if you haven't already. Then again I don't know what you are looking for.

[Stefan Glienke 2003]

GitLab is amazing and you can host on-premise - for some people including our company, this is important.

[Anders Melander 1784]

On 1/12/2023 at 10:00 PM, Anders Melander said:

Thanks. I'll get that fixed.

https://bitbucket.org/anders_melander/map2pdb/issues/4/segment-index-incorrectly-parsed-as-hex

Fixed.

[Jan Rysavy 8]

Are any of you running MAP2PDB with the current version of Intel VTune 2023.1? Both AMD uProf and WinDBG correctly load my PDBs created by the latest version of MAP2PDB, but VTune only shows function addresses without symbolic names.

[shineworld 73]

Incredible tool, congrats.

I've tried to use it with Sydney 10.4.1 and VTune 2023.1.0.
 

Unfortunately, I was not able to get the function as name, I get only func addresses:

 

I've tried with:

D:\x\develop\qem\rosetta_cnc_1>map2pdb.exe rosettacncpph1.64.map -bind
map2pdb - Copyright (c) 2021 Anders Melander
Version 2.8.0

and also with:

D:\x\develop\qem\rosetta_cnc_1>map2pdb.exe -bind rosettacncpph1.64.map
map2pdb - Copyright (c) 2021 Anders Melander
Version 2.8.0

Map is set to Detailed on Sydney 10.4.1:

 

D:\x\develop\qem\rosetta_cnc_1>map2pdb.exe rosettacncpph1.64.map -bind -v
map2pdb - Copyright (c) 2021 Anders Melander
Version 2.8.0

Constructed a new PDB GUID: {5D8591F1-8F38-4E7B-BAE4-1BB55536733F}
Output filename not specified. Defaulting to rosettacncpph1.64.pdb
Reading MAP file
- Segments
- Modules
- Symbols
Warning: [116390] Failed to resolve symbol to module: [0004:00000000000002C8] SysInit.TlsLast
Warning: [116392] Failed to resolve symbol to module: [0003:00000000FE7F6000] SysInit.__ImageBase
- Line numbers
Collected 3.996 modules, 182.925 symbols, 525.223 lines, 985 source files
Constructing PDB file
- Collecting source file names
- Module streams
- Strings stream
- PDB Info stream
- TPI stream
- Symbols stream
- DBI stream
- IPI stream
- Finalizing PDB file
- 9.068 blocks written in 3 intervals
PE filename not specified. Defaulting to rosettacncpph1.64.exe
Patching PE file
- PE32+ image (64-bit)
- Adding .debug section.
- PDB file name has been stored in debug data.
- PE file has been updated.
Elapsed time: 00:00:00.874

 

I hope you have a simple solution to my possible mistakes.

Edited May 22, 2023 by shineworld

[Jan Rysavy 8]

I just want to add that I installed an older version of VTune 2022.4.1 and the function names are displayed correctly. The problem seems to be related to the new version VTune 2023.1.

[shineworld 73]

9 minutes ago, Jan Rysavy said:

I just want to add that I installed an older version of VTune 2022.4.1 and the function names are displayed correctly. The problem seems to be related to the new version VTune 2023.1.

I've tried to find a way to download the older version (2022.4.1) but no way found on the intel VTune pages....

[Stefan Glienke 2003]

I just upgraded from 2023.0.0 where it worked to 2023.1.0 where it does not.

 

What I noticed in the change log of VTune 2023.1.0 is this:

 

Debug Support 

Support for Large PDB Files 
Starting with the 2023.1 version, on Windows* systems, you can resolve debug information from PDB files larger than 4GB. 

 

Could it be that the address bitness written by map2pdb is not correct now?

 

In the collection log of VTune I can see this warning:

 

Cannot locate debugging information for file `somepath\Tests.exe'. Cannot match the module with the symbol file `somepath\Tests.pdb'. Make sure to specify the correct path to the symbol file in the Binary/Symbol Search list of directories.

Edited May 22, 2023 by Stefan Glienke

[Anders Melander 1784]

45 minutes ago, shineworld said:

I've tried to find a way to download the older version (2022.4.1) but no way found on the intel VTune pages....

This might help:

https://www.intel.com/content/www/us/en/developer/articles/troubleshooting/older-version-product.html

 

14 hours ago, Jan Rysavy said:

PDBs created by the latest version of MAP2PDB, but VTune only shows function addresses without symbolic names.

Please create an issue at the repository: https://bitbucket.org/anders_melander/map2pdb/issues/new

 

The problem is most likely caused by msdia140.dll; The library VTune uses to read pdb files. You can try replacing the one that VTune installs with an older version. See:

 

[Anders Melander 1784]

2 minutes ago, Stefan Glienke said:

Could it be that the address bitness written by map2pdb is not correct now?

Please elaborate

[Stefan Glienke 2003]

Just now, Anders Melander said:

Please elaborate

I can't - as I confirmed the breaking change is from 2023.0.0 to 2023.1.0 and that explicit mention of large pdb file support was the most obvious clue. I don't know anything about the pdb format.