MAP2PDB - Profiling with VTune

[Anders Melander 1635]

11 minutes ago, Anders Melander said:

address bitness

I meant what do you mean by this ^

 

Do you mean 32- vs 64-bit addresses? AFAIR there's no choice or ambiguities in the PDB format with regard to the size of an address value, relative or absolute, but I would have to take a look at the source to make sure.

[Stefan Glienke 1901]

2 minutes ago, Anders Melander said:

I meant what do you mean by this ^

 

Do you mean 32- vs 64-bit addresses? AFAIR there's no choice or ambiguities in the PDB format with regard to the size of an address value, relative or absolute, but I would have to take a look at the source to make sure.

Yes - because apparently according to the warning, there is a problem with "match the module with the symbol file". It could be very well a VTune bug though - after all the 2023.1.0 we currently get is a "pre-release".

[Anders Melander 1635]

51 minutes ago, Stefan Glienke said:

Cannot locate debugging information for file `somepath\Tests.exe'. Cannot match the module with the symbol file `somepath\Tests.pdb'. Make sure to specify the correct path to the symbol file in the Binary/Symbol Search list of directories.

I think that is a generic error message meaning "Something went wrong and our error handling sucks". As far as I remember you get a message like that regardless of what problem VTune encounters when resolving through the PDB file.

[Jan Rysavy 8]

Created issue: https://bitbucket.org/anders_melander/map2pdb/issues/5/intel-vtune-20231-does-not-load-generated

[Jan Rysavy 8]

2 hours ago, Anders Melander said:

The problem is most likely caused by msdia140.dll; The library VTune uses to read pdb files. You can try replacing the one that VTune installs with an older version.

 

I can confirm replacing C:\Program Files (x86)\Intel\oneAPI\vtune\2023.1.0\bin64\amplxe_msdia140.dll (version 14.34.31942.0) with version 14.28.29910.0 from VTune 2022.4.1 solves this problem!

Edited May 22, 2023 by Jan Rysavy

[Anders Melander 1635]

8 minutes ago, Jan Rysavy said:

I can confirm replacing C:\Program Files (x86)\Intel\oneAPI\vtune\2023.1.0\bin64\amplxe_msdia140.dll (version 14.34.31942.0) with version 14.28.29910.0 from VTune 2022.4.1 solves this problem!

That means the bug is most likely in map2pdb because that DLL is Microsoft's API for reading PDB files.

[Jan Rysavy 8]

This also probably explains why Profiler in Visual Studio (2022) stopped displaying function names in recent versions. It uses newer versions of msdia140.dll.

 

[Jan Rysavy 8]

VTune 2023.1.0 works fine also with C:\Program Files\AMD\AMDuProf\bin\msdia140.dll (version 14.29.30035.0).

[Jan Rysavy 8]

I tried to load the PDB using the current version of C:\Program Files\Microsoft Visual Studio\2022\Community\DIA SDK (14.36.32532.0) and the method loadDataFromPdb returns the error E_PDB_FORMAT.

 

The same test with older msdia140.dll (14.29.30035.0) works.

Edited May 22, 2023 by Jan Rysavy

[Stefan Glienke 1901]

This could be some relevant info: https://randomascii.wordpress.com/2023/03/08/when-debug-symbols-get-large/

[Anders Melander 1635]

28 minutes ago, Stefan Glienke said:

This could be some relevant info: https://randomascii.wordpress.com/2023/03/08/when-debug-symbols-get-large/

Looks like it.

 

 

1 hour ago, Jan Rysavy said:

This also probably explains why Profiler in Visual Studio (2022) stopped displaying function names in recent versions. It uses newer versions of msdia140.dll.

Ooooh, interesting. Maybe they've accidentally broken support for the older format and not noticed it because they're only testing the new format now.

 

The article Stefan linked to makes me think that even though the PDB format supported large PDB files, the PDB reader (msdia140.dll) didn't. Otherwise, they would only have had to update their PDB writer to support large PDB files.

[shineworld 68]

2 hours ago, Jan Rysavy said:

 

I can confirm replacing C:\Program Files (x86)\Intel\oneAPI\vtune\2023.1.0\bin64\amplxe_msdia140.dll (version 14.34.31942.0) with version 14.28.29910.0 from VTune 2022.4.1 solves this problem!

Can you attach the working DLL to try ?

 

[Jan Rysavy 8]

26 minutes ago, shineworld said:

Can you attach the working DLL to try ?

 

 

msdia140.zip

[Jan Rysavy 8]

MS released debug symbols for msdia140.dll... nice

[shineworld 68]

Seems to work....


Original version raise error on PDB file reading

Edited May 22, 2023 by shineworld

[Stefan Glienke 1901]

@Anders Melander Might be worth looking into https://github.com/llvm/llvm-project/blob/main/llvm/lib/DebugInfo/PDB/Native/PDBFileBuilder.cpp

[mvanrijnen 121]

Just a quick question is the 16Mb limit still valid for the tool?

 

i keep getting:
"Cannot locate debugging information for file `D:\Data\company\Delphi\Projects\SVN\appname4\Clients\appname\Win32\Debug\appname.exe'. Cannot match the module with the symbol file `D:\Data\company\Delphi\Projects\SVN\appname4\Clients\appname\Win32\Debug\appname.pdb'. Make sure to specify the correct path to the symbol file in the Binary/Symbol Search list of directories."

While the .pdb is there.

(i replaced the msdia dll)

 

works, replaced the wrong msdia.dll 🙂

 

Edited May 22, 2023 by mvanrijnen

[shineworld 68]

13 minutes ago, mvanrijnen said:

8 minutes ago, mvanrijnen said:

Just a quick question is the 16Mb limit still valid for the tool?

I'm not an expert with PDB but that is what I've done and now seems to work:
 

- Installed the latest version of VTune Profiler (2023.1.0).
- Changed the original amplxe_media140.dll with an old version from Jan Rysavy (previous posts).
- Enabled map in Detailed mode.
- Compiled a project of 1.359.947 code lines (including library sources) which generate a map file of 49.696.190 bytes.
- Executed map2pdb.exe rosettacncpph1.64.map -bind -v

D:\x\develop\qem\rosetta_cnc_1>map2pdb.exe rosettacncpph1.64.map -bind -v
map2pdb - Copyright (c) 2021 Anders Melander
Version 2.8.0

Constructed a new PDB GUID: {F2D8CB4B-DA08-4BBD-A399-DBC449AF1364}
Output filename not specified. Defaulting to rosettacncpph1.64.pdb
Reading MAP file
- Segments
- Modules
- Symbols
Warning: [116390] Failed to resolve symbol to module: [0004:00000000000002C8] SysInit.TlsLast
Warning: [116392] Failed to resolve symbol to module: [0003:00000000FE7F6000] SysInit.__ImageBase
- Line numbers
Collected 3.996 modules, 182.925 symbols, 525.223 lines, 985 source files
Constructing PDB file
- Collecting source file names
- Module streams
- Strings stream
- PDB Info stream
- TPI stream
- Symbols stream
- DBI stream
- IPI stream
- Finalizing PDB file
- 9.068 blocks written in 3 intervals
PE filename not specified. Defaulting to rosettacncpph1.64.exe
Patching PE file
- PE32+ image (64-bit)
- Adding .debug section.
- PDB file name has been stored in debug data.
- PE file has been updated.
Elapsed time: 00:00:00.895

This has generated a PDB file of 37.142.528 bytes

- Started profiling of EXE in VTune profiler which generates this log at the profiling stop:

Data collection is completed successfully

May 22 2023 18:06:59 The result file 'XXX\r001hs\r001hs.vtune' is successfully created and added to the project .

Finalization completed with warnings

May 22 2023 18:08:33 Result finalization has completed with warnings that may affect the representation of the analysis data. Please see details below.

Cannot locate debugging information for file `C:\WINDOWS\System32\msvcrt.dll'.
Cannot locate debugging information for file `C:\WINDOWS\WinSxS\amd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.19041.1110_none_60b5254171f9507e\COMCTL32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\WinSxS\amd64_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.19041.2251_none_91a40448cc8846c1\gdiplus.dll'.
Cannot locate debugging information for file `C:\WINDOWS\SYSTEM32\ntdll.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\GDI32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\SETUPAPI.DLL'.
Cannot locate debugging information for file `C:\WINDOWS\System32\KERNEL32.DLL'.
Cannot locate debugging information for file `C:\WINDOWS\System32\cfgmgr32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\gdi32full.dll'.
Cannot locate debugging information for file `C:\Program Files\Bitdefender\Endpoint Security\bdhkm\dlls_266262988153465131\bdhkm64.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\user32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\combase.dll'.
Cannot locate debugging information for file `C:\Program Files\Bitdefender\Endpoint Security\atcuf\dlls_266575548366517634\atcuf64.dll'.
Cannot locate debugging information for file `C:\WINDOWS\system32\mswsock.dll'.
Cannot locate debugging information for file `C:\WINDOWS\system32\uxtheme.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\ole32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\SYSTEM32\opengl32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\SYSTEM32\DEVOBJ.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\KERNELBASE.dll'.
Cannot locate debugging information for file `C:\WINDOWS\SYSTEM32\TextShaping.dll'.
Cannot locate debugging information for file `C:\Windows\System32\msxml6.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\WS2_32.dll'.
Cannot locate debugging information for file `C:\WINDOWS\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_c1a085cc86772d3f\nvoglv64.dll'.
Cannot locate debugging information for file `C:\WINDOWS\SYSTEM32\HID.DLL'.
Cannot locate debugging information for file `C:\WINDOWS\System32\win32u.dll'.
Cannot locate debugging information for file `C:\Program Files (x86)\Intel\oneAPI\vtune\latest\bin64\tpsstool.dll'.:

And all symbols project symbols are visible (not for below list of DLL):

[Anders Melander 1635]

45 minutes ago, Stefan Glienke said:

@Anders Melander Might be worth looking into https://github.com/llvm/llvm-project/blob/main/llvm/lib/DebugInfo/PDB/Native/PDBFileBuilder.cpp

Yeah... Not too keen on that as a first approach.

The last time I tried using the llvm pdb support as a reference I wasted a lot of time before I found out that it was very incomplete to the point of being unusable by VTune. It has probably improved but since then it's hard to tell what state it's in.

https://github.com/llvm/llvm-project/issues/37279

https://github.com/llvm/llvm-project/issues/28528

 

I will try to see if I can reproduce and spot the problem in the source before I go down that road. Thanks anyway.

[Anders Melander 1635]

2 hours ago, Jan Rysavy said:

MS released debug symbols for msdia140.dll... nice

Neat. If you can spot where it gives up on the pdb file and returns an error that would be suuuuper nice.

 

Does it produce any debug output while loading?

[Anders Melander 1635]

3 hours ago, Stefan Glienke said:

This could be some relevant info: https://randomascii.wordpress.com/2023/03/08/when-debug-symbols-get-large/

Quote

we added a use_large_pdbs build setting which would switch the PDB page size to 8 KiB. However this setting initially needed to be off by default due to a lack of complete tool support.

If the block size being 4096 is the only problem (I somehow doubt that I'm that lucky) then this is the line that needs to be changed to write 8192-byte blocks:

https://bitbucket.org/anders_melander/map2pdb/src/2341200827af24f7dd75cb695a668dfa9564bcf5/Source/debug.info.writer.pdb.pas#lines-225
 

constructor TDebugInfoPdbWriter.Create;
begin
  Create(4096);
end;

 

[Jan Rysavy 8]

4 hours ago, Anders Melander said:

Does it produce any debug output while loading?

Seems there is some logger framework, probably undocumented. In msdia140.zip is attached 'wt' command output for msdia140!CDiaDataSource::loadDataFromPdb.

msdia140.zip

[Anders Melander 1635]

35 minutes ago, Jan Rysavy said:

In msdia140.zip is attached 'wt' command output for msdia140!CDiaDataSource::loadDataFromPdb.

Excellent!

A few quick observations:

 

First of all, it's strange that the error isn't logged. That would

 

StrmTbl::internalSerializeBigMsf
Lots of calls to this. I'm guessing it's reading MSF blocks and deblocking them into linear memory streams. This is probably the new code that supports the 8192-byte "big" MSF block size.

 

MSF_HB::load

Probably the code that loads the PDB tables from the memory streams.

 

StrmTbl::~StrmTbl

Lots of calls to this. Probably clean up after the load has been aborted.

 

PortablePDB::PortablePDB
Something wrong here. "Portable PDB" is the .NET PDB format. It's a completely different file format.
I'm guessing it's falling back to that format after failing to validate the file as PDB.

[Jan Rysavy 8]

15 minutes ago, Anders Melander said:

PortablePDB::PortablePDB

Something wrong here. "Portable PDB" is the .NET PDB format. It's a completely different file format.
I'm guessing it's falling back to that format after failing to validate the file as PDB.

Yes, see attached 'wt' output from old msdia140.dll 14.29.30035.0 for the same input PDB file. In this case loadDataFromPdb succeeds.

msdia14_142930035.zip

[Anders Melander 1635]

39 minutes ago, Jan Rysavy said:

Yes, see attached 'wt' output from old msdia140.dll 14.29.30035.0 for the same input PDB file. In this case loadDataFromPdb succeeds.

msdia14_142930035.zip

Ew! It looks like they have done a complete rewrite. No wonder it's broken.

 

So I guess this is an example of the main problem with the PDB format: Microsoft considers it their own internal format to do with what they like. They have their own (undocumented) writer, their own reader, and no documentation.