Mahdi Safsafi

2 hours ago, dummzeuch said:

and I haven't seen any code that checks that result.

ReadProcessMemory does an internal check at the end of the operation. If it fails an exception is thrown then.

Quote

 My assumption from the values I have seen is that these functions return the number of bytes read / written, but is that true? 

On the right road 🙂 

Quote

Is it documented anywhere?

No idea but I doubt if it was ... I mean ToolsApi documentation is very poor  

LSP does not fit with GExperts philosophy ! GExperts is a plugin that supports older versions perfectly. Adopting LSP means ... you know !

2 hours ago, Marat1961 said:

Karatsuba's algorithm is a fast multiplication algorithm.

Karatsuba noted that in fact only three multiplications of n / 2 - digit numbers are enough, since
(ad + bc) = (a + b) * (c + d) - ac - bd.

 

That is, using 64/2 = 32 bit numbers, you can get a 64 bit result.

 

N mod m where m = 2 ^ k is a power of two, you can discard bit digits more than k.
N mod 2 ^ 32 is a 32 bit number.

Yes that's right but Karatsuba has its own applications and would be great for BigInteger. But for OP I think that's too overhead compared to native HW multiplication.

27 minutes ago, David Heffernan said:

I don't think so. I think that you don't seem to understand the requirement. You certainly can't achieve the functionality shown in the original post from a native executable file (like the ones that Delphi produce).  

I clearly understand the requirement. What I discussed with you is different thing than what I suggested to Thomas. My answers were based on your statement which used a word output (BTW you still didn't explain what you mean by that) that makes me think to binary graph.

8 minutes ago, David Heffernan said:

A map file doesn't list functions calls. You can't map the graph of function calls from a map file.

Yes that's true and I'm aware about that. In fact my DebugEngine  is powered by detailed map file. It just seems to me that you didn't understand my comment. So here I explain again : Typically you only need the binary image (exe, dll) to generate the graph of functions ... map is just a sugar that gives additional information(location, ...).

Quote

Parsing isn't enough. You also need to interpret the tokens that the parser emits, using the syntax of the language.

For sure you need a parser. But it's not enough. You need more.

Sure DelphiAST doesn't give you that out of the box ... Although you need to make some effort.

26 minutes ago, David Heffernan said:

No. You need way more than a map file to find all the incoming references to a function.

I use external debugger heavily and all what I need to find references to a given function is the executable only ... map/pdb is just a sugar.  

If map wasn't that in your mind ... what's the actual one ?

Quote

This won't tell you the information either.

if you parse all units you can than get the information.

16 minutes ago, David Heffernan said:

grep is useless for this purpose. This functionality in VS works because the tooling is able to compile the code and understand all the references from the output of that compilation. A naive text match using grep will give nothing whatsoever of value.

Agree ! 

The output ... you mean map file ?

54 minutes ago, dummzeuch said:

GExperts does not paint into the editor window and that's not easy to implement (CnWizards does it though).

Definitely true ! However I've some useful information for anyone that is interested: 

CnWizards and typically Bookmarks (by David Millington) use DDetours to intercept some functions and paint their stuffs. David on his blog gave a good explanation on how to do that on two part:

Part1: https://parnassus.co/mysteries-of-ide-plugins-painting-in-the-code-editor-part-1/

Part2: https://parnassus.co/mysteries-ide-plugins-painting-code-editor-part-2/

BTW, this was the motivation behind developing chained hook for DDetours v2(because CnWizards and Bookmarks couldn't work correctly if they were active together at the same time) 🙂 

Quote

Also, Grep only searches for strings / regular expressions, it knows nothing about context, so it would be wrong for non unique identifiers, so this would be of very limited use.

1) You can use DelphiAST (by Roman Yankovsky) to parse the unit. 

2) If I recall correctly on the days were the Delphi community was on G+, someone (I think it was Stefan gleinke) he made a plugin that displays all symbols and types location for all Delphi units ... Unfortunately all what I remember is that the plugin supposed to work as a cache.

5 minutes ago, dummzeuch said:

Thanks a lot for volunteering.

 

Ijust made a release, the code to be tested is included in that. So in order to test it, simply install the release, enable the "Filter Exceptions" expert and debug code that raises exceptions. If the fix worked, you will get the new filter dialog for non-Windows projects. If it didn't, strange things (most likely AVs in the IDE) will happen. In the latter case, disable the "Filter Exceptions" expert and try again. The normal IDE dialog should appear and no AVs or other strange things should happen.

My expectation is a failure because in the last commit I saw, GetVmtOffset didn't implement offset(vmtParent, vmtClassName) for iOS/Android. 

function GetVmtOffset(Process: IOTAProcess; Offset: TVmtOffset): Int64;
begin
{$IFDEF IS_WIN32_ONLY}
  Result := GetVmtOffsetWin32(Offset);
{$ELSE}
  { Each platform should have corresponding constant value. }
  case Process.GetProcessType of
    optWin32:
      Result := GetVmtOffsetWin32(Offset);
    optWin64:
      Result := GetVmtOffsetWin64(Offset);
    // implement others...
  else
    raise Exception.Create('Please implement me.');
  end;
{$ENDIF}
end;

50 minutes ago, Attila Kovacs said:

@Kas Ob. I went through all these versions in my mind, even with clearing in the class operators, but it's just not right. You also have to know the implementation details for that version.



This doesn't sound problematic for me. A specialized string is expected to follow string rules hence same implementation details. After all that's what it designed for.

What about something like that ?

type
  TURL = type string;

  TURLHelper = record helper for TURL
    function AddFile(const AFile: string): TURL;
    function AddDir(const ADir: string): TURL;
  end;
  

@Vincent Parrett AFAIK there is no way ! The far thing I could get from dcc32 --version was the CompilerVersion. I also scanned all source files and it appears that none of them declare an update version. So what about writing a simple expert plugin that detects the update version at runtime and defines some macros, and then you can test for those macros at compile time ?

1 hour ago, Uwe Raabe said:

Are you sure? I guess that just boils down to declared(TObject) in both cases.

1 hour ago, Fr0sT.Brutal said:

upd

But not this way:

{$if not declared(TObject.Foo)}
  BOO! // will be ignored
{$endif}

Seems this check only works for members that actually exist

The path must be fully qualified (includes full unit).

{$IF  declared(System.TObject.Foo)}
 foo
{$ENDIF}

7 hours ago, Arnaud Bouchez said:

I know this.
The comment is correct in the context of our code base, of course, not absolutely.
Returning 0 and returning a not random value (i.e. checking the carry flag) won't hurt in our mORMot code base, since RDRAND is never used directly as random source, just as entropy source.

Unfortunately, I’m not in a position to discuss your internal design for the great mORMot library. So literally I’m just going to speak theoretically.

You used a DRNG instead of PRNG(LCG or whatever) -because you wanted a true RNG- to initialize an entropy source(ES) right ? What I’m seeing here is that your implementation breaks two fundamental rule of RNG : uniform distribution and unpredictable sequence ! How ? by counting 0 as TRN. This technically makes your ES vulnerable for backdoor !  

- An attacker may predicate 0 just because he knows that when RDRAND fails, it returns 0.

- What if he is knowing how to make RDRAND fail ? Now your ES is filled with zeros ! Intel didn’t describe all the circumstances that may lead to a failure. All what we know for now is a failure is expected if RN is not available didn’t pass the self-test ? In fact many people questioned Intel intentions when it putted some pressure on Linux kernel to use RDRAND/RDSEED … Eventually many concluded that a 3rd party (NSA?) was involved and may predicate/influence the output !!! Just google for the reason why the Linux kernel chipped out RDRAND/RDSEED.

If you permit me, I’d like to give some suggestions:

- Change the implementation by checking for CF and doing 10 time attempt when CF=0. I believe this will cost nothing compared to the additional security you get. BTW, that’s what Intel recommends:

Quote

 

5.2.1 Retry Recommendations

It is recommended that applications attempt 10 retries in a tight loop in the unlikely event

that the RDRAND instruction does not return a random number. This number is based on

a binomial probability argument: given the design margins of the DRNG, the odds of ten failures in a row are astronomically small and would in fact be an indication of a larger

CPU issue.

 

- Add another (optional) way to initialize the ES : e.g : CSPRNG, OS random data.

On 10/22/2020 at 3:22 PM, Arnaud Bouchez said:

 

// returns in eax, ignore carry flag (eax=0 won't hurt)

 

Not so ! 

Short answer : 0 is not granted to be a random value on all architectures.  

Long answer : https://software.intel.com/content/www/us/en/develop/blogs/rdrand-do-i-need-to-check-the-carry-flag-or-can-i-just-check-for-zero.html

Many compilers typically inserts some validations routines but all can be disabled (optional). Contracts are also optional. range checking for c++ vector can be enabled/disabled using some macro (its enabled by default for Debug and disabled for Release). So IMHO, I think it's better to follow the tradition and let the user decides whether he wants to compile with range checking or not. Personally I enable it under the debugger and disable it for Release.

Thanks Lars ! 

Just one small thing : the last URL that points to MS is invalid. 

1 hour ago, Marat1961 said:

When I see "division by zero" or "Access violation" it is generally a disaster - a tragedy.

This means that the problem did not occur at the library level, but at the hardware and processor levels.

For me, this is a signal that the memory is destroyed, or the code does not understand what.

There're a lot of place where an AV must be used/expected by tests to ensure a logic consistence. An example would be to test if a ROM structure holds at a given time ! Simply you can't use a comparison with an initial value because a write with the same initial value will make your test pass successfully. Another example is from madExcept instantly crash on buffer, if you want to validate that functionality, your test must expect a hardware exception when running overrun/underrun!

The point is : its all fine if those exceptions are expected to happen whether being hardware or software ! The not fine is when you expect your test to throw an exception and it does not ! 

While the library's concept is very interesting and worth investigating it, however, I found it incompatible with the Delphi language(at least in its current specs). In other word : what applies to c++ does not necessary apply to Delphi (the reverse holds as well). For example, the library has a sealed architecture ! Being based on records, it means that you can't roll easily your own collection based on the existing one (at least in the way we know all) just because the current status of the language does not permit record inheritance. In the other hand, RTL/Spring4D are more flexible in this area. 

Another thing, a quick look at the Heap manager, it appears that its based on the ordinal memory manager, in other word: a memory manager on the top of another memory manager. This technically may cause all sort of memory troubles.

I believe X->Free() is coming from the Delphi compatibility and is only used when importing Delphi object. Meaning it works only with TObject descendant. On the other hand delete X is a standard c++ operator that works with all objects whether they descend from TObject or not.

On this page, there is an explicit note that says : 

Quote

Note: In C++ code, do not use Free to destroy an object. Use the delete keyword.

View -> Debug Windows -> Breakpoints -> Delete all breakpoints.

It's really hard to tell but from what you described I think you're facing a memory corruption, I'd say a stack corruption -based on the raise; fails, raise exception; works-

I recommend the following things: 

- First, what happens if you omit the LogErrors  call ? does the AV gone ?

- Second, use madExcept and activate instanly crash on buffer this will help spotting heap corruption by over boundary.

- Third, put BP on raise statement and fall into RaiseAgain an see what line/instruction is causing the AV.

Please Thomas, check your Gexperts tickets.

4 hours ago, dummzeuch said:

I got it to display the exception address so far and changed the exception name to EAccessViolation. It's too late in the evening to get more work done, but I'll try to get the rest of the information too. It works for the Win32 only IDEs so far, but FDebugEvent is not available for later IDEs, so I need a different way to get the ExceptionInformation array contents for those.

 

You can get that info from the Parsed structure (see GetExceptionObjectNew) like this :

type
  TExceptionInformation = array [0 .. 14] of UInt64; // this is a UInt64 and not NativeUInt as RTL !
  PExceptionInformation = ^TExceptionInformation;

var Params:   PExceptionInformation;
  
  P := @Parsed[0];
  Inc(P, $A8);                         
  P := PPointer(P)^;                    // -----> Internal Exception record (not like RTL record !) 
  Params := (Pointer(PByte(P) + $30));  // -----> ExceptionInformation params. 
  

I guess, you will need to adjust GetExceptionObjectNew to return the new extra information.  

1 hour ago, Kas Ob. said:

I don't have RTL/VCL sources so i have to debug and walk them, if i am not wrong this is how to read access violation according to https://channel9.msdn.com/Shows/Inside/C0000005

and from debugging, i think there is GetExceptionObject in Sysutils that shows how AV message been built, this will help in rebuilding it for 64bit exactly as for 32bit

using FDebugEvent.Exception.ExceptionRecord you can read ExceptionAddress for the address instead of reading EIP, while the parameters list

ExceptionInformation[0] will hold the operation read,write,execute (the values can be obtained from msdn link above)

ExceptionInformation[1] will hold of the invalid address

Indeed everything is explained in SysUtils.GetExceptionObject. But I don't understand why Delphi IDE treats AV for 32bit/64bit differently.