Mahdi Safsafi

Excellent work ... Thanks ! I was using the tool with some map file and I got an AV error. I started investigating and I found a bug in the map parser : // debug.info.reader.map.pas (line 349): var SegmentID: Cardinal := HexToInt16(Reader.LineBuffer, n); The SegmentID should be read as a DECIMAL value and not as a HEX ! if the map file contains more than 9 segment, the next segment is emitted like (0010:xxxxx) but you're reading it like a hex (so its ID becomes 16 !) and then there is this line FSegments[ListIndex] := Result; // FSegments[10 .. ListIndex - 1] = nil => AV A simple fix : function DecToInt32(const s: string; var Offset: integer): integer; var P: PChar; begin Result := 0; P := PChar(s); while Ord(P^) in [$30 .. $39] do begin Result := (Result * 10) + (Ord(P^) - $30); Inc(P); Inc(Offset); end; end; // ... var SegmentID: Cardinal := DecToInt32(reader.LineBuffer, n);

@Pawel Piotrowski My bad ! I was much focusing on the pre-allocation rather than the actual code ! Thanks!

@Kas Ob. As I told you ... its a problem related to VM ! // What happens when variable wasn't declared statically !!! MoveInterceptRec := GetMemory(xx); // the variable can be located far >2GB.

@Kas Ob. No ! its more complicated than what you're describing ! Its not about the ability of the compiler to generate more than 2GB ! its about the virtual memory ! On x64, the OS can load DLL at the highest address and your hook can be located at the lowest address ... you see ? Also what happens if the variable is created at runtime (through a call to MM)!

Mike, you should always rationalize your resource and don't let someone else do it for you (MM) because this can have a wide effect : - OS may start to page things. - All sort of thrashing issues (check the link). - Cache miss. - Performance penalty. - ...

No! as I told you can quickly run out of memory. Imagine what will happen when you use 1 GB and another thread just reclaimed a big chunk of memory ! out of memory !

When you ask for memory, MM(FastMM) asks the OS for a large chunks and then it splits them and gives you a piece (based on the size you need). When the object is destroyed (free), the memory is returned to the MM. Now, based on the returned size, the MM may either choose to recycle the object location (if its small) or return the memory to the OS (a real-free-op). What you're doing in MultiplyStr is not just wasteful but extremely harmful ! For each iteration you're reallocating memory. Allocating a new block and copying the old block to the new one. It's very important to know that small block are implemented as a segregated list. i.e if you ask for a 32 bytes, MM on reality allocates an entire table i.e 32x32=1024 bytes and yields first block. In your example you said you used 1GB ! this is extremely bad because you're not economizing resources and you'll quickly run out of memory i.e another thread that asks for a large chunk. It's indeed a good practice to pre-allocate memory : function MultiplyStr(aStr: string; aMultiplier: integer): string; var i: Integer; begin SetLength(Result, length(aStr) * aMultiplier); for i := 1 to aMultiplier do Result := Result + aStr; end; Please run the above and notice the memory and performance !!! Also a small remark ! aStr should be const !!!

@Kas Ob. Cool idea ... I love it ! Although, this could be somehow challenging on x64 where data (variable that holds the pointer) could be located so far (exceeding 32-bit range) ... But I have a couple of ideas how to handle it ... Again thanks for the great idea 🙂

Ah I see ... I thought you were referring to : - Build - Pack - Run one time and generate a patched version - ... Thanks ! I'll investigate further on your idea and if all works great, I'll release it as an IDE plugin 🙂

Thanks I got clearly your idea ... but I have some hard time to digest this statement : Technically this won't work ! The packer unpacks the exe in memory, then DDetours applies patch ... right ? But now we ended-up with an unpacked exe ... You can't simply dump this on disk !

Why ? the packer encrypts/compresses the exe in disk but soon the exe is loaded in memory, packer had to decrypt/decompress the exe and DDetours applies hook at runtime (in memory). So I don't think that using DDetours prevents anyone from using a Packer. Please if you encountered such a case send me some details 🙂 Indeed ! Personally I spotted many place where core RTL functions were doing crappy things. Here is what I understand from your idea (please correct me if I'm wrong): I'm I right to think that you want DDetours to hook functions on the first run and then generates a patched exe file that eventually will be saved on disk ? If so, then this can't be done because DDetours can't pack/compress exe ... In other word, it will fail the day you use a packer. Thanks for your ideas 🙂

AFAIK, there is no CE for RAD Studio ! There is only CE for Delphi or C++Builder and you can't have both.

@Lars Fosdal I have a weird issue ... sometime my comments are erased when I try to insert a Code box. Do you know anything about this ?

It's an intrinsic and has no implementation ... the function is evaluated on the fly (no code will be generated). BTW, Spring4D makes extensively use of it.

No ! I'm sure you'll become a good developer and BTW I don't find your topics annoying. I just don't want you to spend your valuable time on something that doesn't really make difference. You said you're working alone ... so literally you can pick whatever naming you like. In other word, your topic is just a color and you know that there is no color better than other 🙂

Please when you have some spry time, take a look at this : Parkinson’s Law of Triviality "bike-shedding"

Please see wiki : https://github.com/MahdiSafsafi/DDetours/wiki#enterrecursivesection

Maybe but its not just related to jagged array or weak refs. Here is a simple example where a copy happens too: procedure TForm1.Button1Click(Sender: TObject); var // array of simple type LArray, LArray2: array of Integer; begin SetLength(LArray, 100); LArray2 := LArray; SetLength(LArray2, 100); // copy happens here too end; With all that being said, I think that a fix at the RTL level is necessary.

Or addressing it at the RTL level by implementing a simple check in DynArraySetLength.

@David Heffernan Please step into for the second call and you'll clearly see that the second call performs a copy even size didn't changed : type TRec = record // [Weak] FInterface: IInterface; end; procedure TForm1.Button1Click(Sender: TObject); var LArray: array of TRec; begin SetLength(LArray, 100); SetLength(LArray, 100); // step in end;

All the points I demonstrated are related to the use of SetLength without a check. A copy happens when array contains weak-references: // DynArraySetLength: if SysHasWeakRef(PTypeInfo(ElTypeInfo)) then begin ... GetMem(pp, neededSize); FillChar((PByte(pp) + SizeOf(TDynArrayRec))^, minLength * elSize, 0); MoveArray(PByte(pp) + SizeOf(TDynArrayRec), PByte(p) + SizeOf(TDynArrayRec), ElTypeInfo, minLength); end

OP was referring to SetLength (not jagged/multidimensional array) and he was wondering whether a check against old-length should be performed before calling SetLength. So I think it makes a sense and worth to be mentioned 🙂

In fact it can be a serious bottleneck !!! 1 - A copy operation may happen even if the block wasn't resized (same size) ... weak-ref : procedure DynArraySetLength(var a: Pointer; typeInfo: Pointer; dimCnt: NativeInt; lengthVec: PNativeint); //.... if SysHasWeakRef(PTypeInfo(ElTypeInfo)) then begin if newLength < oldLength then minLength := newLength else minLength := oldLength; GetMem(pp, neededSize); FillChar((PByte(pp) + SizeOf(TDynArrayRec))^, minLength * elSize, 0); if p <> nil then begin // ---> here <--- MoveArray(PByte(pp) + SizeOf(TDynArrayRec), PByte(p) + SizeOf(TDynArrayRec), ElTypeInfo, minLength); if newLength < oldLength then FinalizeArray(PByte(p) + SizeOf(TDynArrayRec) + newLength*elSize, ElTypeInfo, oldLength - newLength); FreeMem(p); end; end 2 - The operation executes on O(n) when the array is multidimensional : // Take care of the inner dimensions, if any if dimCnt > 1 then begin Inc(lengthVec); Dec(dimCnt); i := 0; try while i < newLength do begin DynArraySetLength(PPointerArray(p)[i], ElTypeInfo, dimCnt, lengthVec); Inc(i); end; except // Free arrays on exception for j := 0 to i do _DynArrayClear(PPointerArray(p)[j], ElTypeInfo); _DynArrayClear(p, typeInfo); raise; end; end; //--------------------------------- var LArray: array of array of Integer; begin SetLength(LArray, 100,10); SetLength(LArray, 100,10); // DynArraySetLength x100 end; 3 - The function can ruin the cache ! because it dereferences some rtti data and it relies on MM to check for the block size. If the check was implemented inside the function than we won't dereference any unnecessary data. 4 - Adding a simple check to compare old vs new length will be much better and avoids all the issues above. I think that you should fire a case.

The last time when I tried it, it was far far than any other AV. Good to hear that you're having a great experience with it.

I use KIS and sometime I do got some issue (such yours) ... But usually all works great when I edit the trust list. Make sure you're running it as Admin.