Mahdi Safsafi

7 minutes ago, FredS said:

@Mahdi Safsafi Might be simpler to just redirect System.GetMemory instead of the memory manager's GetMem which is called by other methods doing a reOutOfMemory check themselves.

 

 

 

Indeed you're absolutely right. But the intention of the above solution I proposed was to be a temporary fix that doesn't use any external library. 

When he would have some time, he may need to investigate further for a suitable solution. 

@dummzeuch Yes FastMM can handle it but OS provides much options(ZeroInit, FileMapping, ...). Besides the behavior when OS fails is very well documented.

 Anyway, there is no harm to use FastMM ... after all it will call OS functions. 

15 minutes ago, dummzeuch said:

Great! 😞

Now I have to go through the whole program and change the code to either call GetMem or check the result.

or temporary solution :

var
  OriginalMemoryManager: TMemoryManagerEx;
  NewMemoryManager: TMemoryManagerEx;

function NewGetMem(Size: NativeInt): Pointer;
begin
  Result := OriginalMemoryManager.GetMem(Size);
  if not Assigned(Result) then
    raise EOutOfMemory.Create('Error Message');
end;

begin
  GetMemoryManager(OriginalMemoryManager);
  NewMemoryManager := OriginalMemoryManager;
  NewMemoryManager.GetMem := NewGetMem;
  SetMemoryManager(NewMemoryManager);
end.

Why you're using Delphi MM for such large block ? It would be better to use OS functions.

36 minutes ago, Stefan Glienke said:

Currently very much enjoying this feature while running unit tests with a lot of exception checking under the debugger! 👍

Yes it comes very handy and it would be much more powerful when it supports class inheritance check(ignoring all exceptions that descent from a particular class). 

One thing worth to mention, Thomas told me that he managed to make it works on all Delphi versions >= 2005.

@Kas Ob. Well I was referring to the decision. The chance to improve quality is higher when you only focus on one thing.

Quote

Their focus is on Windows, not cross-platform right now.

IMO, this is the right decision. What do you think guys ? 

20 minutes ago, Attila Kovacs said:

btw. in the header and also on the screenshot from Mahdi the first parameters are PCWSTR (LPCWSTR) (const) and not always PWSTR (LPWSTR).

this however makes no difference, at least at the moment, as they are defined the same in delphi 

Good point Attila. It also makes comparing c code against Delphi code much easy. 

44 minutes ago, Attila Kovacs said:

btw. he is using IDA

Not IDA ... its x64dbg 🙂   

28 minutes ago, Kas Ob. said:

Now the question is : How many of those overloaded are in Delphi libraries ? and is this documented behaviour ?
 

I really don't know.

Quote

If you create COM object with overloaded methods with Delphi , then you can't use it from different platform, right? and the worse question if you went ahead and used CBuilder is the interface usable ?

Well I never tried to consume Delphi stuff from c/c++ mostly because c/c++ has a much native frameworks than Delphi ... However I do consume c/c++ stuff from Delphi.

11 minutes ago, pyscripter said:

1. There are many more overloads in the include file implemented as inline functions.   Do those interfere with the above?

Those are not stored inside vtable ... you need to skip them(don't declare them).

Quote

1.  After the first set of Set/GetAttributeValue there is a GetAttributeValueLength function.  Does this comes last?

Exactly! 

That's my bad man ... I'm a little bit bad at explaining since I'm not a native English speaker.

First step, add all methods in the order they declared(skip all overloaded versions but keep the original). second step for each added method "A" add just after it "A" all its overloaded versions in the order they declared too.

Here is how your vtable looks like.

46 minutes ago, pyscripter said:

Thanks, but this is puzzling.  at $7C is the GetAttributeValue, which I know it works  and  at $78 should be the one before. (right?).  In the include file the function before the GetAttributeValue is the relevant SetAttributeValue.   How can I find the correct order?

Basically vtable respects the order in which functions are declared. But here the overloading is tweaking. Try to respect both the order and overloading name. The below code should demonstrate what I mean.

// SetAttributeValue declared first ... we put it first
function SetAttributeValue(name: LPWSTR; _type: D2D1_SVG_ATTRIBUTE_STRING_TYPE; value: LPWSTR): HResult; overload; stdcall;
// then we put all overloaded methods SetAttributeValue but in the order they were declared:
function SetAttributeValue(name: LPWSTR; _type: D2D1_SVG_ATTRIBUTE_POD_TYPE; value: Pointer; valueSizeInBytes: UINT32): HResult; overload; stdcall; // SetAttributeValue2
function SetAttributeValue(name: LPWSTR; value: ID2D1SvgAttribute): HResult; overload; stdcall; // SetAttributeValue3
// GetAttributeValue comes after  SetAttributeValue:
function GetAttributeValue(name: LPWSTR; _type: D2D1_SVG_ATTRIBUTE_STRING_TYPE; out value: PWideChar; valueCount: UINT32): HResult; overload; stdcall; 
// overloaded2 first and then overloaded3
function GetAttributeValue(name: LPWSTR; _type: D2D1_SVG_ATTRIBUTE_POD_TYPE; value: Pointer; valueSizeInBytes: UINT32): HResult; overload; stdcall; // overloaded2
function GetAttributeValue(name: LPWSTR; const riid: TGUID; var value: Pointer): HResult; overload; stdcall;  // overloaded3

PS: I got the smile SVG 🙂 

@pyscripter

Quote

I did check that the correct overload is called. (looked at the assembly code calliing the function: call dword ptr [eax+$7c] for GetAttributeValue  , dword ptr [eax+$78] for SetAttributeValue  , tripled checked with the C header file order of interface members etc).

I believe you're wrong on $78 ! its for an overloaded GetAttributeValue function (not SetAttributeValue). So your code is calling an overloaded version of GetAttributeValue instead of SetAttributeValue.

3 hours ago, Arnaud Bouchez said:

Such abstract/broad input about non Delphi compilers has nothing to do with the initial question of this thread, which is performance of string process with Delphi. It is clearly out of scope and subject.

Yes you're absolutely right ... I don't really know how the post ended like this. Anyway, it was really nice to have this little discussion with you. Thanks for your time and the valuable information/material you provide.

1 hour ago, Mike Torrettinni said:

Does that mean you need a trial and error to get to know all these little 'secret' tips how to write best possible optimized code, for each compiler?

Secret ? no, they're publicly published (agner, intel/amd doc, optimization guide, paper, LLVM, ...). I read them and experiment things my self and when I have no things to do I compare results on different compilers. Sometime, when I'm lucky, I find an already benchmark that lists the results.

Error happens a lot specially when porting code from/to another platform ... but I deeply inspects the error and try to learn from it : why it happened ? a workaround ? how to avoid it?.

2 hours ago, Arnaud Bouchez said:

@Mahdi Safsafi

My experiment with the Delphi compiler and pointers is not the same as yours. In mORMot, I ended up with using pointers (PUTF8Char=^AnsiChar) for the raw UTF-8 processing, which generated faster code.

Yes, I use several compilers (not limited to Delphi) and I can sadly tell you that Pascal compilers didn't evolved in a good way like other compilers.

Quote

Anyone don't like pointers? Don't use them. But they are pretty convenient and efficient for low-level processing, if the goal is performance.

Its not about whether you like them or not. In fact for me, I don't find any issue using low level stuff pointer or assembly. But today, I try to be much wise and choose the best friendly way for the compiler. What are you missing is that today sophisticated compiler in many case can do better job than you can do by using low level stuff like assembly or pointer. gcc for example is able to recognize a pattern (while do) that calculates bits count and replace this loop by a single instruction. It can reorder things for you when it detects that you're misusing some functionality:  see this SO. A compiler could generate much better code only when it can understood your code. If you manage to write a hard code that your compiler can't understand it wan't be optimized correctly.

Here is a simple example to show what would happen when the compiler does not understand your code: both swap/swap2 function do the same job but one is decrementing i and the other one is incrementing i. and both log the result using a for loop. The interesting part is that unlike swap, the two loop of swap2 were merged together

__declspec(noinline)

void swap(short int* src, int* dst,  int count) {
	short int* p = src;
	int* r = dst;
	int i = count;
	// loop 1
	while (i--) {
		*r++ = swapint(*p++);
	}
    // loop 2
	for (int i = 0; i < count; i++) printf("%d = %d\n", i, dst[i]);
	return ;
}
__declspec(noinline)
void swap2(short int* src, int* dst, int count) {
	short int* p = src;
	int* r = dst;
	int i = count;
	// compiler merged/combined the two loop !                         
	while (i++ != count) {
		*r++ = swapint(*p++);
	}
	for (int i = 0; i < count; i++) printf("%d = %d\n", i, dst[i]);
	return;
}

int main()
{
	short int src[10] = { 1,2,3,4,5 };
	int dst[10] = { 0 };
	
	swap(&src[0], &dst[0], 10);
	swap2(&src[0], &dst[0], 10);
	return 0;
}

1 hour ago, dummzeuch said:

One question:

I understand why I should protect access to FDebugEvent for multithreading. What I don't get is why a stack should be necessary. Could you please give an example?

WaitForDebugEvent function is called from the debug loop thread but DoShowException function is called from another thread. A fraction of eye before DoShowException is called, WaitForDebugEvent may receive a debug event exception which may override FDebugEvent variable and you may end up processing the new exception and missing the old one. Using a list of FDebugEvent will guarantee that for each exception, you have a valid associated FDebugEvent struct.

I don't know if there is a better way to work around without using a list of FDebugEvent. If you find a better way, please let me know.

Apart from that, are you making any progress ? did the non-WaitForDebugEvent-method worked for all new Delphi versions ? 

13 hours ago, Remy Lebeau said:

Be sure to file these issues to Quality Portal so Embarcadero sees them for future updates.

 

I think that the right thing is to report it as a false positive at BD. 

@pyscripter I just made a benchmark against Perl regex. Yes its weird to compare a compiler vs interpreter ... but Perl beats Delphi own & study regex. However it failed to bit jit regex. A script and detailed result are attached.

Benchmark.rar

@pyscripter Here is the x86 function.

But its really weird, the original version shipped with Delphi declares it as empty and it works. I don't understand why it crashed with you when you defined it as empty.

@Kas Ob's point is very interesting. I didn't read the configuration's documentation. But he definitely did ... Perhaps he is right. I suggest that you check with him that point. And who knows ... you may not need to use chkstk.

chkstk.txt

2 minutes ago, Kas Ob. said:

@pyscripter /Gs100000000 did remove the need for chkstk for both 32 and 64 bit, as i see on my VS 2017, now i am not sure why the protection should be needed here as the parameters passed in stack are only to 2 and to hit the stack limits, it should be recursively called 128000 times on 32bit 1mb stack, so may be you should consider remove it.

 

Now on other hand you can simply disable recursion to begin with

SET(PCRE_NO_RECURSE OFF CACHE BOOL
    "If ON, then don't use stack recursion when matching. See NO_RECURSE in config.h.in for details.")

then use that 

/Gs100000000 

to remove chkstk call

Wow ... that's a 100mb ! 

4 minutes ago, pyscripter said:

There are no conversions needed for Win64.

Yeah I know that, I early proposed to you to use MSVC for x64 and bcc32c for x86.

Quote

• Saddly, I have more trust in the Visual Studio compiler, especially for code that requires stack checking.  Does bcc32c do that?

Sadly bcc32c (at least the free version) is build on an old LLVM compiler. 

Quote

 When I finish I will release the stuff at github, so people can take a look and make adjustments.   Embarcadero can use their own compiler if they wish.

Good luck ! if you need some help let me know. 

Quote

That works and produces the correct results!! Thanks.  So win64 is complete and I will now try to get win32 right. 

 

From what I understood from your comment, you're going to compile using MSVC and then use some tools to convert the output ? why not give bcc32c a chance ? if it does not work ... you can simply use plan B. 

@pyscripter 

Anyway I compiled (using CLANG) a simple c code that I tweaked to use chkstk. and then I dumped the function and edited it to be compatible with Delphi :

procedure chkstk();
asm
  .NOFRAME
  sub rsp, $10
  mov [rsp], r10
  mov [rsp+8], r11
  xor r11,r11
  lea r10, [rsp+$18]
  sub r10,rax
  cmovb r10,r11
  mov r11, qword ptr gs:[$10]
  cmp r10,r11
  db $f2
  jae @@L1
  and r10w,$F000
@@L2:
  lea r11, [r11-$1000]
  mov byte [r11],0
  cmp r10,r11
  db $f2
  jne @@L2
@@L1:
  mov r10, [rsp]
  mov r11, [rsp+8]
  add rsp, $10
  db $f2
  ret
end;

EDIT: I didn't pay attention to the GAS syntax for branch and interpreted it as hex ... My bad.

7 minutes ago, pyscripter said:

Could anyone help translate the LLMV code to Delphi PLEASE?  The prize is that everyone gets a much faster System.RegularExpressions and we may even convince Embarcadero to implement the changes. 

I'd help ... I'm just checking other options.

Yes disabling stack check will not suppress ___chkstk_ms. Only /GsX will do provided that this option is supported by your compiler and X is higher than probe size. 

Now. are you sure that the crash comes from ___chkstk_ms ? also have you disabled other runtime check ?