dummzeuch

function TMemoryStream.Write(const Buffer; Count: Longint): Longint; var Pos: Int64; begin if (FPosition >= 0) and (Count >= 0) then begin Pos := FPosition + Count; if Pos > 0 then begin if Pos > FSize then begin if Pos > FCapacity then SetCapacity(Pos); FSize := Pos; end; System.Move(Buffer, (PByte(FMemory) + FPosition)^, Count); FPosition := Pos; Result := Count; Exit; end; end; Result := 0; end; This is TMemoryStream.Write from System.Classes (which apart from using Int64 now seems to be unchanged since at least Delphi 2007). It looks rather inefficient to me: It checks for Count >= 0 while in my opinion it should check for Count > 0 because otherwise nothing gets written anyway and Result gets set to 0 (Remember Count is 0 in that case) which is done after the if statement anyway. If it did check for Count > 0 it could simply omit the Pos > 0 check because we already know that it must be because FPosition >= 0 and Count > 0, so FPosition + Count must also be > 0. So, my implementation would look like this: function TMemoryStream.Write(const Buffer; Count: Integer): Longint; var Pos: Int64; begin if (FPosition < 0) or (Count <= 0) then begin Result := 0; Exit; //==> end; Pos := FPosition + Count; if Pos > FSize then begin if Pos > FCapacity then SetCapacity(Pos); FSize := Pos; end; System.Move(Buffer, Pointer(Longint(FMemory) + FPosition)^, Count); FPosition := Pos; Result := Count; end; Am I missing something? Another oddity is that the GetSize method is not overridden to simply return FSize rather than calling Seek three times. Given that there are quite a few places in the RTL / VCL (and also JCL / JVCL, Indy and of course my own code) which use TMemoryStream as a convenient way to move around some bytes, it's odd that it still has these inefficiencies.

Judging from the VERxxx constants in the package source, SynEdit2 supports Delphi 2006 and up, correct? Could you please add that to the project description? @pyscripter

Quite possible. But on the other hand I had not yet arrived at the optimization step. I was still looking at the code and tried to understand what it was doing and why. And of course there is always the "I'm doing this because it's fun" part of optimizing. Btw: The best optimization for the code I actually care about (it's about writing JPEGs and text to an MJPEG video) probably is not using TMemoryStream at all but a simple byte buffer instead. That stream just was a convenience there.

Longint = integer since basically forever.

OK, so I understand that there might be the possibility of an Int64 overflow, here: which could result in Pos becoming negative. On the other hand that would mean a stream with about MaxInt64 bytes of data, which is quite a lot: 9,223,372,036,854,775,807 or 2^33 gibibytes. Do 64 bit processors actually have the capacity of addressing that much (virtual) memory? Are there any other bugs in my implementation? There might also be no performance gain at all due to highly sophisticated features of the CPU. And of course: There might be other possible improvements If I want this to be optimized I should submit a report to Embarcadero Why did I actually bring this up? I was looking at some code that was extensively using TMemoryStream as a buffer for various operations, where it accessed the Position and Size properties all the time and thought about the possible overhead of these method calls. Then I looked at the implementation of GetSize and found 3 calls to Seek in there, just to eventually get the value of a field. And then I looked at various other methods of T(Custom)MemoryStream and thought WTF is this? That I used the Write method as an example above was pure chance, it was just the last method I had been looking at.

In ISO 9001 consistency is everything. You must have rules and follow them. It doesn't matter whether the result is correct or not, it must reproducible every single time. (OK, I'll stop now, I just have suffered too much pain due to this idiocy.)

No, there is no way to change that order. But there is the configurable Editor Popup Menu (in Editor Experts), which might help you there.

Given a large directory tree with many files and sub directories and a depth of about 5 located on a Samba server share which must be fully traversed: What is more efficient: Depth First Search: Recursive FindFirst/Next which immediately handles each sub directory when it is encountered. Breadth First Search: Iterative FindFirst/Next which maintains a list of directories to process and appends sub directories to this list when they are encountered to be processed later. I have so far implemented DFS, simply because it was the easiest. It takes a very long time (about 60 seconds in my test scenario). Before I try to implement BFS: Has anybody ever compared the performance of these approaches on Windows + network shares? I googled, but could not find anything definitive (which really surprised me), maybe my google fu has let me down again.

I did some more tests: On Windows 10 1909 on a physical computer and a local drive I found that it makes no difference whether I use FindFirstFileEx or FindFirstFile. On Windows 10 1909 on a Samba share (same as in the first tests) there is still an advantage of using FindFirstFileEx, but it isn't as big as in my tests on Windows 8.1. Since the contents of the drive have changed in the meantime and the Windows 10 installation is on a virtual machine while the Windows 8.1 installation is on a physical computer I can't meaningfully compare them to the original tests.

I'm happy to report that the problem with the icons in the popup menus causing empty entries seems to have been fixed in Delphi 10.4. So I have now enabled them in GExperts again.

It doesn't do anything at all. And if you remove the conditional define testWizard, it doesn't get installed. That item was just a convenient way to test some possible wizard functionality without having to write a new plugin.

The GExperts Rename Components expert has some limited functionality similiar to Quick Edit and it has a keyboard shortcut. https://blog.dummzeuch.de/2018/07/21/selecting-alignment-and-anchors-in-the-rename-components-expert/ Adding a keyboard shortcut for Quick Ediit is on my wish list, but not as easy as I expected. I already wasted several hours on this.

Once I've restored the backup of my data hard disk (which crashed on me last weekend), I will have a look and tell you what it does. Until then, you will either have to look yourself or live with the suspense. 😉

I wonder what that menu item does? 😉 Acutally I have no idea, it's probably a leftover from some testing code which I forgot to remove again. This is the first time I heard from somebody else who actually used that plugin.

I have seen this error several times already: I close a project and get the error in the title. Has anybody else experienced this?

Hey, give us something to laugh! Would make for a nice change for me right now.

Since I have no idea which functionality of the IDE FixInsight might hook (or even which events or maybe what else it might in stall), I have no idea where to look. I know that there are still some issues with GExperts crashing when exiting the IDE but none of them so far was reproducible enough to track down the issue. And as David Heffernan said: It could be anything. There aren't many safe guards inside the IDE to separate itself from plugins and plugins from each other.

GExperts has an option to automatically close it on successful compile. Maybe you have enabled it? (Default is disabled though.)

I revisited my dzBdsLauncher tool again – no idea why, it just occurred to me 😉 – and added quite a few improvements: It now supports .dof (Delphi 6 and 7) and .bdsproj (Delphi 2005 and 2006) files. In addition to the previous checks it now also looks at the disabled packages list to determine which Delphi version to start. That’s the only option for Delphi 2005 and 2006 because these files are nearly identical. It can now also handle .dpr files by looking for corresponding .dproj, .bdsproj and .dof files (in that order) and taking these to determine the correct Delphi version. As a side effect I found a problem with the Delphi 10.1 version of the GExperts .dproj file. It had a wrong ProjectVersion entry. The blog post with download link is here.

I must be doing something wrong here: var Stopwatch: TStopwatch; begin Stopwatch.Start; sleep(3000); Stopwatch.Stop; m_Result.Lines.Add(format('Took %.3f seconds', [Stopwatch.Elapsed.TotalSeconds])); end; The result is "-130847749795,796 seconds", I was, of course, expecting something around 3. What am I missing?

One of the bad habits I have picked up over the years is not looking into the documentation. The only reason I can give is that the documentation was so bad for some time that the time it took usually was wasted. Since documentation has become much better (but is still far from perfect, especially some of the examples), this no longer applies though.

What i don't need are snide remarks like this.

Conclusion: It doesn't matter whether I use Breadth First or Depth First Search, the performance is the same. Switching from SysUtils.FindFirst/FindNext to Windows.FindFirstFileEx/FindNextFile with only basic file information and the flag FIND_FIRST_EX_LARGE_FETCH brought me a performance gain of about 25% which is not too bad. I found no way to benefit from using TDirectory. The only test I made got worse performance than FindFirst/Next. Test environment: Windows 8.1 client accessing a share on a Samba server over 1 GBit Ethernet with minimal traffic (it's a weekend). The computer is a reasonably fast Intel Xeon E3 with 3.4 GHz with 8 cores and 16 GByte RAM. The Server has an Intel Core I9-9940X processor with 16 cores and with 64 GBytes of RAM. The share is stored on a 2 TB Intel M2 NVMe SSD. It's running Ubuntu 18.04.4 LTS and Samba 4.7.6-Ubuntu. The test program was compiled with Delphi 10.4. There was a no noticable difference between building with Debug and Release config. The directory tree is 4 levels deep and contains only a few directories on the first 3 levels but a total of 898 directories on the deepest level. Each of these directories on the deepest level contain from several hundred up to several thousand jpeg files. The purpose of the code is to find all directories on any level containing at least one jpeg file but no subdirectories. All tests were run several times in varying order to prevent any caching effects to distort the result.

As promised, here the BFS version using FindFirstFileEx. And as expected, it also takes about 45 seconds in my test setup. function FindFirstFileEx(lpFileName: LPWSTR; fInfoLevelId: TFindexInfoLevels; lpFindFileData: Pointer; fSearchOp: TFindexSearchOps; lpSearchFilter: PWin32FindData; dwAdditionalFlags: DWORD): THandle; stdcall; external kernelbase Name 'FindFirstFileExW'; function FindNextFile(hFindFile: THandle; lpFindFileData: PWin32FindData): BOOL; stdcall; external kernelbase Name 'FindNextFileW'; procedure TForm1.b_FindFirstExNextBFSClick(Sender: TObject); var DirsWithJpg: TArray<string>; DirsToSearch: TStringList; procedure CheckDirectory(const _Dir: string); const FIND_FIRST_EX_LARGE_FETCH = 2; var DirBs: string; SearchHandle: THandle; sr: TWin32FindData; ContainsFiles: boolean; ContainsSubdirs: boolean; fn: string; begin ContainsFiles := False; ContainsSubdirs := False; DirBs := IncludeTrailingPathDelimiter(_Dir); SearchHandle := FindFirstFileEx(PChar(DirBs + '*.*'), FindExInfoBasic, @sr, FindExSearchNameMatch, nil, FIND_FIRST_EX_LARGE_FETCH); if SearchHandle <> INVALID_HANDLE_VALUE then begin try repeat fn := sr.cFileName; if (fn = '.') or (fn = '..') then begin // ignore end else if (sr.dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) <> 0 then begin // directory ContainsSubdirs := true; // add to list for later processing DirsToSearch.Add(DirBs + fn); end else if sr.dwFileAttributes and (FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM or FILE_ATTRIBUTE_REPARSE_POINT) = 0 then begin // regular file if not ContainsFiles then begin if SameText(ExtractFileExt(fn), '.jpg') then begin ContainsFiles := true; end; end; end else begin // ignore special files end; until not FindNextFile(SearchHandle, @sr); finally Winapi.Windows.FindClose(SearchHandle); end; end; if ContainsFiles then begin if ContainsSubdirs then begin m_Result.Lines.Add(Format('Directory %s contains files and subdirectories, ignoring it.', [_Dir])); end else begin DirsWithJpg := DirsWithJpg + [_Dir]; end; end; end; var Stopwatch: TStopwatch; begin Stopwatch.Reset; Stopwatch.Start; DirsToSearch := TStringList.Create; try DirsToSearch.Add('\\server\share\dir'); while DirsToSearch.count > 0 do begin CheckDirectory(DirsToSearch[0]); DirsToSearch.Delete(0); end; finally FreeAndNil(DirsToSearch); end; Stopwatch.Stop; m_Result.Lines.Add(Format('FindFirstExNext BFS: Found %d dirs in %.3f seconds', [Length(DirsWithJpg), Stopwatch.Elapsed.TotalSeconds])); end;

OK, so here is the DFS implementation using FindFirstFileEx and only getting basic info. In my tests on the same computer and with the same Samba server and share it only takes about 45 seconds (rather than the 60 seconds with SysUtils.FindFirst). The result is the same. For completeness I am going to implement BFS for this too, but I don't expect any change really. function FindFirstFileEx(lpFileName: LPWSTR; fInfoLevelId: TFindexInfoLevels; lpFindFileData: Pointer; fSearchOp: TFindexSearchOps; lpSearchFilter: PWin32FindData; dwAdditionalFlags: DWORD): THandle; stdcall; external kernelbase Name 'FindFirstFileExW'; function FindNextFile(hFindFile: THandle; lpFindFileData: PWin32FindData): BOOL; stdcall; external kernelbase Name 'FindNextFileW'; procedure TForm1.b_FindFirstExNextDFSClick(Sender: TObject); var DirsWithJpg: TArray<string>; procedure CheckDirectory(const _Dir: string); const FIND_FIRST_EX_LARGE_FETCH = 2; var DirBs: string; SearchHandle: THandle; sr: TWin32FindData; ContainsFiles: boolean; ContainsSubdirs: boolean; fn: string; begin ContainsFiles := False; ContainsSubdirs := False; DirBs := IncludeTrailingPathDelimiter(_Dir); SearchHandle := FindFirstFileEx(PChar(DirBs + '*.*'), FindExInfoBasic, @sr, FindExSearchNameMatch, nil, FIND_FIRST_EX_LARGE_FETCH); if SearchHandle <> INVALID_HANDLE_VALUE then begin try repeat fn := sr.cFileName; if (fn = '.') or (fn = '..') then begin // ignore end else if (sr.dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) <> 0 then begin // directory ContainsSubdirs := true; // recursive Depth First Search CheckDirectory(DirBs + fn); end else if sr.dwFileAttributes and (FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM or FILE_ATTRIBUTE_REPARSE_POINT) = 0 then begin // regular file if not ContainsFiles then begin if SameText(ExtractFileExt(fn), '.jpg') then begin ContainsFiles := true; end; end; end else begin // ignore special files end; until not FindNextFile(SearchHandle, @sr); finally Winapi.Windows.FindClose(SearchHandle); end; end; if ContainsFiles then begin if ContainsSubdirs then begin m_Result.Lines.Add(Format('Directory %s contains files and subdirectories, ignoring it.', [_Dir])); end else begin DirsWithJpg := DirsWithJpg + [_Dir]; end; end; end; var Stopwatch: TStopwatch; begin Stopwatch.Reset; Stopwatch.Start; CheckDirectory('\\server\share\dir); Stopwatch.Stop; m_Result.Lines.Add(Format('FindFirstExNext DFS: Found %d dirs in %.3f seconds', [Length(DirsWithJpg), Stopwatch.Elapsed.TotalSeconds])); end; I redeclared FindFirstFileEx and FindNextFile for this because the declarations in WinApi.Windows are inconsistent in the declaration of the lpfFindFileData parameter. Now both declare it as PWin32FindData.