dummzeuch 1517 Posted November 28, 2018 (edited) When searching for "windows fastest way to check if a file exists" I get lots of hits that claim there is no API for it and one should use GetFileAttributes or FindFirst instead. But there is PathFileExistsA which according to the description does exactly that: Check if a file exists. It has been around since Windows 2000. Â Yet, FileExists in System.Sysutils still uses GetFileAttributes. Â Am I missing something? Is maybe GetFileAttributes actually faster than PathFileExistsA ? Edited November 28, 2018 by dummzeuch 1 Share this post Link to post
Primož Gabrijelčič 223 Posted November 28, 2018 Why don't you test and tell us? 🙂  (I have no idea but it would be good to know.) 1 Share this post Link to post
Arnaud Bouchez 407 Posted November 28, 2018 FindFirst is definitively the slowest. It needs at least two calls (+ FindClose), and maintain some state in-between.  GetFileAtributes() is the fastest under Windows, and fpaccess/euidaccess on POSIX systems. Note that a fallback to FileAge() is needed in case of sharing violation:  function FileExists(const FileName: string): Boolean; {$IFDEF MSWINDOWS} // use GetFileAttributes: much faster than original, which uses FileAge=FindFirst var Attr: Integer;   LastError: Cardinal; begin   Attr := Integer(GetFileAttributesW(pointer(FileName)));  if Attr <> -1 then   Result := Attr and FILE_ATTRIBUTE_DIRECTORY = 0 else begin   LastError := GetLastError;   Result := (LastError <> ERROR_FILE_NOT_FOUND) and        (LastError <> ERROR_PATH_NOT_FOUND) and        (LastError <> ERROR_INVALID_NAME) and        // (use FileAge to test SHARE_EXCLUSIVE files)        ((LastError = ERROR_SHARING_VIOLATION) or (FileAge(FileName)<>-1));  end; end; {$ENDIF} {$IFDEF LINUX} begin  Result := euidaccess(PChar(FileName), F_OK) = 0; end; {$ENDIF} (extracted from our Enhanced RTL source code)  But the version currently included in Delphi 10.3 Rio SysUtils.pas is just as fast on Windows. On POSIX, it uses stats, which is slower than euidaccess(). 4 2 Share this post Link to post
Tommi Prami 131 Posted November 29, 2018 15 hours ago, Arnaud Bouchez said: Â ...(extracted from our Enhanced RTL source code) Â But the version currently included in Delphi 10.3 Rio SysUtils.pas is just as fast on Windows. On POSIX, it uses stats, which is slower than euidaccess(). Does that work with UNC paths? (Just asking) Share this post Link to post
Arnaud Bouchez 407 Posted November 29, 2018 I don't see any reason why it shouldn't. 1 Share this post Link to post
KodeZwerg 54 Posted November 29, 2018 On Windows i use FileAge(), fastest method i know, hope it helps. Share this post Link to post
KodeZwerg 54 Posted November 29, 2018 Hello again, you could also try with PIDL, i really dont know whats faster, FileAge() or something like that: (not optimized and just for local files...) function SHGetIDListFromPath(Path: TFileName; var ShellFolder: IShellFolder): PItemIDList; var TempPath, NextDir: TFileName; SlashPos: Integer; Folder, subFolder: IShellFolder; PIDL, PIDLbase: PItemIDList; ParseStruct: TStrRet; ParseNAme: String; EList: IEnumIDList; DidGet: Cardinal; ScanParam: Integer; begin SHGetDesktopFolder(Folder); SHGetSpecialFolderLocation(0, CSIDL_DRIVES, PIDLbase); OLECheck(Folder.BindToObject(PIDLbase, nil, IID_IShellFolder, Pointer(SubFolder))); TempPath:=Path; NextDir:=''; while Length(TempPath)>0 do begin SlashPos:=Pos('\', TempPath); if SlashPos > 0 then begin if Pos(':', TempPath) > 0 then NextDir:=Copy(TempPath, 1, 3) else NextDir:=SlashDirName(NextDir)+Copy(TempPath, 1, SlashPos-1); TempPath:=Copy(TempPath, SlashPos+1, Length(TempPath)); end else begin if NextDir='' then NextDir:=TempPath else NextDir:=SlashDirName(NextDir)+TempPath; TempPath:=''; end; PIDL:=PidlBase; ScanParam:=SHCONTF_FOLDERS or SHCONTF_INCLUDEHIDDEN; if (NextDir=Path) and (not DirectoryExists(Path)) then ScanParam:=ScanParam or SHCONTF_NONFOLDERS; if S_OK=SubFolder.EnumObjects(0, ScanParam, EList) then while S_OK=EList.Next(1, pidl, DidGet) do begin OLECheck(SubFolder.GetDisplayNameOf(PIDL, SHGDN_FORPARSING, ParseStruct)); case ParseStruct.uType of STRRET_CSTR: ParseName:=ParseStruct.cStr; STRRET_WSTR: ParseName:=WideCharToString(ParseStruct.pOleStr); STRRET_OFFSET: Parsename:=PChar(DWORD(Pidl)+ParseStruct.uOffset); end; if UpperCase(Parsename)=UpperCase(NextDir) then Break; end else begin Folder:=nil; Result:=nil; Exit; end; if DidGet=0 then begin Folder:=nil; Result:=nil; Exit; end; PIDLBase:=PIDL; Folder:=subFolder; if not FileExists(NextDir) then OLECheck(Folder.BindToObject(Pidl, nil, IID_IShellFolder, Pointer(SubFolder))); end; ShellFolder:=Folder; if ShellFolder=nil then Result:=nil else Result:=PIDL; end; I welcome to see some quality Benchmark Results. Share this post Link to post
dummzeuch 1517 Posted November 29, 2018 FileAge uses GetFileAttributesEx internally, which in my tests was slower than GetFileAttributes which the RTL uses in FileExists. Share this post Link to post
KodeZwerg 54 Posted November 29, 2018 uses ActiveX, Shlobj, IOUtils; function TestFileExists( Filename: String ): Boolean; begin Result := FileExists( Filename ); end; type TParseDisplayName = function(pszPath: PWideChar; pbc: IBindCtx; var pidl: PItemIDList; sfgaoIn: ULong; var psfgaoOut: ULong): HResult; stdcall; var SHParseDisplayName: TParseDisplayName; SHELL32DLLHandle : THandle; function TestPIDL( Filename: String ): PItemIdList; var PIDL: PItemIdList; Attrs: DWORD; begin Result := nil; try CoInitialize(nil); if ( SHParseDisplayName( PChar( Filename ), nil, PIDL, 0, Attrs ) = S_OK ) then if Assigned( PIDL ) then Result := PIDL; finally CoUnInitialize(); end; end; function TestCreateFile( Filename: String ): Boolean; var hFile : THandle; begin Result := False; hFile := CreateFile(PChar( Filename ), GENERIC_READ, FILE_SHARE_READ, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0); if hFile <> INVALID_HANDLE_VALUE then begin Result := True; CloseHandle(hFile); end; end; function TestGetFileAtrributes( Filename: String ): Boolean; var i: Cardinal; begin Result := False; i := GetFileAttributes( PChar( Filename ) ); if i <> INVALID_FILE_ATTRIBUTES then begin Result := True; end; end; function TestFileAge( Filename: String ): Boolean; var DT: TDateTime; begin Result := False; if FileAge( Filename, DT ) then begin Result := True; end; end; function TestFileGetAttr( Filename: String ): Boolean; begin Result := False; if FileGetAttr( Filename ) <> 0 then begin Result := True; end; end; function TestTFile( Filename: String ): Boolean; begin Result := False; if TFile.Exists( Filename ) then begin Result := True; end; end; function TestFindFirst( Filename: String ): Boolean; var sr: TSearchRec; begin Result := False; if FindFirst( Filename, faAnyFile, sr ) = 0 then begin Result := True; end; FindClose(sr); end; procedure TForm1.btnDoJobClick(Sender: TObject); var Start, Stop, Frequency: Int64; Filename: String; i: Integer; Max: Integer; begin Filename := edFilename.Text; try Max := StrToInt( edLoops.Text ); except Max := 1000; edLoops.Text := IntToStr( Max ); end; Memo1.Clear; (* if FileExists( Filename ) then Memo1.Lines.Add( 'File located/cached, begin testing.' ) else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Exit; end; *) Memo1.Lines.Add( 'Begin Test #1: FileExists (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFileExists( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #2: PIDL (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if ( TestPIDL( Filename ) <> nil ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #3: CreateFile (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestCreateFile( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #4: GetFileAtrributes (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestGetFileAtrributes( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #5: FileAge (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFileAge( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #6: FileGetAttr (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFileGetAttr( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #7: TFile (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestTFile( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #8: FindFirst (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFindFirst( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( '' ); Memo1.Lines.Add( 'Job Done.' ); end; I build a small quick bencher, to me GetFileAttributes() is the winner and PItemIdList is by far biggest looser (if i implemented correct way.....) FindFile.7z 1 2 Share this post Link to post
KodeZwerg 54 Posted November 30, 2018 Please remove my crappy PIDL try, it cause MemoryLeak. Share this post Link to post
dummzeuch 1517 Posted November 30, 2018 2 hours ago, KodeZwerg said: Please remove my crappy PIDL try, it cause MemoryLeak. In the first 24h you can edit the post and remove the text yourself, but you cannot remove the post. You should use the "Report Post" function for that, so one of the admins gets notified who can do that. Â Share this post Link to post
KodeZwerg 54 Posted November 30, 2018 17 hours ago, KodeZwerg said: open/closeuses ActiveX, Shlobj, IOUtils; function TestFileExists( Filename: String ): Boolean; begin Result := FileExists( Filename ); end; type TParseDisplayName = function(pszPath: PWideChar; pbc: IBindCtx; var pidl: PItemIDList; sfgaoIn: ULong; var psfgaoOut: ULong): HResult; stdcall; var SHParseDisplayName: TParseDisplayName; SHELL32DLLHandle : THandle; function TestPIDL( Filename: String ): PItemIdList; var PIDL: PItemIdList; Attrs: DWORD; begin Result := nil; try CoInitialize(nil); if ( SHParseDisplayName( PChar( Filename ), nil, PIDL, 0, Attrs ) = S_OK ) then if Assigned( PIDL ) then Result := PIDL; finally CoUnInitialize(); end; end; function TestCreateFile( Filename: String ): Boolean; var hFile : THandle; begin Result := False; hFile := CreateFile(PChar( Filename ), GENERIC_READ, FILE_SHARE_READ, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0); if hFile <> INVALID_HANDLE_VALUE then begin Result := True; CloseHandle(hFile); end; end; function TestGetFileAtrributes( Filename: String ): Boolean; var i: Cardinal; begin Result := False; i := GetFileAttributes( PChar( Filename ) ); if i <> INVALID_FILE_ATTRIBUTES then begin Result := True; end; end; function TestFileAge( Filename: String ): Boolean; var DT: TDateTime; begin Result := False; if FileAge( Filename, DT ) then begin Result := True; end; end; function TestFileGetAttr( Filename: String ): Boolean; begin Result := False; if FileGetAttr( Filename ) <> 0 then begin Result := True; end; end; function TestTFile( Filename: String ): Boolean; begin Result := False; if TFile.Exists( Filename ) then begin Result := True; end; end; function TestFindFirst( Filename: String ): Boolean; var sr: TSearchRec; begin Result := False; if FindFirst( Filename, faAnyFile, sr ) = 0 then begin Result := True; end; FindClose(sr); end; procedure TForm1.btnDoJobClick(Sender: TObject); var Start, Stop, Frequency: Int64; Filename: String; i: Integer; Max: Integer; begin Filename := edFilename.Text; try Max := StrToInt( edLoops.Text ); except Max := 1000; edLoops.Text := IntToStr( Max ); end; Memo1.Clear; (* if FileExists( Filename ) then Memo1.Lines.Add( 'File located/cached, begin testing.' ) else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Exit; end; *) Memo1.Lines.Add( 'Begin Test #1: FileExists (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFileExists( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #2: PIDL (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if ( TestPIDL( Filename ) <> nil ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #3: CreateFile (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestCreateFile( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #4: GetFileAtrributes (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestGetFileAtrributes( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #5: FileAge (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFileAge( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #6: FileGetAttr (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFileGetAttr( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #7: TFile (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestTFile( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( 'Begin Test #8: FindFirst (' + IntToStr( Max ) + ' repeats)' ); QueryPerformanceFrequency(Frequency); QueryPerformanceCounter(Start); for i := 0 to Max do if TestFindFirst( Filename ) then else begin Memo1.Lines.Add( 'File not found. Test canceled.' ); Break; end; QueryPerformanceCounter(Stop); Memo1.Lines.Add( 'Results for ' + IntToStr( Max ) + ' repeats: ' + FormatFloat('0.00', ( Stop - Start ) * 1000 / Frequency) + ' Milliseconds' ); Memo1.Lines.Add( '' ); Memo1.Lines.Add( 'Job Done.' ); end; I build a small quick bencher, to me GetFileAttributes() is the winner and PItemIdList is by far biggest looser (if i implemented correct way.....) FindFile.7z Please exclude TestPIDL method out of DoJob method, it produce MemoryLeak and is slowest methode anyway. Share this post Link to post
Remy Lebeau 1436 Posted February 8, 2019 Why is GetFileAttributes the way old-timers test file existence? 1 3 Share this post Link to post
Holger Flick 12 Posted March 25, 2019 On 11/28/2018 at 6:09 AM, dummzeuch said: Am I missing something? Is maybe GetFileAttributes actually faster than PathFileExistsA ? There is the new unit IOUtils. What does TFile, TPath, or TDirectory use? Share this post Link to post
David Heffernan 2353 Posted March 25, 2019 3 hours ago, Holger Flick said: There is the new unit IOUtils. What does TFile, TPath, or TDirectory use? You can read the code to find out 1 Share this post Link to post
dummzeuch 1517 Posted March 25, 2019 10 hours ago, Holger Flick said: There is the new unit IOUtils. What does TFile, TPath, or TDirectory use? They simply call SysUtils.FileExists. Share this post Link to post
Holger Flick 12 Posted March 26, 2019 On 2/7/2019 at 8:24 PM, Remy Lebeau said: Why is GetFileAttributes the way old-timers test file existence? Using classes or records from a framework is too easy 🙂 1 Share this post Link to post
Uwe Raabe 2064 Posted March 26, 2019 9 minutes ago, Holger Flick said: Using classes or records from a framework is too easy and often more readable. It also has the ability to adapt to different situations or even use a better approach without the need for adjusting a plethora of code. Of course you can wrap that in a function, but that is also somehow using classes or records from a famework. Share this post Link to post
Holger Flick 12 Posted March 26, 2019 Just now, Uwe Raabe said: and often more readable. It also helps with code-compatibility long-term. If MS makes changes to the API, Embarcadero will amend the VCL/RTL accodingly. If you call the API directly, it will not work after that change. You use the framework, it'll be good long-term. 1 1 Share this post Link to post
Stefan Glienke 2019 Posted March 26, 2019 1 hour ago, Holger Flick said: Using classes or records from a framework is too easy Except when the framework is very naively written and adds a lot of unnecessary overhead and indirections. And yes, unfortunately for some situations IOUtils and other RTL units do exactly that. Share this post Link to post
Holger Flick 12 Posted March 26, 2019 6 minutes ago, Stefan Glienke said: Except when the framework is very naively written and adds a lot of unnecessary overhead and indirections. And yes, unfortunately for some situations IOUtils and other RTL units do exactly that. That's why I consider your classes also part of the framework 🙂 Share this post Link to post