Pulling all files and folder names into a list

[JohnLM 14]

Specs: win10, delphi 12, VCL app design

 

I am searching for the best or optimum way of pulling all filenames (and folder names) from my current tablet. The hdd is an SSD. 

 

In the past, I used to use a dos command and run dir c:\*.* /A x y z >h:\cdrive.txt and save that as a text file to archive. But its been a long time since I've performed this and can't remember the exact parameters I used to use. That was windows XP and early windows 7 desktop pc. 

 

Now I want to do the same in a dos window, but I see the command seems to have changed and/or new ones added since. This is windows 10 now. 

 

I would use Delphi command instead but I'm not savy in that area. So I want to continue doing it the dos way if possible, or I would like to try both methods: dos and delphi. 

 

The reason I've used the dos method is because it is the fastest. There was a command to turn off screen updates and it would fly, 1, 2, 3 and it was done. 

 

TIA

[FPiette 383]

To help help for command prompt (There is no more DOS and all), type this:

dir /?

If you want to do it with Delphi code, look at Delphi documentation about finding files.

[JohnLM 14]

I've been at this since I first wrote this topic, but I did not receive a notification of a reply.  Anyway. 

 

I've found the params that I used to use and tried them.  They work okay, but rethinking this project idea I decided to go the delphi route and I've been searching around for old code that I once tried before and had it working then but I can't find it.  It was a project I started about 7 years ago and lost code snippet I was working on.  All I have is the incomplete app.

 

Now I am re-researching old and new code snippets via google and found several so far.  But I've had mixed results with them.  I want to include the recursive portion.  That was the hardest part for me to understand back then, and now I want to try and figure it out again and get working.  In one of the example recursive snippet I found via google, I entered c:\windows as the search path to recursive folder lookup and it took over 15 minutes to complete and give me 126,504 files.  I will do further searches for more efficient code snippets in this area. Next time I will test on a smaller set of recursive folders. 

[Uwe Raabe 2057]

The following code lists 355480 files in less than 15 seconds on my system.

program TestGetFiles;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  System.IOUtils,
  System.Diagnostics;

procedure Main;
begin
  var sw := TStopwatch.StartNew;
  var arr := TDirectory.GetFiles('C:\Windows', '*', TSearchOption.soAllDirectories);
  Writeln(Format('%d Files, %d ms', [Length(arr), sw.ElapsedMilliseconds]));
end;

begin
  try
    Main;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  Readln;
end.

 

The dir command needs a lot of time to display the files, while the Delphi code avoids that.

[DelphiUdIT 176]

2 hours ago, Uwe Raabe said:

The following code lists 355480 files in less than 15 seconds on my system.

..............................

The dir command needs a lot of time to display the files, while the Delphi code avoids that.

The dir command redirect on file (not on screen) stay near 2 seconds to list 141792 files in 14500 directories, the Delphi code stay 339 millisec. ... with Delphi 12 ...

 

Over that was for the cache operations.... without cache the Delphi code and dir command (on file) are exactly the same (1,9 sec.)

 

On Windows directory the code (and dir at the same time) the time was 6.8 sec for 170500 files and 85000 directories.

 

 

Edited November 13, 2023 by DelphiUdIT

[FPiette 383]

3 hours ago, JohnLM said:

I entered c:\windows as the search path to recursive folder lookup and it took over 15 minutes to complete and give me 126,504 files

It takes some time to scan all files and folders recursively but 15 minutes is way to much. One thing that slow down the process is the display on the user interface for each fil found, one by one. You'd better load the list in a TStringList or other structure suitable to your application without displaying anything. And then if required display all files. If you display in a TMemo, use TMemo BeginUpdate/EndUpdate so that the UI is not updated while the lines are added. And note that it is not useful to fill any screen with hundreds of filename but that is another discussion.

[JohnLM 14]

Uwe's code snippet (above), On my win7 laptop (sandybridge i3-2370M 2.40GHz cpu), first run, it was 681,960 files, 418185ms (6969.75 secs) but my cellphone stopwatch

said +/- 7m:30s 

 

Edited November 13, 2023 by JohnLM
added additional text

[JohnLM 14]

I ran the code again, 11.x seconds both stopwatches. By this time it was reading cached memory, but accurate. 

 

Edited November 13, 2023 by JohnLM
typo

[JohnLM 14]

I will attempt to add Uwe's code snippet along with the other snippets I found and tried, into the VCL app I was working on...

[JohnLM 14]

After some mods and additions to the code I managed to get the filenames and folder names to display.  The only problem I am having is with getting the date, mod date, and time. 

 

I believe it is TDirectory.GetCreationDate() 

 

My updated code snippet, below.  I'm sure it can be enhanced better and/or less, but it works. 

 

uses
 strutils;

function padspaceRT(S: string; Len: Integer): string;
begin // this is my custom-made string padder, it insers spaces to a given length - its prob the slowest part and can be enhanced i'm sure. 
  s := midstr(s,1,len);
  result := S; // give it a value right away.
  while Length(Result) < Len do Result := Result + ' ';
end;

procedure tform1.findfiles(list: tstrings);
begin   list.Clear;
  var sw := TStopwatch.StartNew;
  var fname: string;
  var FileARR := TDirectory.GetFiles(eb1.text, '*', TSearchOption.soAllDirectories);
  var DateARR := tdirectory.GetCreationTime(eb1.Text);
  for fname in fileARR do
  begin
    list.Add(padspaceRT(tpath.GetFileName(fname),30) +' -- ' + TPath.GetDirectoryName(fname));
  end;
  list.add(Format('%d Files, %d ms', [Length(FileARR), sw.ElapsedMilliseconds]));
end;

procedure TForm1.btnPullClick(Sender: TObject);
begin
  m1.Lines.Clear; // tmemo 
  m1.Lines.BeginUpdate; // for speed but prob not necessary
  try
    findfiles(m1.Lines);
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  m1.Lines.EndUpdate;
end;

 

Edited November 13, 2023 by JohnLM
fixed typo in code

[JohnLM 14]

I now have the date included.  Again, this is not efficient, and with every new piece I add to the code, it gets slower, but still works. 

 

Below is the updated source code with added functions.  Next, I will add the filename timestamp...

uses
 strutils;

// sec 1
function GetFileCDateTime(FileName: String): TDateTime;
begin
  GetFileCDateTime := 0;
  If (FileName <> '') and (FileExists(FileName) = True) then begin
     GetFileCDateTime := TFile.GetCreationTime(FileName);
  end;
end;

// sec 2
function padspaceRT(S: string; Len: Integer): string;
begin // this is my custom-made string padder, it insers spaces to a given length - its prob the slowest part and can be enhanced i'm sure. 
  s := midstr(s,1,len);
  result := S; // give it a value right away.
  while Length(Result) < Len do Result := Result + ' ';
end;

// sec 3
function padspaceLF(S: string; Len: Integer): string;
begin
  result := S; // give it a value right away.
  while Length(Result) < Len do Result := ' ' + Result;
end;

// sec 4
procedure tform1.findfiles(list: tstrings);
begin
  list.Clear;
  var sw := TStopwatch.StartNew;
  var fname, fdate, ftime: string;
  var FileARR := TDirectory.GetFiles(eb1.text, '*', TSearchOption.soAllDirectories);
  var DateARR := tdirectory.GetCreationTime(eb1.Text);
  for fname in fileARR do begin
    fdate := datetostr(getfilecdatetime(fname));
    ftime := timetostr(getfilecdatetime(fname));
    list.Add(padspaceLF(fdate,12) + ' ' + padspaceLF(ftime,12) + ' | ' + padspaceRT(tpath.GetFileName(fname),30) +' | ' + TPath.GetDirectoryName(fname));
  end;
  st1.Caption := Format('%d Files, %d ms, (%f s) ', [Length(FileARR), sw.ElapsedMilliseconds, sw.ElapsedMilliseconds / 1000]);
end;

// sec 5
procedure TForm1.btnPullClick(Sender: TObject);
begin
  m1.Lines.Clear; // tmemo 
  m1.Lines.BeginUpdate; // for speed but prob not necessary
  try
    findfiles(m1.Lines);
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  m1.Lines.EndUpdate;
end;

 

Edited November 14, 2023 by JohnLM
added missing word, 'add', later, added updated code in // sec 4

[JohnLM 14]

I have updated the source code to include the time info (in the sec 4 portions of the source code, in the previous post above). 

 

Here is a demo of the final output: 

 

   1/27/2016  12:11:27 AM | Project2.cfg                   | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Project2.dof                   | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Project2.dpr                   | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Project2.exe                   | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Project2.res                   | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Unit1.dcu                      | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Unit1.ddp                      | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Unit1.dfm                      | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:27 AM | Unit1.pas                      | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:28 AM | Unit1.~ddp                     | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:28 AM | Unit1.~dfm                     | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:28 AM | Unit1.~pas                     | I:\delphi\d7\Activity List\ver01
   1/27/2016  12:11:32 AM | Project1.dof                   | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:32 AM | Project1.dpr                   | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:32 AM | hdd.s.exe                      | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:33 AM | Project1.res                   | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:33 AM | Unit1.dcu                      | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:33 AM | Unit1.dfm                      | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:33 AM | Unit1.pas                      | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:33 AM | Unit1.~dfm                     | I:\delphi\d7\hdd serial\v01
   1/27/2016  12:11:33 AM | Unit1.~pas                     | I:\delphi\d7\hdd serial\v01
   7/10/2016  11:10:03 PM | Unit2.fmx.~1~                  | I:\delphi\xe7\FMX\fishfacts\v01\__history
   7/10/2016  11:10:03 PM | Unit2.vlb                      | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:10:08 PM | Project1.dpr                   | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:10:08 PM | Project1.dproj                 | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:10:24 PM | Unit2.dcu                      | I:\delphi\xe7\FMX\fishfacts\v01\Win32\Debug
   7/10/2016  11:10:24 PM | Project1.exe                   | I:\delphi\xe7\FMX\fishfacts\v01\Win32\Debug
   7/10/2016  11:10:23 PM | Project1.res                   | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:10:03 PM | Unit2.pas                      | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:10:03 PM | Unit2.fmx                      | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:10:08 PM | Project1.dproj.local           | I:\delphi\xe7\FMX\fishfacts\v01
   7/10/2016  11:17:48 PM | Project1.identcache            | I:\delphi\xe7\FMX\fishfacts\v01
   9/16/2017  11:15:18 PM | readme_KEEP.txt                | I:\delphi\xe7\FMX\fishfacts\v01
  11/13/2016   1:00:08 PM | data_Notes20161113sun1259pm.zi | I:\delphi\misc
   1/13/2019   1:47:50 PM | data_Notes20190113sun0139pm.zi | I:\delphi\misc

 

[JohnLM 14]

update..

 

I've also compiled the test app in Delphi 11 and 12 on my Chuwi Hi10X tablet.  11.2 works fine, but 12.0 hit or miss.  Sometimes the machine locks up, other times the window (the memo) is blank. Makes no sense to me, but I'm late for work and will have to figure it out later.