TDirectory.GetFiles is listing the contents of Packages such as PhotosLibrary

[Incus J 10]

I'm using System.IOUtils TDirectory.GetFiles function to list files (and recurse into subfolders) on macOS.

 

Trouble is, in addition to normal folders, it is also listing the contents of Packages.  For example, if I ask GetFiles to recursively scan the standard /Pictures folder, the internal contents of the PhotosLibrary.photoslibrary package also gets added to the list.  This sort of makes sense in a way, since on macOS a Package is essentially a special kind of folder:  In Finder I can right-click on a Package and select View Package Contents.

 

Question is though, how can I prevent GetFiles from looking inside and listing Package contents, and just return a list of the files a user would normally see, instead?

Edited June 29, 2022 by Incus J

[Incus J 10]

Apple's developer documentation mentions:

static var skipsPackageDescendants: FileManager.DirectoryEnumerationOptions
An option to treat packages like files and not descend into their contents.

I think that is the behaviour I'm looking for, but I'm not sure how I can achieve it with Delphi.

 

Perhaps if I can determine which folders/paths are packages ahead of time, I could then use the Filter Predicate function to filter out all files on those paths:

TDirectory.GetFiles(path, TSearchOption.soAllDirectories, MyFilterFunction);

 

 

Edited June 29, 2022 by Incus J

[Remy Lebeau 1393]

4 hours ago, Incus J said:

Apple's developer documentation mentions:

static var skipsPackageDescendants: FileManager.DirectoryEnumerationOptions
An option to treat packages like files and not descend into their contents.

I think that is the behaviour I'm looking for, but I'm not sure how I can achieve it with Delphi.

Unfortunately, I see nothing in Delphi that would allow you to use that flag in this situation.

 

TDirectory.GetFiles() is just a wrapper for a typical SysUtils.FindFirst/Next() loop, so it handles whatever files and directories that loop would normally report.  If a Package file is treated as if it were a directory (ie FindFirst/Next() returns TSearchRec.Attr with the faDirectory flag set), then TDirectory.GetFiles() is going to recurse into the Package's content if the SearchOption parameter is set to soAllDirectories (as it is in your example).

 

Also, more importantly, FindFirst/Next() simply don't use the FileManager API to enumerate files, they use the POSIX opendir() and readdir_r() functions instead, which have no such skip-Package flag available.

Quote

Perhaps if I can determine which folders/paths are packages ahead of time, I could then use the Filter Predicate function to filter out all files on those paths:

TDirectory.GetFiles(path, TSearchOption.soAllDirectories, MyFilterFunction);

 

Unfortunately, the TDirectory.GetFiles() predicate is only called for files and not for directories (ie, only for items that do not have the faDirectory attribute).  Which makes sense, since TDirectory.GetFiles() is looking for only files to add to its output array, so there is no point in it filtering out directories.

 

To prevent TDirectory.GetFiles() from recursing into Packages automatically, you would have to call it with its SearchOption parameter set to soTopDirectoryOnly (which is its default value), and then manually call TDirectory.GetFiles() for any non-Package sub-directory you are interested in.  Of course, TDirectory.GetFiles() won't report directories to you, so you would have to use a separate call to TDirectory.GetDirectories() (with SearchOption=soTopDirectoryOnly) and filter out the Packages as needed.  Which means, your enumeration time is going to double from having to scan each directory twice.

 

So, in this case, you will likely have to use the FileManager API directly, and not use TDirectory at all.

Edited June 29, 2022 by Remy Lebeau

[corneliusdavid 214]

I wrote a quick test app and then drilled down into some folders using the app and compared with the Mac's native "Finder" to look at a set of virtual machines.

 

The Mac's Finder recognized the "Virtual Machines" entry as a folder and its file contents were the same list of virtual machines I can see in VMWare Fusion, each entry is a simple VM name, and the details pane shows when it was created, the bundle size, etc.

 

My test app, using just the TDirectory and TPath classes, showed "Virtual Machines.localized" as a folder name and allowed me to drill into that folder and then again into one of the ".vmwarevm" files, listing all the .vmx, .vmdk, .plist, and other files that make up a virtual machine package.

 

Same with the Applications folder: Finder shows the application icons whereas my app shows a ".app" extension for each entry which is a sub-folder that opens up the contents of the app giving me Info.plist, PkgInfo, etc.

 

My conclusion is that your app will need to recognize the file extension or file type and handle it differently--or like @Remy Lebeau says, use the FileManager API directly.

[Incus J 10]

Thank you Remy, thank you David.  Your test app description confirms my sinking feeling - TDirectory GetDirectories & GetFiles functions are not currently geared up to handling the nuances of the macOS file system.  They would need to become package aware at the very least - perhaps something for a future Delphi release.

 

12 hours ago, Remy Lebeau said:

Unfortunately, the TDirectory.GetFiles() predicate is only called for files and not for directories (ie, only for items that do not have the faDirectory attribute).

 

That in itself might be OK.  I could let GetFiles drill down into packages, but in the predicate reject each file that is on a package path - since each file passed into the predicate will include access to its path.

 

So I would (somehow) build a TStringList of folders that are packages, in a first pass e.g.

/Pictures/PhotosLibrary.photoslibrary

/Pictures/Photo Booth Library

 

Then once I've built that list, call GetFiles.  When the predicate receives /Pictures/PhotosLibrary.photoslibrary/somefile.ext I can compare its path against my list and reject it.  Not very efficient though.

 

Any tips for using the FileManager API directly?  Never tried to call macOS/Cocoa directly from Delphi.  Does the RTL include functions to help with that?  I'm guessing converting parameter types from Cocoa to Delphi and back again might be tricky.

[Brian Evans 105]

1 hour ago, Incus J said:

Any tips for using the FileManager API directly?  Never tried to call macOS/Cocoa directly from Delphi.  Does the RTL include functions to help with that?  I'm guessing converting parameter types from Cocoa to Delphi and back again might be tricky.

There is some code / interfaces in Macapi.foundation including one for NSFileManager, at least in Delphi 11.1.  Used by TPath.InternalGetMACOSPath in System.IOUtils when {$IFDEF MACOS} which you can look at to get some ideas at least.

Edited June 30, 2022 by Brian Evans

[Wil van Antwerpen 25]

On macOS if a folder is a bundle or not, is not just determined by a file extension.

It's something called bundle bits.
You can check those with command "xattr -lx"

For more info see: https://ss64.com/osx/xattr.html