The plFileSystem provides high-level functionality to manage files in a virtual file system. More...

#include <FileSystem.h>

Classes
struct	FileEvent
	The event data that is broadcast by the plFileSystem upon certain file operations. More...

struct	FileEventType
	Describes the type of events that are broadcast by the plFileSystem. More...

Static Public Member Functions
static plEventSubscriptionID	RegisterEventHandler (plEvent< const FileEvent & >::Handler handler)
	Registers an Event Handler that will be informed about all the events that the file system broadcasts.

static void	UnregisterEventHandler (plEvent< const FileEvent & >::Handler handler)
	Unregisters a previously registered Event Handler.

static void	UnregisterEventHandler (plEventSubscriptionID subscriptionId)

static plResult	CreateDirectoryStructure (plStringView sPath)

static void	DeleteFile (plStringView sFile)
	Deletes the given file from all data directories, if possible.

static bool	ExistsFile (plStringView sFile)
	Checks whether the given file exists in any data directory.

static plResult	GetFileStats (plStringView sFileOrFolder, plFileStats &out_stats)
	Tries to get the plFileStats for the given file. Typically should give the same results as plOSFile::GetFileStats, but some data dir implementations may not support retrieving all data (e.g. GetFileStats on folders might not always work).

static plResult	ResolvePath (plStringView sPath, plStringBuilder out_pAbsolutePath, plStringBuilder out_pDataDirRelativePath, const plDataDirectoryInfo **out_pDataDir=nullptr)
	Tries to resolve the given path and returns the absolute and relative path to the final file.

static plResult	FindFolderWithSubPath (plStringBuilder &ref_sResult, plStringView sStartDirectory, plStringView sSubPath, plStringView sRedirectionFileName={})
	Starts at szStartDirectory and goes up until it finds a folder that contains the given sub folder structure.

static bool	ResolveAssetRedirection (plStringView sPathOrAssetGuid, plStringBuilder &out_sRedirection)
	Returns true, if any data directory knows how to redirect the given path. Otherwise the original string is returned in out_sRedirection.

static plStringView	MigrateFileLocation (plStringView sOldLocation, plStringView sNewLocation)
	Migrates a file from an old location to a new one, and returns the path that should be used to open it (either the old or the new path).

Special Directories
static plResult	DetectSdkRootDirectory (plStringView sExpectedSubFolder="Data/Base")
	Searches for a directory to use as the "Sdk" special directory.

static void	SetSdkRootDirectory (plStringView sSdkDir)
	the special directory ">Sdk" is the root folder of the SDK data, it is often used as the main reference from where other data directories are found. For higher level code (e.g. plApplication) it is often vital that this is set early at startup.

static plStringView	GetSdkRootDirectory ()
	Returns the previously set Sdk root directory.

static void	SetSpecialDirectory (plStringView sName, plStringView sReplacement)
	Special directories are used when mounting data directories as basic references.

static plResult	ResolveSpecialDirectory (plStringView sDirectory, plStringBuilder &out_sPath)
	Returns the absolute path to szDirectory.

Misc
static plMutex &	GetMutex ()
	Returns the (recursive) mutex that is used internally by the file system which can be used to guard bundled operations on the file system.

Friends
class	plDataDirectoryReaderWriterBase

class	plFileReaderBase

class	plFileWriterBase

Data Directory Modifications
All functions that add / remove data directories are not thread safe and require that this is done on a single thread with no other thread accessing anything in plFileSystem simultaneously.
using	plDataDirFactory = plDataDirectoryType* (*)(plStringView, plStringView, plStringView, plDataDirUsage)
	This factory creates a data directory type, if it can handle the given data directory. Otherwise it returns nullptr.

static void	RegisterDataDirectoryFactory (plDataDirFactory factory, float fPriority=0)
	This function allows to register another data directory factory, which might be invoked when a new data directory is to be added.

static void	ClearAllDataDirectoryFactories ()
	Will remove all known data directory factories.

static plResult	AddDataDirectory (plStringView sDataDirectory, plStringView sGroup={}, plStringView sRootName={}, plDataDirUsage usage=plDataDirUsage::ReadOnly)
	Adds a data directory. It will try all the registered factories to find a data directory type that can handle the given path.

static bool	RemoveDataDirectory (plStringView sRootName)
	Searches for a data directory with the given root name and removes it.

static plUInt32	RemoveDataDirectoryGroup (plStringView sGroup)
	Removes all data directories that belong to the given group. Returns the number of data directories that were removed.

static void	ClearAllDataDirectories ()
	Removes all data directories.

static const plDataDirectoryInfo *	FindDataDirectoryWithRoot (plStringView sRootName)
	If a data directory with the given root name already exists, it will be returned, nullptr otherwise.

static plUInt32	GetNumDataDirectories ()
	Returns the number of currently active data directories.

static plDataDirectoryType *	GetDataDirectory (plUInt32 uiDataDirIndex)
	Returns the n-th currently active data directory.

static const plDataDirectoryInfo &	GetDataDirectoryInfo (plUInt32 uiDataDirIndex)
	Returns the info about the n-th currently active data directory.

static void	ReloadAllExternalDataDirectoryConfigs ()
	Calls plDataDirectoryType::ReloadExternalConfigs() on all active data directories.

Detailed Description

The plFileSystem provides high-level functionality to manage files in a virtual file system.

There are two sides at which the file system can be extended: Data directories are the 'sources' of data. These can be simple folders, zip files, data-bases, HTTP servers, etc. Different plDataDirectoryType's can implement these different 'decoding methods', i.e. they handle how to actually access the data and they use their own readers/writers to implement a common interface for passing data streams to and from the data directory. On the other end there are the actual file readers/writers, which implement policies how to optimize these reads/writes. The default plFileReader and plFileWriter implement a buffering policy, i.e. they use an internal cache to only sporadically read or write to the actual data stream. A 'threaded' or 'parallel' file reader/writer could implement a different policy, where a file is read/written in a thread and thus allows to have non-blocking file accesses.

Which policy to use is defined by the user every time he needs to access a file, by simply using the desired reader/writer class. How to mount data directories (i.e. with which plDataDirectoryType) is defined by the 'DataDirFactories', which are functions that create plDataDirectoryType's. This way one can mount the same data directory (e.g. "MyTestDir") differently, depending on which Factories have been registered previously. This allows to easily configure how to set up data directories. E.g. by default ordinary folders will be mounted to be read from the local file system. However, by registering a different Factory, the same directory could also be mounted over a network on a remote file serving machine.

Additionally plFileSystem also broadcasts events about which files are (about to be) accessed. This allows to hook into the system and implement stuff like automatic asset transformations before/after certain file accesses, checking out files from revision control systems, or simply logging all file activity.

All operations that go through the plFileSystem are protected by a mutex, which means that opening, closing, deleting files, as well as adding or removing data directories etc. will be synchronized and cannot happen in parallel. Reading/writing file streams can happen in parallel, only the administrative tasks need to be protected. File events are broadcast as they occur, that means they will be executed on whichever thread triggered them. Since they are executed from within the filesystem mutex, they cannot occur in parallel.

Member Typedef Documentation

◆ plDataDirFactory

using plFileSystem::plDataDirFactory = plDataDirectoryType* (*)(plStringView, plStringView, plStringView, plDataDirUsage)

This factory creates a data directory type, if it can handle the given data directory. Otherwise it returns nullptr.

Every time a data directory is supposed to be added, the file system will query its data dir factories, which one can successfully create an plDataDirectoryType. In this process the last factory added has the highest priority. Once a factory is found that was able to create a plDataDirectoryType, that one is used. Different factories can be used to mount different types of data directories. But the same directory can also be mounted in different ways. For example a simple folder could be mounted on the local system, or via a HTTP server over a network (lets call it a 'FileServer'). Thus depending on which type of factories are registered, the file system can provide data from very different sources.

Member Function Documentation

◆ AddDataDirectory()

plResult plFileSystem::AddDataDirectory	(	plStringView	sDataDirectory,
		plStringView	sGroup = {},
		plStringView	sRootName = {},
		plDataDirUsage	usage = plDataDirUsage::ReadOnly )

static

Adds a data directory. It will try all the registered factories to find a data directory type that can handle the given path.

If Usage is ReadOnly, writing to the data directory is not allowed. This is independent of whether the data directory type COULD write anything. szGroup defines to what 'group' of data directories this data dir belongs. This is only used in calls to RemoveDataDirectoryGroup, to remove all data directories of the same group. You could use groups such as 'Base', 'Project', 'Settings', 'Level', 'Temp' to distinguish between different sets of data directories. You can also specify the exact same string as szDataDirectory for szGroup, and thus uniquely identify the data dir, to be able to remove just that one. szRootName is optional for read-only data dirs, but mandatory for writable ones. It has to be unique to clearly identify a file within that data directory. It must be used when writing to a file in this directory. For instance, if a data dir root name is "mydata", then the path ":mydata/SomeFile.txt" can be used to write to the top level folder of this data directory. The same can be used for reading exactly that file and ignoring the other data dirs.

◆ DeleteFile()

void plFileSystem::DeleteFile ( plStringView sFile )

static

Deletes the given file from all data directories, if possible.

The path must be absolute or rooted, to uniquely identify which file to delete. For example ":appdata/SomeData.txt", assuming a writable data directory has been mounted with the "appdata" root name.

◆ DetectSdkRootDirectory()

plResult plFileSystem::DetectSdkRootDirectory ( plStringView sExpectedSubFolder = "Data/Base" )

static

Searches for a directory to use as the "Sdk" special directory.

It does so by starting at the directory where the application binary is located and then goes up until it finds a folder that contains the given sub-folder. The sub-folder is usually where the engine loads the most basic data from, so it should exist.

Additionally the 'redirection file' feature of plFileSystem::FindFolderWithSubPath() is used to allow finding a relocated SDK folder. To do that, place a file called 'plSdkRoot.txt' in your top level folder. It should contain the relative path pointing to the SDK folder.

Upon success SetSdkRootDirectory() is called with the resulting path.

Note: If the Sdk root directory has been set before, this function does nothing! It will not override a previously set value. If that is desired, call SetSdkRootDirectory("") first.

See also: plFileSystem::FindFolderWithSubPath()

◆ ExistsFile()

bool plFileSystem::ExistsFile ( plStringView sFile )

static

Checks whether the given file exists in any data directory.

The search can be restricted to directories of certain categories (see AddDataDirectory).

◆ FindFolderWithSubPath()

plResult plFileSystem::FindFolderWithSubPath	(	plStringBuilder &	ref_sResult,
		plStringView	sStartDirectory,
		plStringView	sSubPath,
		plStringView	sRedirectionFileName = {} )

static

Starts at szStartDirectory and goes up until it finds a folder that contains the given sub folder structure.

Returns PL_FAILURE if nothing is found. Otherwise result is the absolute path to the existing folder that has a given sub-folder.

Parameters

result	If successful, this contains the folder path in which szSubPath exists.
szStartDirectory	The directory in which to start the search and iterate upwards.
szSubPath	the relative path to look for in each visited directory. The function succeeds if such a file or folder is found.
szRedirectionFileName	An optional file name for a redirection file. If in any visited folder a file with this name is found, it will be opened, read entirely, and appended to the current search path, and it is checked whether szSubPath can be found there. This step is not recursive and can't result in an endless loop. It allows to relocate the SDK folder and still have it found, by placing such a redirection file. A common use case, is when plEngine is used as a Git submodule and therefore the overall file structure is slightly different.

◆ GetSdkRootDirectory()

plStringView plFileSystem::GetSdkRootDirectory ( )

static

Returns the previously set Sdk root directory.

Note: Asserts that the path is not empty!

See also: SetSdkRootDirectory; DetectSdkRootDirectory

◆ MigrateFileLocation()

plStringView plFileSystem::MigrateFileLocation	(	plStringView	sOldLocation,
		plStringView	sNewLocation )

static

Migrates a file from an old location to a new one, and returns the path that should be used to open it (either the old or the new path).

If the file does not exist in the old location, nothing is done, and the new location is returned. Otherwise, it is attempted to move the file from the old location to the new location. In case that fails (target not writeable or so), the old path is returned, so that code that needs to read that file, finds it in the correct location. If it succeeds, the new location is returned. Afterwards, the file does not exist in the old location anymore.

◆ RemoveDataDirectory()

bool plFileSystem::RemoveDataDirectory ( plStringView sRootName )

static

Searches for a data directory with the given root name and removes it.

Returns true, if one was found and removed, false if no such data dir existed.

◆ ResolvePath()

plResult plFileSystem::ResolvePath	(	plStringView	sPath,
		plStringBuilder *	out_pAbsolutePath,
		plStringBuilder *	out_pDataDirRelativePath,
		const plDataDirectoryInfo **	out_pDataDir = nullptr )

static

Tries to resolve the given path and returns the absolute and relative path to the final file.

If the given path is a rooted path, for instance something like ":appdata/UserData.txt", (which is necessary for writing to files), the path can be converted easily and the file does not need to exist. Only the data directory with the given root name must be mounted.

If the path is relative, it is attempted to open the specified file, which means it is searched in all available data directories. The path to the file that is found will be returned.

Parameters

sPath	can be a relative, an absolute or a rooted path. This can also be used to find the relative location to the data directory that would handle it.
out_sAbsolutePath	will contain the absolute path to the file. Can be nullptr.
out_sDataDirRelativePath	will contain the relative path to the file (from the data directory in which it might end up in). Can be nullptr.
out_ppDataDir	If not null, it will be set to the data directory that would handle this path.

Returns: The function will return PL_FAILURE if it was not able to determine any location where the file could be read from or written to.

Todo: We might also need the none-redirected path as an output

Todo: We might also need the none-redirected path as an output

◆ ResolveSpecialDirectory()

plResult plFileSystem::ResolveSpecialDirectory	(	plStringView	sDirectory,
		plStringBuilder &	out_sPath )

static

Returns the absolute path to szDirectory.

If the path starts with a known special directory marker (">marker/") it is replaced accordingly. See SetSpecialDirectory() for setting custom special directories.

Built-in special directories (always available) are:

">sdk/" - Resolves to what GetSdkRootDirectory() returns. ">user/" - Resolves to what plOSFile::GetUserDataFolder() returns. ">temp/" - Resolves to what plOSFile::GetTempDataFolder() returns. ">appdir/" - Resolves to what plOSFile::GetApplicationDirectory() returns.

Returns PL_FAILURE if szDirectory starts with an unknown special directory.

◆ SetSdkRootDirectory()

void plFileSystem::SetSdkRootDirectory ( plStringView sSdkDir )

static

the special directory ">Sdk" is the root folder of the SDK data, it is often used as the main reference from where other data directories are found. For higher level code (e.g. plApplication) it is often vital that this is set early at startup.

See also: DetectSdkRootDirectory()

◆ SetSpecialDirectory()

void plFileSystem::SetSpecialDirectory	(	plStringView	sName,
		plStringView	sReplacement )

static

Special directories are used when mounting data directories as basic references.

They are indicated with a ">", ie. ">sdk/Test", but using them is only allowed in few places, e.g. in AddDataDirectory(). Special directories are needed to be able to set up other paths relative to them and to be able to use different ones on different PCs. For instance when using file-serve functionality, the special directories may be different on the host and client machines, but the paths used to mount data directories can stay the same because of this.

The documentation for this class was generated from the following files:

Code/Engine/Foundation/IO/FileSystem/FileSystem.h
Code/Engine/Foundation/IO/FileSystem/Implementation/FileSystem.cpp

Classes

Static Public Member Functions

Friends

Data Directory Modifications

Detailed Description

Member Typedef Documentation

◆ plDataDirFactory

Member Function Documentation

◆ AddDataDirectory()

◆ DeleteFile()

◆ DetectSdkRootDirectory()

◆ ExistsFile()

◆ FindFolderWithSubPath()

◆ GetSdkRootDirectory()

◆ MigrateFileLocation()

◆ RemoveDataDirectory()

◆ ResolvePath()

◆ ResolveSpecialDirectory()

◆ SetSdkRootDirectory()

◆ SetSpecialDirectory()