tsdat.io.filehandlers.file_handlers
¶
Module Contents¶
Classes¶
Abstract class to define methods required by all FileHandlers. Classes |
|
Class to provide methods to read and write files with a variety of |
Functions¶
|
Python decorator to register an AbstractFileHandler in the FileHandler |
-
class
tsdat.io.filehandlers.file_handlers.
AbstractFileHandler
(parameters: Dict = {})¶ Abstract class to define methods required by all FileHandlers. Classes derived from AbstractFileHandler should implement one or more of the following methods:
write(ds: xr.Dataset, filename: str, config: Config, **kwargs)
read(filename: str, **kwargs) -> xr.Dataset
- Parameters
parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.
-
write
(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶ Saves the given dataset to a file.
- Parameters
ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None
-
read
(self, filename: str, **kwargs) → xarray.Dataset¶ Reads in the given file and converts it into an Xarray dataset for use in the pipeline.
- Parameters
filename (str) – The path to the file to read in.
- Returns
A xr.Dataset object.
- Return type
xr.Dataset
-
class
tsdat.io.filehandlers.file_handlers.
FileHandler
¶ Class to provide methods to read and write files with a variety of extensions.
-
FILEHANDLERS
:Dict[str, AbstractFileHandler]¶
-
static
_get_handler
(filename: str) → AbstractFileHandler¶ Given the name of the file to read or write, this method applies a regular expression to match the name of the file with a handler that has been registered in its internal dictionary of FileHandler objects and returns the appropriate FileHandler, or None if a match is not found.
- Parameters
filename (str) – The name of the file whose handler should be retrieved.
- Returns
The FileHandler registered for use with the provided filename.
- Return type
-
static
write
(ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶ Saves the given dataset to file using the registered FileHandler for the provided filename.
- Parameters
ds (xr.Dataset) – The dataset ot save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None
-
static
read
(filename: str, **kwargs) → xarray.Dataset¶ Reads in the given file and converts it into an Xarray dataset using the registered FileHandler for the provided filename.
- Parameters
filename (str) – The path to the file to read in.
- Returns
A xr.Dataset object.
- Return type
xr.Dataset
-
static
register_file_handler
(patterns: Union[str, List[str]], handler: AbstractFileHandler)¶ Static method to register an AbstractFileHandler for one or more file patterns. Once an AbstractFileHandler has been registered it may be used by this class to read or write files whose paths match one or more pattern(s) provided in registration.
- Parameters
patterns (Union[str, List[str]]) – The patterns (regex) that should be used to match a filepath to the AbstractFileHandler provided.
handler (AbstractFileHandler) – The AbstractFileHandler to register.
-
-
tsdat.io.filehandlers.file_handlers.
register_filehandler
(patterns: Union[str, List[str]]) → AbstractFileHandler¶ Python decorator to register an AbstractFileHandler in the FileHandler object. The FileHandler object will be used by tsdat pipelines to read and write raw, intermediate, and processed data.
This decorator can be used to work with a specific AbstractFileHandler without having to specify a config file. This is useful when using an AbstractFileHandler for analysis or for tests outside of a pipeline. For tsdat pipelines, handlers should always be specified via the storage config file.
Example Usage:
import xarray as xr from tsdat.io import register_filehandler, AbstractFileHandler @register_filehandler(["*.nc", "*.cdf"]) class NetCdfHandler(AbstractFileHandler): def write(ds: xr.Dataset, filename: str, config: Config = None, **kwargs): ds.to_netcdf(filename) def read(filename: str, **kwargs) -> xr.Dataset: xr.load_dataset(filename)
- Parameters
patterns (Union[str, List[str]]) – The patterns (regex) that should be used to match a filepath to the AbstractFileHandler provided.
- Returns
The original AbstractFileHandler class, after it has been registered for use in tsdat pipelines.
- Return type