`tsdat.io.filehandlers.file_handlers`¶

Module Contents¶

Classes¶

`AbstractFileHandler`	Abstract class to define methods required by all FileHandlers. Classes
`FileHandler`	Class to provide methods to read and write files with a variety of

Functions¶

register_filehandler(patterns: Union[str, List[str]]) → AbstractFileHandler

Python decorator to register an AbstractFileHandler in the FileHandler

class tsdat.io.filehandlers.file_handlers.AbstractFileHandler(parameters: Union[Dict, None] = None)¶

Abstract class to define methods required by all FileHandlers. Classes derived from AbstractFileHandler should implement one or more of the following methods:

write(ds: xr.Dataset, filename: str, config: Config, **kwargs)

read(filename: str, **kwargs) -> xr.Dataset

Parameters: parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Saves the given dataset to a file.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

read(self, filename: str, **kwargs) → xarray.Dataset¶

Reads in the given file and converts it into an Xarray dataset for use in the pipeline.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

class tsdat.io.filehandlers.file_handlers.FileHandler¶

Class to provide methods to read and write files with a variety of extensions.

FILEHANDLERS :Dict[str, AbstractFileHandler]¶

static _get_handler(filename: str) → AbstractFileHandler ¶

Given the name of the file to read or write, this method applies a regular expression to match the name of the file with a handler that has been registered in its internal dictionary of FileHandler objects and returns the appropriate FileHandler, or None if a match is not found.

Parameters: filename (str) – The name of the file whose handler should be retrieved.
Returns: The FileHandler registered for use with the provided filename.
Return type: AbstractFileHandler

static write(ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Saves the given dataset to file using the registered FileHandler for the provided filename.

Parameters

ds (xr.Dataset) – The dataset ot save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

static read(filename: str, **kwargs) → xarray.Dataset¶

Reads in the given file and converts it into an Xarray dataset using the registered FileHandler for the provided filename.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

static register_file_handler(patterns: Union[str, List[str]], handler: AbstractFileHandler)¶

Static method to register an AbstractFileHandler for one or more file patterns. Once an AbstractFileHandler has been registered it may be used by this class to read or write files whose paths match one or more pattern(s) provided in registration.

Parameters

patterns (Union[str, List[str]]) – The patterns (regex) that should be used to match a filepath to the AbstractFileHandler provided.
handler (AbstractFileHandler) – The AbstractFileHandler to register.

tsdat.io.filehandlers.file_handlers.register_filehandler(patterns: Union[str, List[str]]) → AbstractFileHandler ¶

Python decorator to register an AbstractFileHandler in the FileHandler object. The FileHandler object will be used by tsdat pipelines to read and write raw, intermediate, and processed data.

This decorator can be used to work with a specific AbstractFileHandler without having to specify a config file. This is useful when using an AbstractFileHandler for analysis or for tests outside of a pipeline. For tsdat pipelines, handlers should always be specified via the storage config file.

Example Usage:

import xarray as xr
from tsdat.io import register_filehandler, AbstractFileHandler

@register_filehandler(["*.nc", "*.cdf"])
class NetCdfHandler(AbstractFileHandler):
    def write(ds: xr.Dataset, filename: str, config: Config = None, **kwargs):
        ds.to_netcdf(filename)
    def read(filename: str, **kwargs) -> xr.Dataset:
        xr.load_dataset(filename)

Parameters: patterns (Union[str, List[str]]) – The patterns (regex) that should be used to match a filepath to the AbstractFileHandler provided.
Returns: The original AbstractFileHandler class, after it has been registered for use in tsdat pipelines.
Return type: AbstractFileHandler

tsdat.io.filehandlers.file_handlers¶

Module Contents¶

Classes¶

Functions¶

`tsdat.io.filehandlers.file_handlers`¶