`tsdat.io.filehandlers.file_handlers`¶

Classes¶

`AbstractFileHandler`	Abstract class to define methods required by all FileHandlers. Classes
`FileHandler`	Class to provide methods to read and write files with a variety of

Functions¶

register_filehandler

Python decorator to register an AbstractFileHandler in the FileHandler

Function Descriptions

class tsdat.io.filehandlers.file_handlers.AbstractFileHandler(parameters: Union[Dict, None] = None)[source]¶

Abstract class to define methods required by all FileHandlers. Classes derived from AbstractFileHandler should implement one or more of the following methods:

write(ds: xr.Dataset, filename: str, config: Config, **kwargs)

read(filename: str, **kwargs) -> xr.Dataset

Parameters: parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.

Class Methods

`read`	Reads in the given file and converts it into an Xarray dataset for
`write`	Saves the given dataset to a file.

Method Descriptions

read(self, filename: str, **kwargs) → xarray.Dataset[source]¶

Reads in the given file and converts it into an Xarray dataset for use in the pipeline.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None[source]¶

Saves the given dataset to a file.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

class tsdat.io.filehandlers.file_handlers.FileHandler[source]¶

Class to provide methods to read and write files with a variety of extensions.

FILEREADERS :Dict[str, AbstractFileHandler][source]¶

FILEWRITERS :Dict[str, AbstractFileHandler][source]¶

Class Methods

`read`	Reads in the given file and converts it into an xarray dataset object using the
`register_file_handler`	Method to register a FileHandler for reading from or writing to files matching one or
`write`	Calls the appropriate FileHandler to write the dataset to the provided filename.

Method Descriptions

static read(filename: str, **kwargs) → xarray.Dataset[source]¶

Reads in the given file and converts it into an xarray dataset object using the registered FileHandler for the provided filepath.

Parameters: filename (str) – The path to the file to read in.
Returns: The raw file as an Xarray.Dataset object.
Return type: xr.Dataset

static register_file_handler(method: Literal[read, write], patterns: Union[str, List[str]], handler: AbstractFileHandler)[source]¶

Method to register a FileHandler for reading from or writing to files matching one or more provided file patterns.

Parameters

method ("Literal") – The method the FileHandler should call if the pattern is
Must be one of (matched.) – “read”, “write”.
patterns (Union[str, List[str]]) – The file pattern(s) that determine if this
should be run on a given filepath. (FileHandler) –
handler (AbstractFileHandler) – The AbstractFileHandler to register.

static write(ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None[source]¶

Calls the appropriate FileHandler to write the dataset to the provided filename.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to the file where the dataset should be written.
config (Config, optional) – Optional Config object. Defaults to None.

tsdat.io.filehandlers.file_handlers.register_filehandler(patterns: Union[str, List[str]]) → AbstractFileHandler [source]¶

Python decorator to register an AbstractFileHandler in the FileHandler object. The FileHandler object will be used by tsdat pipelines to read and write raw, intermediate, and processed data.

This decorator can be used to work with a specific AbstractFileHandler without having to specify a config file. This is useful when using an AbstractFileHandler for analysis or for tests outside of a pipeline. For tsdat pipelines, handlers should always be specified via the storage config file.

Example Usage:

import xarray as xr
from tsdat.io import register_filehandler, AbstractFileHandler

@register_filehandler(["*.nc", "*.cdf"])
class NetCdfHandler(AbstractFileHandler):
    def write(ds: xr.Dataset, filename: str, config: Config = None, **kwargs):
        ds.to_netcdf(filename)
    def read(filename: str, **kwargs) -> xr.Dataset:
        xr.load_dataset(filename)

Parameters: patterns (Union[str, List[str]]) – The patterns (regex) that should be used to match a filepath to the AbstractFileHandler provided.
Returns: The original AbstractFileHandler class, after it has been registered for use in tsdat pipelines.
Return type: AbstractFileHandler

tsdat.io.filehandlers.file_handlers¶

Classes¶

Functions¶

`tsdat.io.filehandlers.file_handlers`¶