`tsdat.io.filehandlers`¶

This module contains the File Handlers that come packaged with tsdat in addition to methods for registering new File Handler objects.

Submodules¶

Classes¶

`AbstractFileHandler`	Abstract class to define methods required by all FileHandlers. Classes
`CsvHandler`	FileHandler to read from and write to CSV files. Takes a number of
`FileHandler`	Class to provide methods to read and write files with a variety of
`NetCdfHandler`	FileHandler to read from and write to netCDF files. Takes a number of
`SplitNetCdfHandler`	FileHandler to read from and write to netCDF files. Takes a number of

class tsdat.io.filehandlers.AbstractFileHandler(parameters: Union[Dict, None] = None)[source]¶

Abstract class to define methods required by all FileHandlers. Classes derived from AbstractFileHandler should implement one or more of the following methods:

write(ds: xr.Dataset, filename: str, config: Config, **kwargs)

read(filename: str, **kwargs) -> xr.Dataset

Parameters: parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.

Class Methods

`read`	Reads in the given file and converts it into an Xarray dataset for
`write`	Saves the given dataset to a file.

Method Descriptions

read(self, filename: str, **kwargs) → xarray.Dataset¶

Reads in the given file and converts it into an Xarray dataset for use in the pipeline.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Saves the given dataset to a file.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

class tsdat.io.filehandlers.CsvHandler(parameters: Union[Dict, None] = None)[source]¶

Bases: tsdat.io.filehandlers.file_handlers.AbstractFileHandler

FileHandler to read from and write to CSV files. Takes a number of parameters that are passed in from the storage config file. Parameters specified in the config file should follow the following example:

parameters:
  write:
    to_dataframe:
      # Parameters here will be passed to xr.Dataset.to_dataframe()
    to_csv:
      # Parameters here will be passed to pd.DataFrame.to_csv()
  read:
    read_csv:
      # Parameters here will be passed to pd.read_csv()
    to_xarray:
      # Parameters here will be passed to pd.DataFrame.to_xarray()

Parameters: parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.

Class Methods

`read`	Reads in the given file and converts it into an Xarray dataset for
`write`	Saves the given dataset to a csv file.

Method Descriptions

read(self, filename: str, **kwargs) → xarray.Dataset¶

Reads in the given file and converts it into an Xarray dataset for use in the pipeline.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Saves the given dataset to a csv file.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

class tsdat.io.filehandlers.FileHandler[source]¶

Class to provide methods to read and write files with a variety of extensions.

FILEREADERS :Dict[str, AbstractFileHandler]¶

FILEWRITERS :Dict[str, AbstractFileHandler]¶

Class Methods

`read`	Reads in the given file and converts it into an xarray dataset object using the
`register_file_handler`	Method to register a FileHandler for reading from or writing to files matching one or
`write`	Calls the appropriate FileHandler to write the dataset to the provided filename.

Method Descriptions

read(self, filename: str, **kwargs) → xarray.Dataset¶

Reads in the given file and converts it into an xarray dataset object using the registered FileHandler for the provided filepath.

Parameters: filename (str) – The path to the file to read in.
Returns: The raw file as an Xarray.Dataset object.
Return type: xr.Dataset

register_file_handler(self, method: Literal[read, write], patterns: Union[str, List[str]], handler: AbstractFileHandler)¶

Method to register a FileHandler for reading from or writing to files matching one or more provided file patterns.

Parameters

method ("Literal") – The method the FileHandler should call if the pattern is
Must be one of (matched.) – “read”, “write”.
patterns (Union[str, List[str]]) – The file pattern(s) that determine if this
should be run on a given filepath. (FileHandler) –
handler (AbstractFileHandler) – The AbstractFileHandler to register.

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Calls the appropriate FileHandler to write the dataset to the provided filename.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to the file where the dataset should be written.
config (Config, optional) – Optional Config object. Defaults to None.

class tsdat.io.filehandlers.NetCdfHandler(parameters: Union[Dict, None] = None)[source]¶

Bases: tsdat.io.filehandlers.file_handlers.AbstractFileHandler

FileHandler to read from and write to netCDF files. Takes a number of parameters that are passed in from the storage config file. Parameters specified in the config file should follow the following example:

parameters:
  write:
    to_netcdf:
      # Parameters here will be passed to xr.Dataset.to_netcdf()
  read:
    load_dataset:
      # Parameters here will be passed to xr.load_dataset()

Parameters: parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.

Class Methods

`read`	Reads in the given file and converts it into an Xarray dataset for
`write`	Saves the given dataset to a netCDF file.

Method Descriptions

read(self, filename: str, **kwargs) → xarray.Dataset¶

Reads in the given file and converts it into an Xarray dataset for use in the pipeline.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Saves the given dataset to a netCDF file.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

class tsdat.io.filehandlers.SplitNetCdfHandler(parameters: Union[Dict, None] = None)[source]¶

Bases: NetCdfHandler

FileHandler to read from and write to netCDF files. Takes a number of parameters that are passed in from the storage config file. Parameters specified in the config file should follow the following example:

parameters:
  write:
    to_netcdf:
      # Parameters here will be passed to xr.Dataset.to_netcdf()
  read:
    load_dataset:
      # Parameters here will be passed to xr.load_dataset()

Parameters: parameters (Dict, optional) – Parameters that were passed to the FileHandler when it was registered in the storage config file, defaults to {}.

Class Methods

`read`	Reads in the given file and converts it into an Xarray dataset for
`write`	Saves the given dataset to netCDF file(s) based on the ‘time_interval’

Method Descriptions

abstract read(self, filename: str, **kwargs)¶

Reads in the given file and converts it into an Xarray dataset for use in the pipeline.

Parameters: filename (str) – The path to the file to read in.
Returns: A xr.Dataset object.
Return type: xr.Dataset

write(self, ds: xarray.Dataset, filename: str, config: tsdat.config.Config = None, **kwargs) → None¶

Saves the given dataset to netCDF file(s) based on the ‘time_interval’ and ‘time_unit’ config parameters.

Parameters

ds (xr.Dataset) – The dataset to save.
filename (str) – The path to where the file should be written to.
config (Config, optional) – Optional Config object, defaults to None

tsdat.io.filehandlers¶

Submodules¶

Classes¶

`tsdat.io.filehandlers`¶