tsdat.io.storage

Classes

FileSystem

Handles data storage and retrieval for file-based data formats. Formats that write to

class tsdat.io.storage.FileSystem

Bases: tsdat.io.base.Storage

Handles data storage and retrieval for file-based data formats. Formats that write to directories (such as zarr) are not supported by the FileSystem storage class.

Parameters

• parameters (Parameters) – File-system specific parameters, such as the root path to

• files should be saved (where) –

• additional keyword arguments to specific (or) –

• used by the storage API. See the FileSystemStorage.Parameters class for (functions) –

• details. (more) –

• handler (FileHandler) – The FileHandler class that should be used to handle data

• within the storage API. (I/O) –

class Parameters

Bases: pydantic.BaseSettings

file_timespan :Optional[str]

merge_fetched_data_kwargs :Dict[str, Any]

storage_root :pathlib.Path

The path on disk where data and ancillary files will be saved to. Defaults to the storage/root folder in the active working directory. The directory is created as this parameter is set, if the directory does not already exist.

handler :tsdat.io.handlers.FileHandler

parameters :FileSystem.Parameters

Class Methods

fetch_data	Fetches data for a given datastream between a specified time range.
save_ancillary_file	Saves an ancillary filepath to the datastream’s ancillary storage area.
save_data	Saves a dataset to the storage area. At a minimum, the dataset must have a ‘datastream’

Method Descriptions

fetch_data(self, start: datetime.datetime, end: datetime.datetime, datastream: str) → xarray.Dataset

Fetches data for a given datastream between a specified time range.

Note: this method is not smart; it searches for the appropriate data files using their filenames and does not filter within each data file.

Parameters

• start (datetime) – The minimum datetime to fetch.

• end (datetime) – The maximum datetime to fetch.

• datastream (str) – The datastream id to search for.

Returns

A dataset containing all the data in the storage area that spans the specified datetimes.

Return type

xr.Dataset

save_ancillary_file(self, filepath: pathlib.Path, datastream: str)

Saves an ancillary filepath to the datastream’s ancillary storage area.

Parameters

• filepath (Path) – The path to the ancillary file.

• datastream (str) – The datastream that the file is related to.

save_data(self, dataset: xarray.Dataset)

Saves a dataset to the storage area. At a minimum, the dataset must have a ‘datastream’ global attribute and must have a ‘time’ variable with a np.datetime64-like data type.

Parameters

dataset (xr.Dataset) – The dataset to save.