tsdat.pipeline.base
Classes
Base class for tsdat data pipelines. |
- class tsdat.pipeline.base.Pipeline[source]
Bases:
tsdat.utils.ParameterizedClass
,abc.ABC
Base class for tsdat data pipelines.
- dataset_config: tsdat.config.dataset.DatasetConfig[source]
Describes the structure and metadata of the output dataset.
- quality: tsdat.qc.base.QualityManagement[source]
Manages the dataset quality through checks and corrections.
- retriever: tsdat.io.base.Retriever[source]
Retrieves data from input keys.
- storage: tsdat.io.base.Storage[source]
Stores the dataset so it can be retrieved later.
- triggers: List[Pattern] = [][source]
Regex patterns matching input keys to determine when the pipeline should run.
Class Methods
Modifies the retrieved dataset by dropping variables not declared in the
Runs the data pipeline on the provided inputs.
Method Descriptions
- prepare_retrieved_dataset(dataset: xarray.Dataset) xarray.Dataset [source]
Modifies the retrieved dataset by dropping variables not declared in the DatasetConfig, adding static variables, initializing non-retrieved variables, and importing global and variable-level attributes from the DatasetConfig.
- Parameters:
dataset (xr.Dataset) – The retrieved dataset.
- Returns:
xr.Dataset – The dataset with structure and metadata matching the DatasetConfig.