tsdat.pipeline.pipelines
¶
Classes¶
Pipeline class designed to read in raw, unstandardized time series data and enhance |
-
class
tsdat.pipeline.pipelines.
IngestPipeline
[source]¶ Bases:
tsdat.pipeline.base.Pipeline
Pipeline class designed to read in raw, unstandardized time series data and enhance its quality and usability by converting it into a standard format, embedding metadata, applying quality checks and controls, generating reference plots, and saving the data in an accessible format so it can be used later in scientific analyses or in higher-level tsdat Pipelines.
Class Methods
User-overrideable code hook that runs after the retriever has retrieved the
User-overrideable code hook that runs after the dataset quality has been managed
User-overrideable code hook that runs after the dataset has been saved by the
Runs the data pipeline on the provided inputs.
Method Descriptions
-
hook_customize_dataset
(self, dataset: xarray.Dataset) → xarray.Dataset[source]¶ User-overrideable code hook that runs after the retriever has retrieved the dataset from the specified input keys, but before the pipeline has applied any quality checks or corrections to the dataset.
- Parameters
dataset (xr.Dataset) – The output dataset structure returned by the retriever
API. –
- Returns
The customized dataset.
- Return type
xr.Dataset
-
hook_finalize_dataset
(self, dataset: xarray.Dataset) → xarray.Dataset[source]¶ User-overrideable code hook that runs after the dataset quality has been managed but before the dataset has been sent to the storage API to be saved.
- Parameters
dataset (xr.Dataset) – The output dataset returned by the retriever API and
by the hook_customize_retrieved_dataset user code hook. (modified) –
- Returns
The finalized dataset, ready to be saved.
- Return type
xr.Dataset
-
hook_plot_dataset
(self, dataset: xarray.Dataset)[source]¶ User-overrideable code hook that runs after the dataset has been saved by the storage API.
- Parameters
dataset (xr.Dataset) – The dataset to plot.
-
run
(self, inputs: List[str], **kwargs: Any) → xarray.Dataset[source]¶ Runs the data pipeline on the provided inputs.
- Parameters
inputs (List[str]) – A list of input keys that the pipeline’s Retriever class
use to load data into the pipeline. (can) –
- Returns
The processed dataset.
- Return type
xr.Dataset
-