retriever

Classes:

Name	Description
Retriever	Base class for retrieving data used as input to tsdat pipelines.

Classes

Retriever

Bases: ParameterizedClass, ABC

Base class for retrieving data used as input to tsdat pipelines.

Methods:

Name	Description
match_coord
match_data_var
retrieve

Attributes:

Name	Type	Description
coords	Dict[str, Dict[Pattern, RetrievedVariable]]	A dictionary mapping output coordinate names to the retrieval rules and
data_vars	Dict[str, Dict[Pattern, RetrievedVariable]]	A dictionary mapping output data variable names to the retrieval rules and
readers	Optional[Dict[Pattern, Any]]	The mapping of readers that should be used to retrieve data given input_keys and

Attributes

coords instance-attribute

coords: Dict[str, Dict[Pattern, RetrievedVariable]]

A dictionary mapping output coordinate names to the retrieval rules and preprocessing actions (e.g., DataConverters) that should be applied to each retrieved coordinate variable.

data_vars instance-attribute

data_vars: Dict[str, Dict[Pattern, RetrievedVariable]]

A dictionary mapping output data variable names to the retrieval rules and preprocessing actions (e.g., DataConverters) that should be applied to each retrieved data variable.

readers instance-attribute

readers: Optional[Dict[Pattern, Any]]

The mapping of readers that should be used to retrieve data given input_keys and optional keyword arguments provided by subclasses of Retriever.

Functions

match_coord

match_coord(
    var_name: str, input_key: str
) -> RetrievedVariable | None

Source code in tsdat/io/base/retriever.py

def match_coord(self, var_name: str, input_key: str) -> RetrievedVariable | None:
    mapping = self.coords[var_name]
    for pattern, ret_var in mapping.items():
        if pattern.match(input_key):
            return ret_var

match_data_var

match_data_var(
    var_name: str, input_key: str
) -> RetrievedVariable | None

Source code in tsdat/io/base/retriever.py

def match_data_var(self, var_name: str, input_key: str) -> RetrievedVariable | None:
    mapping = self.data_vars[var_name]
    for pattern, ret_var in mapping.items():
        if pattern.match(input_key):
            return ret_var

retrieve abstractmethod

retrieve(
    input_keys: List[str],
    dataset_config: DatasetConfig,
    **kwargs: Any
) -> xr.Dataset

---

Prepares the raw dataset mapping for use in downstream pipeline processes.

This is done by consolidating the data into a single xr.Dataset object. The retrieved dataset may contain additional coords and data_vars that are not defined in the output dataset. Input data converters are applied as part of the preparation process.

Parameters:

Name	Type	Description	Default
input_keys	List[str]	The input keys the registered DataReaders should read from.	required
dataset_config	DatasetConfig	The specification of the output dataset.	required

Returns:

Type	Description
Dataset	xr.Dataset: The retrieved dataset.

---

Source code in tsdat/io/base/retriever.py

@abstractmethod
def retrieve(
    self, input_keys: List[str], dataset_config: DatasetConfig, **kwargs: Any
) -> xr.Dataset:
    """-----------------------------------------------------------------------------
    Prepares the raw dataset mapping for use in downstream pipeline processes.

    This is done by consolidating the data into a single xr.Dataset object. The
    retrieved dataset may contain additional coords and data_vars that are not
    defined in the output dataset. Input data converters are applied as part of the
    preparation process.

    Args:
        input_keys (List[str]): The input keys the registered DataReaders should
            read from.
        dataset_config (DatasetConfig): The specification of the output dataset.

    Returns:
        xr.Dataset: The retrieved dataset.

    -----------------------------------------------------------------------------"""
    ...