Classes

class ConceptDriftMonitor (model_id: str)

A class to monitor concept drift in machine learning models.

Attributes

_model_id : str

A unique identifier for the model.

_spinner : Halo

A spinner instance to show progress or status information.

Example

Initialize the ConceptDriftMonitor with a model ID.

>>> concept_drift_monitor = ConceptDriftMonitor(model_id="model123")

Start monitoring for concept drift.

>>> concept_drift_monitor.benchmark(y_true_train=["true", "false"], y_pred_train=["true", "true"])

Assuming start was successful, update the monitor with new data.

>>> concept_drift_monitor.track(y_true=["false", "true"], y_pred=["true", "false"])

Initializes the ConceptDriftMonitor with a specific model ID.

Args

model_id : str

The unique identifier for the model.

Methods

def benchmark(self, y_true_train: List[str], y_pred_train: List[str])

Starts monitoring for concept drift between training true values and predicted values.

Args

y_true_train : List[str]

The true values from training data.

y_pred_train : List[str]

The predicted values from training data.

Details

The information about y_true_train and y_pred_train is required to create the benchmark for the drift detector (NO_DRIFT case). Based on this data, the drift detector establishes what is not drift. For new samples, it will use statistical analysis to decide whether the model is drifting or not.

Raises

ValueError

If the lengths of y_true_train and y_pred_train do not match.

Note

After calling start successfully, you should call update with new data to continue monitoring.

def track(self, y_true: List[str], y_pred: List[str])

Updates the concept drift monitor with new true and predicted values.

Args

y_true : List[str]

The new true values.

y_pred : List[str]

The new predicted values.

Raises

ValueError

If the lengths of y_true and y_pred do not match.

Note

The update method must be called only after start has been successfully executed.

class ModelMonitor (model_id: str)

A class to manage various model monitoring aspects, including concept drift monitoring.

Attributes

_model_id : str

A unique identifier for the model.

_concept_drift_monitor : ConceptDriftMonitor

An instance to monitor concept drift.

Methods

init(self, model_id: str): Initializes the ModelMonitor with a specific model ID. concept_drift: Returns the ConceptDriftMonitor instance.

Initializes the ModelMonitor with a specific model ID.

Args

model_id : str

The unique identifier for the model.

Instance variables

prop concept_drift

Provides access to the ConceptDriftMonitor instance.

Returns

ConceptDriftMonitor

The concept drift monitor instance.