Module markov.library.mlflow_helper

Functions

def get_load_mlflow_pyfunc_and_predict_script(mlflow_pyfunc_model_path, model_input_csv_path: str)
def load_mlflow_pyfunc_model(uri: str)
def mlflow_pipeline_stage_convertor(markov_stage: Union[MarkovPyfunc, MarkovTransformer, MarkovPredictor])

Classes

class MlflowInferencePipeline

Class representing a pipeline registered via mlflow custom model. We use mlflow to store our inference pipelines because mlflow provides a convenient to store and load any model as a python function. Essentially, we utilize the ability of mlflow to create any custom models. We treat our custom models as a pipeline where each step in the pipeline can be: 1. Pre-processing step (pure python function or a transformer model) 2. Prediction (any supported model) 3. Post-processing step (pure python function)

Methods

def add_pipeline_stage(self, stage: Union[MlflowPyfunc, MlflowTransformer, MlflowModel])
def extend_from_other_pipeline(self, other: MlflowInferencePipeline)

MlflowPipeline can be extended from an existing pipeline such that all steps in existing pipeline are added to the current pipeline.

def register(self, name, code_paths: List[str] = None, pip_requirements=None) ‑> str

Register the pipeline with the given name; Register saves the pipeline as a single mlflow model in the local filesystem

Args

name
The name with which this pipeline needs to be registered
code_paths : List[str]
List of paths to the code that may be required to deserialize the saved model
pip_requirements
Either an iterable of pip requirement strings (e.g. ["scikit-learn", "-r requirements.txt", "-c constraints.txt"]) or the string path to a pip requirements file on the local filesystem (e.g. "requirements.txt")

Returns

The path at which the pipeline is registered

class MlflowModel (name: str, model: Any, flavour: MarkovSupportedFlavours)

Class representing an ML model to be saved via mlflow

Class variables

var flavour : MarkovSupportedFlavours
var model : Any
var name : str

Methods

def save(self, path: str)

Register the model with mlflow at given path and given metadata

Arguments

path (str): The path at which the model needs to be saved via mlflow

class MlflowPipelineConvertor

Abstract class that defines the interface for converting any transformer / inference model to mlflow pipeline

Ancestors

  • abc.ABC

Methods

def to_mlflow_pipeline(self)
class MlflowPyfunc (name: str, pyfunc: Any, additional_kwargs: Dict[str, Any] = None, code_paths: List[str] = None)

Class representing a pure python function to be saved via mlflow

Class variables

var additional_kwargs : Dict[str, Any]
var code_paths : List[str]
var name : str
var pyfunc : Any

Methods

def save(self, path: str)

Saves a particular instance of a pure python function with the given args and kwargs at given path and given metadata

Arguments

path (str): The path at which this pyfunc needs to be stored

class MlflowTransformer (name: str, transformer: Any, flavour: MarkovSupportedFlavours)

Class representing a transformer to be saved via mlflow Transformer is a model which has a "transform" method rather than the predict method

Class variables

var flavour : MarkovSupportedFlavours
var name : str
var transformer : Any

Methods

def save(self, path: str)

Register the transformer with mlflow at given path and given metadata

Arguments

path (str): The path at which the transformer needs to be saved via mlflow