Module markov.api.model.recording

Classes

class ModelRecorder (recorder_config: ModelRecordingConfig, batch_size=1000, **kwargs)

Model Recorder takes in the configuration of a model recording. ModelRecording is used to document & eventually analyze the performance of the model across multiple dimensions. Model recorder stores ModelInferenceRecords. ModelInferenceRecords broadly contains the ground truth value of the input & output of the model for that input. In addition, additional metadata can associated with an inference record for bookkeeping.

Args

recorder_config : ModelRecordingConfig

This configuration contains the important details

of this model recording.

batch_size : int

now many recordings should recorder send in a single batch. Limit the batch_size to

smaller number based on the bandwidth of your internet connection. Default is 1000 **kwargs ():

Static methods

def get_recordings(limit=10)

Get the list of all the recordings that have been registered for this customer in decreasing order.

Args

limit : int

Number of recordings to be returned.

Returns

List of all the recording registered with MarkovML for this customer.

Instance variables

prop num_records

prop recording_id

Recording ID that uniquely identifies this recording.

Returns

A recording ID that uniquely identifies this recording.

Methods

def add_record(self, inference_record: Union[SingleTagInferenceRecord, MultiTagInferenceRecord])

def finish(self) ‑> ModelRecordingFinishResponse

Once all records are added to the recording, call this method to finish recording and trigger computation of metrics. Since metric computation takes time, Client needs to poll to get the recording details on CLI. In future a URL will be returned that will containing the metrics results of this recording when finished.

Returns:

def register(self) ‑> ModelRecorder

Creates ModelInferenceRecorder and registers it with Markov.

Returns

ModelInferenceRecorder

class ModelRecording (recording_id: str)

Result of the ModelRecordingRUn when available.

Get the recording Object.

Args

recording_id : str

A valid recording id for which information is requested.

Methods

def compute_metric(self, force_recompute: bool = False)

Trigger computation of metrics on MarkovML backend

Args

force_recompute : bool

recompute again even if recording is available on the backend.

Returns

dict containing response

def info(self, pretty_display=True) ‑> Dict

Return information about this recording registered with MarkovML when this recording was registered.

Args

pretty_display : bool

Set to true if print to StdOut is desired.

Returns

An attribute dictionary of results.

def results(self, pretty_display=True) ‑> Dict

Return the results of this recording if complete.

Args

pretty_display : bool

Set to true if print to stdout is desired.

Returns

An Attribute Dictionary of results.