Module markov.api.schemas.model_recording

Classes

class EvaluationRecording (name: str, model_id: str, note: str = '', dataset_id: Optional[str] = '', info: dict = <factory>)

The data model to store metadata for evaluations

Class variables

var dataset_id : Optional[str]

Misc information we need to save about this recording with handlers. it's key->value format

var info : dict

var model_id : str

details about this evaluation run that might be relevant for the

var name : str

The id of the model which this evaluation belongs to

var note : str

The dataset id used for this evaluation

Static methods

def create_from_dict(value: dict) ‑> EvaluationRecording

Create this object from the serialized (dictionary) representation of this object where the key's are attributes and values are attribute values.

Args

value : Dict

Dictionary serialized value of this object.

Returns

EvaluationRecording

def create_from_json(value: str) ‑> EvaluationRecording

Create EvaluationRecording object from serialized JSON.

Args

value : str

JSON string that is serialized representation of this object.

Returns

EvaluationRecording object

def from_dict(kvs: Union[dict, list, str, int, float, bool, ForwardRef(None)], *, infer_missing=False) ‑> ~A

def from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) ‑> ~A

def schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) ‑> dataclasses_json.mm.SchemaF[~A]

Methods

def get_dict(self) ‑> dict

def get_json(self) ‑> str

def to_dict(self, encode_json=False) ‑> Dict[str, Union[dict, list, str, int, float, bool, ForwardRef(None)]]

def to_json(self, *, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Union[int, str, ForwardRef(None)] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) ‑> str

class ModelRecordingConfig (name: str, user_model_id: str, user_data_id: str, model_name: str, model_class: str, note: str = '', info: dict = <factory>)

Configuration for Model Recording

Class variables

var info : dict

var model_class : str

details about this model run that might be relevant for the

var model_name : str

Defines the model class (Regression/ Classification/ Ranking etc)

var name : str

Identifier provided by the user to identify their model

var note : str

Misc information we need to save about this recording with handlers. it's key->value format

var user_data_id : str

Human readable name of the model

var user_model_id : str

Unique identifier associated with the dataset that has gathered model inference/ground truth results against

Static methods

def create_from_dict(value: dict) ‑> ModelRecordingConfig

Create this object from the serialized (dictionary) representation of this object where the key's are attributes and values are attribute values.

Args

value : Dict

Dictionary serialized value of this object.

Returns

ModelRecordingConfig

def create_from_json(value: str) ‑> ModelRecordingConfig

Create ModelRecordingConfig object from serialized JSON.

Args

value : str

JSON string that is serialized representation of this object.

Returns

ModelRecordConfig object

def from_dict(kvs: Union[dict, list, str, int, float, bool, ForwardRef(None)], *, infer_missing=False) ‑> ~A

def from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) ‑> ~A

def schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) ‑> dataclasses_json.mm.SchemaF[~A]

Methods

def get_dict(self) ‑> dict

def get_json(self) ‑> str

def to_dict(self, encode_json=False) ‑> Dict[str, Union[dict, list, str, int, float, bool, ForwardRef(None)]]

def to_json(self, *, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Union[int, str, ForwardRef(None)] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) ‑> str

class MultiTagInferenceRecord (urid: str, meta_data: Optional[List[RecordMetaData]] = <factory>, custom_metrics: Optional[List[RecordCustomMetric]] = <factory>, mir_list: List[SingleTagInferenceRecord] = <factory>)

Useful for tagging Inference records where there is a single object (such as an image) and within that image there are multiple tags (objects recognized). MultiTagInferenceRecord is a collection of ModelInferenceRecords, where each inference record captures the actual/predicted/score of each tag.

Class variables

var custom_metrics : Optional[List[RecordCustomMetric]]

var matches

var meta_data : Optional[List[RecordMetaData]]

var mir_list : List[SingleTagInferenceRecord]

var urid : str

meta data_set (optional) to be associated with this inference record. Meta data_set is any additional information that user wants to provide to help them in their analysis.

Static methods

def create_from_dict(value: dict) ‑> MultiTagInferenceRecord

Deserialize the dictionary representation of this object back to an InferenceRecord.

Args

value : dict

Dict representation of this record.

Returns

InferenceRecord Instance.

def create_from_json(value: str) ‑> MultiTagInferenceRecord

Deserialize the json representation of this object back to an InferenceRecord.

Args

value : str

JSON representation of this record

Returns

InferenceRecord Instance

def from_dict(kvs: Union[dict, list, str, int, float, bool, ForwardRef(None)], *, infer_missing=False) ‑> ~A

def from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) ‑> ~A

def schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) ‑> dataclasses_json.mm.SchemaF[~A]

Methods

def add_evaluation(self, actual: Any, predicted: Any, score: float)

Add a tag evaluation to this Mult-Tag inference record for storing/computing the metrics

Args

actual : Any

actual ground truth tag associated

predicted : Any

Predicted tag value

score : float

confidence of prediction or probability score of prediction

Returns

void

def add_meta_data(self, label: str, value: Any, meta_type: RecordMetaType)

Args

label : str

associated label for a metadata to be associated with the record.

MetadataType defines type of information sent so that Frontend can render it correctly

value : str

value associated with this

meta_type : RecordMetaType

supported meta_types for our FE to render the meta_type correctly.

Returns

void

def add_record_meta_data(self, rec_md: RecordMetaData)

Add the instance of RecordMetaData directly to the InferenceRecord

Args

rec_md : RecordMetaData

RecordMetaData object captures the key-value relationship and the meta_data

type information

Returns

void

def compute_tag_ratio(self) ‑> RecordCustomMetric

def get_dict(self) ‑> Dict

Serialize this object to a dictionary.

Returns

Dictionary of this object.

def get_json(self) ‑> str

Serialize this object to JSON

Returns

JSON string of this object

def to_dict(self, encode_json=False) ‑> Dict[str, Union[dict, list, str, int, float, bool, ForwardRef(None)]]

def to_json(self, *, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Union[int, str, ForwardRef(None)] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) ‑> str

class RecordCustomMetric (label: str, value: Union[float, int])

Custom Metric that user can associate with the record for Markov to compute custom statistics

Class variables

var label : str

var value : Union[float, int]

Static methods

def from_dict(kvs: Union[dict, list, str, int, float, bool, ForwardRef(None)], *, infer_missing=False) ‑> ~A

def from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) ‑> ~A

def schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) ‑> dataclasses_json.mm.SchemaF[~A]

Methods

def to_dict(self, encode_json=False) ‑> Dict[str, Union[dict, list, str, int, float, bool, ForwardRef(None)]]

def to_json(self, *, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Union[int, str, ForwardRef(None)] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) ‑> str

class RecordMetaData (label: str, value: Union[Dict, str], meta_type: RecordMetaType)

RecordMetaData contains additional information that user wants to send with each record for MarkovML to register. The information can be one of the RecordMetaType ImageUrl: pointing to an ImageURL hosted at your server or S3. Please make sure CORS Is disabled if you want us to render it the information. Text: Free form text

URI: A URI associated with data_set to href it back

Class variables

var label : str

value of this metadata

var meta_type : RecordMetaType

var value : Union[Dict, str]

Type of this meta data_set

Static methods

def from_dict(kvs: Union[dict, list, str, int, float, bool, ForwardRef(None)], *, infer_missing=False) ‑> ~A

def from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) ‑> ~A

def schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) ‑> dataclasses_json.mm.SchemaF[~A]

Methods

def to_dict(self, encode_json=False) ‑> Dict[str, Union[dict, list, str, int, float, bool, ForwardRef(None)]]

def to_json(self, *, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Union[int, str, ForwardRef(None)] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) ‑> str

class RecordMetaType (value, names=None, *, module=None, qualname=None, type=None, start=1)

Supported MetaData type

Ancestors

• enum.Enum

Class variables

var Histogram : Final

var ImageUrl : Final

This will be on FE as a clickable URI

var Text : Final

Represents an imageURL it will be rendered on FE. Please make sure there is no CORS restriction.

var Uri : Final

Plot this data as a histogram

class SingleTagInferenceRecord (inferred: Any, actual: Any, score: float, urid: str, meta_data: Optional[List[RecordMetaData]] = <factory>, custom_metrics: Optional[List[RecordCustomMetric]] = <factory>)

Inference record converts user data_set into a recording format that MarkovML can understand.

Class variables

var actual : Any

Score for this prediction

var custom_metrics : Optional[List[RecordCustomMetric]]

var inferred : Any

The actual value or the ground truth value of this model

var meta_data : Optional[List[RecordMetaData]]

var score : float

User maintained record id that reference back this particular record back to their dataset for analysis.

var urid : str

meta data_set (optional) to be associated with this inference record. Meta data_set is any additional information that user wants to provide to help them in their analysis.

Static methods

def create_from_dict(value: dict) ‑> SingleTagInferenceRecord

Deserialize the dictionary representation of this object back to an InferenceRecord.

Args

value : dict

Dict representation of this record.

Returns

InferenceRecord Instance.

def create_from_json(value: str) ‑> SingleTagInferenceRecord

Deserialize the json representation of this object back to an InferenceRecord.

Args

value : str

JSON representation of this record

Returns

InferenceRecord Instance

def from_dict(kvs: Union[dict, list, str, int, float, bool, ForwardRef(None)], *, infer_missing=False) ‑> ~A

def from_json(s: Union[str, bytes, bytearray], *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw) ‑> ~A

def schema(*, infer_missing: bool = False, only=None, exclude=(), many: bool = False, context=None, load_only=(), dump_only=(), partial: bool = False, unknown=None) ‑> dataclasses_json.mm.SchemaF[~A]

Methods

def add_custom_metric(self, label: str, value: Union[int, float])

Custom metrics help users evaluate models based on external business metrics they care about. For example, if the user wants to measure how many times a tweet identified as "fun" had a "dog" in it. They can create a custom metric with name "dog_fun_count" and value as number of times "dog" appeared in a tweet.

Args

label : str

Name of the custom metric that you need to track for this evaluation

value : Union[Float,Int]

value associated with the label. It has to be numeric.

Returns:

def add_meta_data_instance(self, key: str, value: Any, meta_type: RecordMetaType)

Add a single meta_data instance with key as a string type and value (any)

Args

key : str

name or the key of the meta_data field you want to store

value : Any

value associated with the key

meta_type : RecordMetaType

type associated with the meta data instance we are adding.

Returns

None

def add_record_meta_data(self, rec_md: RecordMetaData)

def get_dict(self) ‑> dict

Serialize this object to a dictionary.

Returns

Dictionary of this object.

def get_json(self) ‑> str

Serialize this object to JSON

Returns

JSON string of this object

def to_dict(self, encode_json=False) ‑> Dict[str, Union[dict, list, str, int, float, bool, ForwardRef(None)]]

def to_json(self, *, skipkeys: bool = False, ensure_ascii: bool = True, check_circular: bool = True, allow_nan: bool = True, indent: Union[int, str, ForwardRef(None)] = None, separators: Optional[Tuple[str, str]] = None, default: Optional[Callable] = None, sort_keys: bool = False, **kw) ‑> str