Module markov.api.recording.embeddings.embedding_recorder
Classes
class EmbeddingRecorder (name: str, dataset_id: str, notes: str = '', batch_size=1000)-
Embedding Recorder takes in the configuration of a embedding recording. Embedding Recording is used to log custom user tuned embeddings into the system Embedding recorder creates and stores EmbeddingRecording.
Args
name- Name of the embedding recording
notes- Notes/Description for the embedding recording
dataset_id- The dataset id against which we are storing the custom embedding
batch_size:int- how many embeddings should recorder send in a single batch. Limit the batch_size to
smaller number based on the bandwidth of your internet connection. Default is 1000
Instance variables
prop ds_columnsprop embedding_id-
Recording ID that uniquely identifies this recording.
Returns
A recording ID that uniquely identifies this recording.
prop num_records-
Returns: Number of records processed through the recorder and dispatched to MarkovML
Methods
def add_embedding_record(self, dataset_record: List, embedding_record: List)-
Add an embedding record to send to MarkovML backend. The recorder would take care of dispatching records to MarkovML in multiple batches by multi-threading.
Args
- dataset_record : List of values in the dataset record to map to this embedding
embedding_record- Embeddings as a list of values for this record
Returns
None
def finish(self) ‑> EmbeddingRecordingFinishResponse-
This method should be called to finish the recording once all the records are dispatched. This takes care of closing the recorder with MarkovML backend and trigger the computation of metrics.
Returns
EmbeddingRecordingFinishResponse containing the embedding_id
def register(self) ‑> EmbeddingRecorder-
Creates EmbeddingRecorder and registers it with Markov. Without this the data will not be registered with markovML.
Returns
EmbeddingRecorder