Module markov.api.recording.experiments.processors.experiment_recording_processor

Classes

class ExperimentRecordingProcessor (recording_id: str, spinner: halo.halo.Halo = None, dispatcher=None, alerts: List[ExperimentAlert] = None, batch_size: int = 10, batch_processing_interval: int = 5, register_batch_timeout: int = 10, sync_data_timeout: int = 300)

The processor to asynchronously send hyper-parameter recordings to the markov backend

Initializes the processor with given arguments, as well the underlying queue and threads being used

Args

recording_id

The recording id of the hyper parameter recording

dispatcher

The dispatcher used for dispatching recordings depending on online/offline mode

batch_size

The number of recordings to be dispatched in one call

batch_processing_interval

the time interval in which the dispatcher will be called once

register_batch_timeout

timeout to fail one call to the dispatcher

sync_data_timeout

timeout to fail syncing all data using the dispatcher

Class variables

var DEFAULT_BATCH_SIZE

var DEFAULT_EXPERIMENT_REGISTER_BATCH_TIMEOUT

var DEFAULT_EXPERIMENT_SYNC_TIMEOUT

var DEFAULT_PROCESS_BATCH_INTERVAL

Methods

def add_record(self, hyperparam_message: Dict)

def start(self)

Starts the background tasks - consumer thread to periodically read the queue

def stop(self) ‑> bool

Gracefully stops the background tasks and makes sure queue is consumed unless timed out