Module markov.api.connectors.specific_connector
Classes
class StorageConnector (credentials: GenericCredential, storage_type: StorageType)-
" Data connectors connect with various data_set sources (MKV/S3/SnowFlake/etc) and get the raw data_set back to the system for column_stats
Initialize a storage connector.
Args
credentials:Credentials- Credential object that contains information on how to connect with this storage
- and credentials.
storage_type:StorageType- What type of storage is this.
Ancestors
- abc.ABC
Subclasses
Static methods
def create_storage_path(**kwargs) ‑> str-
Args
base- str base path of the location
Returns:
Methods
def download(self, paths: List[str])def upload_df(self, df: DataFrame, path: str)-
Args
df:DataFrame path:str
Returns
response of the upload to storage
def upload_file(self, local_file_path: str, storage_upload_path: str)-
Upload file to storage
Args
local_file_path- location of the file on the local file system
storage_upload_path- where to upload file on the storage.
You can get this path by invoking create_storage_path
Returns
Response from the storage server