evnrg.storage package

Submodules

evnrg.storage.datastorage module

class evnrg.storage.datastorage.DataHandler(storage_info: evnrg.storage.datastorage.StorageInfo)[source]

Bases: object

cache_dir
cleanup()[source]
container
driver
read_data(obj_name: str, fmt: str = 'parquet', read_fn=None, arguments: dict = {})[source]
ready
temp
upload_data(df: pandas.core.frame.DataFrame, obj_path: str, formats: str = 'parquet', keep_temp: bool = False, enable_cleanup: bool = True)[source]
use_bucket(bucket: str, create: False)[source]
class evnrg.storage.datastorage.DataResource(storage_info: evnrg.storage.datastorage.StorageInfo)[source]

Bases: object

handler
info
class evnrg.storage.datastorage.StorageInfo[source]

Bases: tuple

A NamedTuple that tells the simulation how to handle data.

key

ID or login key for the cloud account. For example, Google Cloud Services uses an IAM email address, and can be found in the ‘client_email’ key of a generated JSON file. (See [Google Cloud’s IAM documentation](https://cloud.google.com/iam/docs/) for more.) For local storage, the key is the base path that will be used while emulating cloud storage. Defaults to ~/.local/share/evnrg/local-object-storage.

Type:str
secret

Secret key or passphrase for the cloud account. For Goggle Cloud, it will be in the ‘private_key’ field in the JSON file. Defaults to empty string (‘’).

Type:str
bucket

The name of the container (or bucket) to look for data and store it in. Defaults to ‘evnrg-default’. For local storage, this folder will exist under the key: <key>/<bucket>.

Type:str
provider

The provider to use. Takes any provider string object that [Apache LibCloud](https://libcloud.readthedocs.io/en/latest/supported_providers.html#id195) takes. Additional provider shotcuts are provided as well: * Google Cloud Storage: ‘gcs’, ‘google’ * Amazon S3: ‘s3’, ‘amazon’ * Backblaze B2: ‘b2’, ‘backblaze’ * Digital Ocean Spaces: ‘do’, ‘spaces’, ‘digitalocean’ * Microsoft Azure: ‘azure’, ‘microsoft’ Defaults to ‘local’.

Type:str
cahce_dir

The local cahce directory for use when dumping dataframes for upload, and downloading dataframes for use. Defaults to ~/.cache/evnrg.

Type:str
create_bucket

Specifies if a bucket should be created if it does not exist. Defaults to True.

Type:bool
bucket

Alias for field number 2

cache_dir

Alias for field number 4

create_bucket

Alias for field number 5

key

Alias for field number 0

provider

Alias for field number 3

secret

Alias for field number 1

class evnrg.storage.datastorage.UploadResult[source]

Bases: tuple

Contains data about the uploaded object.

uid

The object’s uuid.

Type:str
filetype

The object’s filetype. Will generally be one of ‘parquet’, ‘json’, ‘records’, or ‘csv’.

Type:str
cache_path

The path to the temporary file.

Type:str
remote_path

The object’s remote object name.

Type:str
obj

The LibCloud storage object.

Type:libcloud.storage.base.Object
cache_path

Alias for field number 2

filetype

Alias for field number 1

obj

Alias for field number 4

remote_path

Alias for field number 3

uid

Alias for field number 0

Module contents