Skip to content

Vault#

lit.sdk.vault #

This module provides functionality for interacting with the Lit vault, including managing and retrieving items stored in the vault.

__doc__ = '\nThis module provides functionality for interacting with the Lit vault, including managing\nand retrieving items stored in the vault.\n' module #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__file__ = '/opt/lit-platform/lit-lib/src/lit/sdk/vault.cpython-312-x86_64-linux-gnu.so' module #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__name__ = 'lit.sdk.vault' module #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__package__ = 'lit.sdk' module #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__test__ = {'VaultItem.details (line 31)': '\n Retrieves detailed information about this vault item from the Lit vault.\n\n Returns:\n (dict | None): A dictionary containing detailed information about this vault item, or None if it does not exist in the vault.\n\n Examples:\n Getting the details of a vault item\n >>> item = VaultItem("contoso", "my_model1")\n >>> item.details\n {\'host\': \'bogdan\', \'project\': \'my_project\', \'runid\': 3, \'epoch\': 1, \'model\': \'model.1.keras\', \'name\': \'my_model1\', \'value\': None, \'color\': None, \'dateAdded\': \'2024-07-12 19:40:19.783233+00:00\', \'workers\': []}\n\n Getting a non-existent vault item\n >>> item = VaultItem("contoso", "my_model0")\n >>> item.details\n None\n ', 'VaultItem.load_model (line 80)': '\n Loads the model.\n\n Returns:\n (tensorflow.keras.models.Model): A model grouping layers into an object with training/inference features.\n \n Examples:\n Load a model.\n\n >>> model = vault_item.load_model()\n >>> model.layers\n [<InputLayer name=input_1, built=True>,\n <Lambda name=lambda, built=True>,\n <Lambda name=lambda_1, built=True>,\n <Flatten name=flatten, built=True>,\n <Flatten name=flatten_1, built=True>,\n <Concatenate name=dense_40c93684-299b-41df-b233-912a47954a84_concatenate, built=True>,\n <Dropout name=dropout, built=True>,\n <Dense name=dense_40c93684-299b-41df-b233-912a47954a84_0, built=True>,\n <GaussianNoise name=gaussian_noise, built=True>,\n <Dropout name=dropout_1, built=True>,\n <Dense name=end_d1657694-e01d-409b-aec8-15b33a9aefbc, built=True>]\n ', 'VaultItem.get_data_from_path (line 120)': '\n Retrieve input data from a given asset path and format it for model prediction.\n\n Args:\n path (Path): The path of the asset\n start (number): Start index [optional]\n stop (number): Stop index [optional]\n\n Returns:\n ([numpy.ndarray]): List of numpy arrays filled with data retrieved from the asset.\n \n Examples:\n Load the model, prepare input from a random index into the first datafile of the first asset used to train the model, make a prediction.\n\n >>> model = vault_item.load_model()\n >>> asset = vault_item.get_assets()[0]\n >>> index = random.randint(0, len(asset.files[0]) - 1)\n >>> x, y = vault_item.get_data_from_path(asset.files[0].path, index)\n >>> model.predict(x)\n array([[0.48219436]], dtype=float32)\n ', 'VaultItem.zeros (line 173)': '\n Creates a single batch of initialized inputs for the underlying model. This can be passed directly into predict.\n\n Returns:\n ([numpy.ndarray]): List of numpy arrays of zero-values that match the expected shape of inputs to the neural network.\n \n Examples:\n Get model, get zeros, make a prediction.\n\n >>> model = vault_item.load_model()\n >>> input_data = vault_item.zeros()\n >>> model.predict(input_data)\n array([[0.49490067]], dtype=float32)\n ', 'VaultItem.rename (line 191)': '\n Renames this vault item to the specified name.\n\n Args:\n new_name (str): The desired new name for this vault item.\n\n Returns:\n (dict): A dictionary containing the status and message of the rename operation.\n\n Examples:\n Renaming a vault item\n >>> item = VaultItem("contoso", "my_model1")\n >>> item.rename("my_model2")\n {\'status\': \'SUCCESS\', \'message\': \'Vault item my_model1 has been renamed my_model2.\'}\n >>> item.name\n \'my_model2\'\n\n Renaming a non-existent vault item\n >>> item = VaultItem("contoso", "my_model0")\n >>> item.rename("new_name")\n {\'status: \'ERROR\', \'message\': \'item my_model0 not found\'}\n >>> item.name\n \'my_model0\'\n\n Renaming a vault item to the name of another vault item\n >>> item = VaultItem("contoso", "my_model1")\n >>> item.rename("my_model1")\n {\'status: \'ERROR\', \'message\': \'name already exists\'}\n >>> item.name\n \'my_model1\'\n ', 'get_vault_items (line 236)': '\n Retrieves a list of vault items for the specified team from the Lit vault.\n\n Args:\n team (str): The name of the team to retrieve vault items for.\n\n Returns:\n list[VaultItem]: A list of VaultItem objects representing the vault items for the specified team.\n\n Examples:\n >>> get_vault_items("contoso")\n [VaultItem(team=\'contoso\', name=\'spy_e1\')]\n ', 'import_model (line 255)': '\n Import a Tensorflow model into the Lit vault.\n\n Args:\n team (str): The name of the team to retrieve vault items for.\n path (str): The path to the saved model.\n name (str): The name of the new model (optional).\n\n Returns:\n VaultItem: A VaultItem objects representing the vault item for the imported model.\n\n Examples:\n >>> import_model("contoso", "/data/raw/import/model.h5")\n VaultItem(team=\'contoso\', name=\'new_model\')\n '} module #

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

Asset #

The data file prepared and optimized for training a neural network. The result of the data build process.

Examples:

>>> asset = Asset("contoso", "schema_1707673805399")
>>> len(asset)
11648
>>> asset.datasets
['100_1_min', 'per_min', 'rr_atr_2_2_1800']

__annotations__ = {'team': 'str', 'name': 'str', 'path': 'Path', 'datasets': 'list[str]'} class #

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ = '\n The data file prepared and optimized for training a neural network. The result of the data build process.\n\n Examples:\n >>> asset = Asset("contoso", "schema_1707673805399")\n >>> len(asset)\n 11648\n >>> asset.datasets\n [\'100_1_min\', \'per_min\', \'rr_atr_2_2_1800\']\n ' class #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__module__ = 'lit.sdk.data.asset' class #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property #

list of weak references to the object

files property #

Retrieves individual data files from the asset.

Returns:

Type Description
list[PaAssetFileth]

A list asset file objects.

Examples:

>>> asset = Asset("contoso", "schema_1707673805399")
>>> asset.files
[Path('/data/contoso/assets/schema_1707673805399/schema_1707673805399_test.h5'), Path('/data/contoso/assets/schema_1707673805399/schema_1707673805399_train_0.h5')]

__init__(team, name) method descriptor #

Initializes a new instance of Asset.

Parameters:

Name Type Description Default
team str

The team which owns the Asset.

required
name str

The name of the Asset.

required

__len__() method descriptor #

Total number of unique data samples in the test/train dataset.

BaseAdapter #

get_bar_daterange(date, unit='tick', size=1, future=False) cached #

Returns#

[ price_open, price_high, price_low, price_close, trade_count, vol_sum, vol_mean, vol_max, timestamp ]

Parameters#

date : pd.Timestamp, optional The day to begin retrieve bars

str, optional

The unit of the bars; one of "tick", "sec", "min", "hour", "day"

boolean, optional

By default, look back into the past (computing features) to retrieve bars. Override to 'True' to retrieve from the future (computing labels).

get_bars(index=None, count=1, unit='tick', size=1, future=False, prevent_lookahead_bias=True) cached #

Returns#

[ price_open, price_high, price_low, price_close, trade_count, vol_sum, vol_mean, vol_max ]

Parameters#

index : int, optional The index at which to begin retrieving bars

int, optional

The number of bars to retrieve

str, optional

The unit of the bars; one of "tick", "sec", "min", "hour", "day"

boolean, optional

By default, look back into the past (computing features) to retrieve bars. Override to 'True' to retrieve from the future (computing labels).

get_date(date) #

Get all data for a single trading date.

Parameters:

Name Type Description Default
date

A date that can be converted to pd.Timestamp (string, datetime, pd.Timestamp)

required

Returns:

Type Description

DataFrame with all records for that trading date

Examples:

>>> df = adapter.get_date("2025-02-03")
>>> df = adapter.get_date(pd.to_datetime("2025-02-03"))

get_max_values() #

returns open, high, low, close, count, vol_sum, vol_sum / count, vol_max,

get_minute_bars(date, size=1, future=False) #

Get precomputed minute bars for a trading date using get_bar_daterange.

Parameters:

Name Type Description Default
date

A date that can be converted to pd.Timestamp

required
size

Bar size in minutes (default 1 for 1-minute bars)

1
future

If False (default), look back into past (features). If True, look forward (labels).

False

Returns:

Type Description

DataFrame with precomputed bars: [open, high, low, close, count, vol_sum, vol_mean, vol_max, timestamp]

Examples:

>>> bars = adapter.get_minute_bars("2025-02-03")  # 1-minute bars, past data
>>> bars = adapter.get_minute_bars("2025-02-03", 5)  # 5-minute bars
>>> bars = adapter.get_minute_bars("2025-02-03", future=True)  # For label computation

Canvas #

Represents a model canvas, the no-code representation of a machine learning model.

inputs property #

Retrieves the names of the inputs to the canvas.

Returns:

Type Description
list[str]

A list input names.

Examples:

>>> canvas = Asset("contoso", "demo")
>>> canvas.inputs
[['90_1daybar_lp%'],
['RSI14TA_day'],
['vwap2060_day'],
['20sma_day'],
['ATR14_day']]

name = name instance-attribute #

The name of the canvas.

outputs property #

Retrieves the names of the outputs from the canvas.

Returns:

Type Description
list[str]

A list input names.

Examples:

>>> canvas = Asset("contoso", "demo")
>>> canvas.outputs
[{'component_name': '',
  'label_index': 4,
  'gaussian_noise': 0.5,
  'dropout': 0.5,
  'class_weighting': 1,
  'activation': 'auto',
  'feature': 'long_updown'}]

path instance-attribute #

The path to the JSON representation of the canvas on disk.

team = team instance-attribute #

The name of a team.

__init__(team, name=None, path=None) #

Initializes a new instance of Canvas.

Parameters:

Name Type Description Default
team str

The team which owns the Canvas.

required
name str

The name of the Canvas.

None

duplicate() #

Duplicates a canvas.

Raises:

Type Description
LitModelError

If an error occurred while duplicating the canvas.

Returns:

Type Description
Canvas

The duplicated canvas.

get_available_components() #

List available components that can be added to a design canvas.

Raises:

Type Description
LitModelError

If an error occurred while listing the available components.

Returns:

Type Description
list

a list of components.

Examples:

>>> canvas = Canvas('contoso', 'demo')
>>> available_components = canvas.get_available_components()

remove() #

Removes a canvas permanently.

Raises:

Type Description
LitModelError

If an error occurred while deleting the canvas.

train(device, session_name=None, temp_canvas=False) #

Starts a new experiment.

Parameters:

Name Type Description Default
device str

The name of a device; e.g. gpu4

required

Raises:

Type Description
LitModelError

If an error occurred while starting the experiment.

Returns:

Name Type Description
session_name str

The name of the screen session running the experiment.

transpile() #

Compiles a design canvas.

Raises:

Type Description
LitModelError

If an error occurred while [ACTION].

Returns:

Type Description
Functional

A Tensorflow Keras model.

Examples:

Compile the canvas named 'demo' for the team named 'contoso' to a Tensorflow Keras model.

>>> canvas = Canvas('contoso', 'demo')
>>> model = canvas.transpile()
>>> model
<keras.src.models.Functional at 0x7f3ed7193850>

LitVaultError #

Bases: lit.sdk.errors.LitError

A custom exception class for Lit vault errors.

__annotations__ = {} class #

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ = 'A custom exception class for Lit vault errors.' class #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__module__ = 'lit.sdk.errors' class #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__init__(*args, code=None) method descriptor #

Initializes a new instance of LitVaultError.

Parameters:

Name Type Description Default
*args object

Variable number of arguments to pass to the base class constructor.

required
code int | None

The error code. Defaults to None.

None

VaultItem #

Represents a single item in the Lit vault.

__annotations__ = {'team': 'str', 'name': 'str'} class #

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__doc__ = 'Represents a single item in the Lit vault.' class #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__module__ = 'lit.sdk.vault' class #

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property #

list of weak references to the object

canvas property #

Retrieves design canvas for the vaulted model.

Returns:

Type Description
dict | None

The design canvas.

details property #

Retrieves detailed information about this vault item from the Lit vault.

Returns:

Type Description
dict | None

A dictionary containing detailed information about this vault item, or None if it does not exist in the vault.

Examples:

Getting the details of a vault item

>>> item = VaultItem("contoso", "my_model1")
>>> item.details
{'host': 'bogdan', 'project': 'my_project', 'runid': 3, 'epoch': 1, 'model': 'model.1.keras', 'name': 'my_model1', 'value': None, 'color': None, 'dateAdded': '2024-07-12 19:40:19.783233+00:00', 'workers': []}

Getting a non-existent vault item

>>> item = VaultItem("contoso", "my_model0")
>>> item.details
None

__init__(team, name) method descriptor #

Initializes a new instance of VaultItem.

Parameters:

Name Type Description Default
team str

The team which owns the VaultItem.

required
name str

The name of the VaultItem.

required

get_data_from_path(path, start=None, stop=None, step=1) method descriptor #

Retrieve input data from a given asset path and format it for model prediction.

Parameters:

Name Type Description Default
path Path

The path of the asset

required
start number

Start index [optional]

None
stop number

Stop index [optional]

None

Returns:

Type Description
[ndarray]

List of numpy arrays filled with data retrieved from the asset.

Examples:

Load the model, prepare input from a random index into the first datafile of the first asset used to train the model, make a prediction.

>>> model = vault_item.load_model()
>>> asset = vault_item.get_assets()[0]
>>> index = random.randint(0, len(asset.files[0]) - 1)
>>> x, y = vault_item.get_data_from_path(asset.files[0].path, index)
>>> model.predict(x)
array([[0.48219436]], dtype=float32)

load_model() method descriptor #

Loads the model.

Returns:

Type Description
Model

A model grouping layers into an object with training/inference features.

Examples:

Load a model.

>>> model = vault_item.load_model()
>>> model.layers
[<InputLayer name=input_1, built=True>,
<Lambda name=lambda, built=True>,
<Lambda name=lambda_1, built=True>,
<Flatten name=flatten, built=True>,
<Flatten name=flatten_1, built=True>,
<Concatenate name=dense_40c93684-299b-41df-b233-912a47954a84_concatenate, built=True>,
<Dropout name=dropout, built=True>,
<Dense name=dense_40c93684-299b-41df-b233-912a47954a84_0, built=True>,
<GaussianNoise name=gaussian_noise, built=True>,
<Dropout name=dropout_1, built=True>,
<Dense name=end_d1657694-e01d-409b-aec8-15b33a9aefbc, built=True>]

rename(new_name) method descriptor #

Renames this vault item to the specified name.

Parameters:

Name Type Description Default
new_name str

The desired new name for this vault item.

required

Returns:

Type Description
dict

A dictionary containing the status and message of the rename operation.

Examples:

Renaming a vault item

>>> item = VaultItem("contoso", "my_model1")
>>> item.rename("my_model2")
{'status': 'SUCCESS', 'message': 'Vault item my_model1 has been renamed my_model2.'}
>>> item.name
'my_model2'

Renaming a non-existent vault item

>>> item = VaultItem("contoso", "my_model0")
>>> item.rename("new_name")
{'status: 'ERROR', 'message': 'item my_model0 not found'}
>>> item.name
'my_model0'

Renaming a vault item to the name of another vault item

>>> item = VaultItem("contoso", "my_model1")
>>> item.rename("my_model1")
{'status: 'ERROR', 'message': 'name already exists'}
>>> item.name
'my_model1'

zeros() method descriptor #

Creates a single batch of initialized inputs for the underlying model. This can be passed directly into predict.

Returns:

Type Description
[ndarray]

List of numpy arrays of zero-values that match the expected shape of inputs to the neural network.

Examples:

Get model, get zeros, make a prediction.

>>> model = vault_item.load_model()
>>> input_data = vault_item.zeros()
>>> model.predict(input_data)
array([[0.49490067]], dtype=float32)

import_model(team, path, features=[], name=None) method descriptor #

Import a Tensorflow model into the Lit vault.

Parameters:

Name Type Description Default
team str

The name of the team to retrieve vault items for.

required
path str

The path to the saved model.

required
name str

The name of the new model (optional).

None

Returns:

Name Type Description
VaultItem VaultItem

A VaultItem objects representing the vault item for the imported model.

Examples:

>>> import_model("contoso", "/data/raw/import/model.h5")
VaultItem(team='contoso', name='new_model')

lit_error_handler(error_type=LitError) method descriptor #

A decorator function that catches and re-raises exceptions with a custom error type.

Parameters:

Name Type Description Default
error_type Type[LitError]

The type of exception to raise. Defaults to LitError.

LitError

Returns:

Type Description
Callable

A wrapper function that catches exceptions and raises the specified error type.

Examples:

>>> @lit_error_handler()
... def raises_error():
...     assert False
...
>>> raises_error()
Traceback (most recent call last):
File "/opt/lit/src/lit/sdk/errors.py", line 130, in wrapper
File "<stdin>", line 3, in raises_error
AssertionError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/lit/src/lit/sdk/errors.py", line 132, in wrapper
lit.sdk.errors.LitError: An error occurred in function 'raises_error'