Notebook Database Documentation

The acorn notebook database is saved as a JSON-serialized dictionary, which we call a “database”. A more accurate name would be a key-value store. The database has the following attributes:

• entities: these are method calls or object instantiation logs. The value of the attribute is a dictionary with UUID keys (see next point) and values having the “entry” format described below.
• uuids: uses Universally Unique Identifier (UUID) strings to identify each of the objects that are referenced in the entities entries. This dictionary has UUID keys and values are dictionaries returned by the ACORN Object Descriptor Interface.

Entity Entry Format

Each entry in the database can either be a method/function call, or a constructor call to initialize an object. For constructor calls, we log only the __new__ methods; the __init__ methods are automatically called as well, but they do not produce a log in the database. Entries are JSON dictionaries with the following attributes:

• args: is a dictionary. It has an __ attribute for the positional arguments, and then additional attributes for each keyword argument.
• analysis: if a method call is configured to trigger analysis, this attribute will be a dictionary returned by Method Call Analysis for the specific object type.
• elapsed: how many ticks elapsed in order to run the method. We create the entry before calling the method and then update the elapsed time once the result has returned. For constructors, this includes the time required to call __init__.
• start: time at which the method/constructor call was started.
• returns: object representation for the result of the function call (see the discussion on object representations below).
• method: the FQDN of the method being called. This can be a constructor, instance method, static method, class method, function, built-in, etc.
• error: if the call to the constructor/function raised an unhandled exception, this is the error message of the exception. The exception is still raised to the main notebook thread, we just intercept it to create a log.

Short Representations for Objects

When a new object is detected that hasn’t been handled before, it gets passed to the tracker() method, which determines how best to represent the object.

• For simple object types (scalars), the object is returned.
• For collections in which every item is a simple scalar type, the len, min and max as well as the type of the collection are extracted and pretty-printed in a string.
• For collections of non-scalar types, tracker() is called for each member of the collection.
• If the object has type type, then it’s name is returned.
• lambda functions (have type LambdaType) are represented by lambda(args...), where the “args” list is extracted from the __code__ special attribute. If the lambda object has a __fqdn__ special attribute, it is used instead.
• For objects of type FunctionType or MethodType, the __name__ attribute is returned.
• numpy.ufunc instances have their FQDN returned.
• In all other cases, a acorn.logging.database.Instance object is initialized with the object. It is then represented by a uuid.uuid4() value and its description is handled by ACORN Object Descriptor Interface.

Database Naming Conventions: Projects and Tasks

acorn exposes a method acorn.set_task() that sets a project and task name for a live session. These values can be changed during a session to work on a different database. The JSON file name is project.task.json and gets stored in the database directory specified by the Global and Package Configuration in ACORN. The database can also be set to be not writeable, in which case the entries are just logged in-memory. This is useful for unit-testing and debugging.

API Documentation

Methods and classes for creating a JSON database from a tree of calls to methods with various objects passed as arguments.

class acorn.logging.database.Instance(pid, obj)

Represents a class instance in the current session which can be passed as an argument to method calls, or have unbound methods called in it.

Parameters:pid (int) – python memory address (returned by id()).

uuid

str – uuid.uuid4() for the object.

obj

original object instance that this represents.

describe()

Returns a dictionary describing the object based on its type.

class acorn.logging.database.TaskDB(dbdir=None)

Represents the database for a single task.

entities

dict – keys are entity ids (fqdn or uuid); values are a list of entries generated for that entity during this task.

uuids

dict – keys are uuid values for class instances; values are dicts with attributes describing the class instance’s origin.

dbpath

str – full path to the database JSON file for this task database.

lastsave

float – timestamp since the last time the DB was saved.

static get_option(option, default=None, cast=None)

Returns the option value for the specified acorn database option.

load()

Deserializes the database from disk.

log_uuid(uuid)

Logs the object with the specified uuid to self.uuids if possible.

Parameters:uuid (str) – string value of uuid.uuid4() value for the object.

record(ekey, entry, diff=False)

Records the specified entry to the key-value store under the specified entity key.

Parameters:

• ekey (str) – fqdn/uuid of the method/object to store the entry for.
• entry (dict) – attributes and values gleaned from the execution.
• diff (bool) – when True, the “c” element of entry will be diffed against previous entries under the same ekey if their method (attribute “m”) matches.

save(force=False)

Serializes the database file to disk.

Parameters:force (bool) – when True, the elapsed time since last save is ignored and the database is saved anyway (subject to global writeable setting).

acorn.logging.database.active_db()

Returns the active TaskDB for the current project and task.

acorn.logging.database.cleanup()

Saves all the open databases to JSON so that the kernel can be shut down without losing in-memory collections.

acorn.logging.database.dbdir = None

str – full path to the directory where the databases are being stored. If this is overwritten programatically, then the global settings for acorn will not be checked.

acorn.logging.database.dbs = {}

dict – keys are tuple (project, task), values are the TaskDB instances for the specified project and task.

acorn.logging.database.list_tasks(target=None)

Returns a list of all the projects and tasks available in the acorn database directory.

Parameters:target (str) – directory to list the projects for. Defaults to the configured database directory. Returns:

keys are project names; values are lists of tasks associated with the

project.

Return type:dict

acorn.logging.database.oids = {}

dict – keys are python id() values, values are the Instance class instances from which JSON database can be constructed.

acorn.logging.database.project = 'default'

str – currently active project name under which all logging is being saved.

acorn.logging.database.record(ekey, entry, diff=False)

Records the specified entry to the key-value store under the specified entity key.

Parameters:

• ekey (str) – fqdn/uuid of the method/object to store the entry for.
• entry (dict) – attributes and values gleaned from the execution.
• diff (bool) – when True, the “c” element of entry will be diffed against previous entries under the same ekey if their method (attribute “m”) matches.

acorn.logging.database.save_image(byteio, imgfmt)

Saves the specified image to disk.

Parameters:

• byteio (bytes) – image bytes to save to disk.
• imgfmt (str) – used as the extension of the saved file.

Returns:

a uuid for the saved image that can be added to the database entry.

Return type:

str

acorn.logging.database.set_dbdir(dbdir_)

Sets the path to the directory where the JSON files should be stored. Calling this method side-steps the configuration settings in the acorn.cfg global config file.

Parameters:dbdir (str) – path to the dbdir; can be relative.

acorn.logging.database.set_task(project_, task_)

Sets the active project and task. All subsequent logging will be saved to the database with that project and task.

Parameters:

• project (str) – active project name; a project can have multiple tasks.
• task (str) – active task name. Logging is separated at the project and task level.

acorn.logging.database.set_writeable(writeable_)

Sets whether the database is being written to disk, or just organized in memory.

Parameters:writeable (bool) – when True, calling acorn.database.record() will eventually result in an entry being saved to disk; otherwise, records are organized in memory, but the disk write is disabled.

acorn.logging.database.task = 'default'

str – currently active task name under which all logging is being saved.

acorn.logging.database.tracker(obj)

Returns the Instance of the specified object if it is one that we track by default.

Parameters:obj (object) – any python object passed as an argument to a method. Returns:

if the object is trackable, the Instance instance of

that object; else None.

Return type:Instance

acorn.logging.database.uuids = {}

dict – keys are the UUID string values; values are Instance class instances (i.e., same values as oids, but keyed by uuid.

acorn.logging.database.writeable = True

bool – when True, calling acorn.logging.database.record() will eventually result in an entry being saved to disk; otherwise, records are organized in memory, but the disk write is disabled.