Automatic Decoration of Packages

Every package is different. Some define new built-in types, others use C-extension modules to optimize the code being run. In a simple world, all the packages would just be decoratable using standard approaches. However, this doesn’t always work, which is why large and complicated decoration logic was required to get acorn to work.

The basic flow of the decoration is as follows for a new object:

1. Try to determine the fully-qualified domain name (FQDN) of the object, e.g., numpy.core.multiarray.array. If the FQDN can be found, proceed to the next step; otherwise, we can’t decorate the object. This happens very seldomly (perhaps 1/10,000 objects).
2. Try to set an attribute on the object. If that works, proceed to step 4; otherwise go to step 3.
3. Create an extension for the object. For classes, this means automatically sub-classing the object (to inherit all the same attributes as the parent). If the object can’t be sub-classed, we treat it specially (see below) or set it as not-decoratable. For functions, we define a new function locally that calls the old one; we then copy across all the attributes of the function. Extended objects keep a reference to the original, unextended object in the __acornext__ special attribute.
4. Proceed to decorate the object. For functions and methods, we wrap the object using acorn.logging.decoration.CallingDecorator and then overwrite the attribute in the parent object to point to the new function. The original function is copied to the __acorn__ special attribute. If the methods were originally static or class methods, we use staticmethod and classmethod to change their behavior. Attributes from the original functions (such as __doc__) are also copied across so that the function seems identical to the end user. For classes, we overwrite the __new__ method with acorn.logging.decoration.creationlog() and copy the wrapped __new__ to __old__ on the object. Sometimes packages have multiple references to the same object, but located in different parts of the package as attributes with different names. When this is discovered, the already-decorated object is used in a simple re-assignment of attributes.
5. If the object is of type classobj or module, we recursively examine each of the methods and attributes of the class to determine if they need to be decorated or not. Instance methods remain instance methods; class and static methods also retain their identity, but are replaced with decorated analogs.

We use aggressive caching of the objects to try and speed things up. We don’t decorate any object that already has an __acorn__ attribute. Once a package has been decorated, we don’t decorate it ever again.

API Documentation

Most of the logic for the decoration is buried in “private” methods of the module. However, certain public methods are available. However, we recommend to not call these unless you know what you are doing. It is better to use the built-in acorn importer.

Methods for dynamically adding decorators to all the methods and classes within a module. The entries created by the decorators have been pre-compressed for clarity and file size. Thus the short key names mean the following:

m = method
a = args
r = returns
s = start
e = elapsed
x = analysis
c = code

Examples:

Decorate a new package using the acorn decorator machinery.

>>> from acorn.logging.decoration import decorate
>>> import package
>>> decorate(package)

The decorate routine replaces all the members of the package with decorated ones and does this recursively until the whole package is decorated. We can also use the automatic decoration machinery with acorn. Open the acorn.cfg file and add the name of the package to [acorn.packages]:

package=1

Then, you can auto-decorate the package using:

import acorn.package

class acorn.logging.decoration.CallingDecorator(func)

Decorator for wrapping package library methods for intelligent logging.

Parameters:func (function) – the function to wrap with a logging decorator.

Examples

Replace a function myfunc that was declared in a module mymod with a decorated version. The fully-qualified name of the object can be queried using acorn.logging.decoration._fqdn().

>>> from acorn.logging.decoration import CallingDecorator as CD
>>> decor = CD(myfunc)
>>> setattr(mymod, "myfunc", decor(fqdn, package, None))

acorn.logging.decoration.creationlog(base, package, stackdepth=4)

Decorator for wrapping the creation of class instances that are being logged by acorn.

Parameters:

• base – base class used to call __new__ for the construction.
• package (str) – name of (global) package the class belongs to.
• stackdepth (int) – if the calling stack is less than this depth, than include the entry in the log; otherwise ignore it.

acorn.logging.decoration.decorate(package)

Decorates all the methods in the specified package to have logging enabled according to the configuration for the package.

acorn.logging.decoration.decorate_obj(parent, n, o, otype, recurse=True, redecorate=False)

Adds the decoration for automated logging to the specified object, if it hasn’t already been done.

Parameters:

• parent – object that o belongs to.
• n (str) – name in the parent’s dictionary.
• o (type) – instance of the object’s type.
• otype (str) – one of [‘classes’, ‘functions’, ‘methods’, ‘modules’]; specifies which group the object belongs to.
• recurse (bool) – when True, the objects methods and functions are also decorated recursively.

Examples

Decorate the function mymod.myfunc to log automatically to the database.

>>> from acorn.logging.decoration import decorate_obj
>>> import mymod
>>> decorate_obj(mymod, "myfunc", mymod.myfunc, "functions")

acorn.logging.decoration.decorating = False

bool – when True, the script is decorating objects; if any other objects have methods that call decorated objects, we don’t want the decorations to log entries since we are still in the initialization phase.

acorn.logging.decoration.filter_name(funcname, package, context='decorate', explicit=False)

Returns True if the specified function should be filtered (i.e., included or excluded for use in the specified context.)

Parameters:

• funcname (str) – name of the method/function being called.
• package (str) – name of the package that the method belongs to.
• context (str) – one of [‘decorate’, ‘time’, ‘analyze’]; specifies which section of the configuration settings to check.
• explicit (bool) – when True, if a name is not explicitly specified for inclusion, then the function returns False.

Returns:

specifying whether the function should be decorated, timed or

analyzed.

Return type:

bool

acorn.logging.decoration.iswhat(o)

Returns a dictionary of all possible identity checks available to inspect applied to o.

Returns:

keys are inspect.is* function names; values are bool results

returned by each of the methods.

Return type:dict

acorn.logging.decoration.name_filters = {}

dict – keys are package names; values are dicts of lists. 1) fnmatch() patterns; 2) re.match() patterns.

acorn.logging.decoration.post(fqdn, package, result, entry, bound, ekey, *argl, **argd)

Adds logging for the post-call result of calling the method externally.

Parameters:

• fqdn (str) – fully-qualified domain name of the function being logged.
• package (str) – name of the package we are logging for. Usually the first element of fqdn.split(‘.’).
• result – returned from calling the method we are logging.
• entry (dict) – one of the values returned by pre().
• bound (bool) – true if the method is bound.
• ekey (str) – key under which to store the entry in the database.

acorn.logging.decoration.postfix(package)

Makes sure that any additional imported names in the specified package that were decorated in the context of a different package still get redirected to the decorated objects.

Parameters:package – package object to examine for compliance.

Examples: When scipy is imported, it imports most of numpy automatically and then has references to the original, undecorated numpy functions. We use postfix() to set the scipy references to the decorated numpy functions.

>>> import scipy
>>> import acorn.numpy
>>> from acorn.logging.decoration import postfix, decorate
>>> decorate(scipy)
>>> postfix(scipy)

When we call decorate() on scipy, the references to numpy functions are ignored, because they don’t belong to the scipy package. This filtering is a feature to prevent acorn repeatedly visiting commonly used modules that are imported.

acorn.logging.decoration.pre(fqdn, parent, stackdepth, *argl, **argd)

Adds logging for a call to the specified function that is being handled by an external module.

Parameters:

• fqdn (str) – fully-qualified domain name of the function being logged.
• parent – object that the function belongs to.
• stackdepth (int) – maximum stack depth before entries are ignored.
• argl (list) – positional arguments passed to the function call.
• argd (dict) – keyword arguments passed to the function call.

acorn.logging.decoration.set_decorating(decorating_)

Sets whether the module is operating in decorating mode.

acorn.logging.decoration.set_streamlining(streamline)

Sets whether acorn logging logic should be disabled temporarily.

acorn.logging.decoration.streamlining = False

bool – when True, a method has disabled all logging for subsequent calls. The method will reset this variable once it has executed. This is needed to optimize packages like matplotlib that make thousands of calls for a high-level method like plot.