titus.producer.transformation.Transformation

class titus.producer.transformation.Transformation(*indexed, **named)[source]

Bases: object

Represents an expression that can be applied to Numpy arrays in Python and included in a PFA scoring engine.

__init__(*indexed, **named)

Create a Transformation either from an ordered list of Python expressions or by keywords.

Although it’s possible to mix the ordered list method and the keyword method, doing so could be confusing.

Expressions loaded as positional arguments are given keyword names “_0”, “_1”, “_2”, etc., and all methods proceed as though they were loaded as keywords.

Parameters:
  • indexed (strings (as positional arguments)) – executable Python code in strings
  • named (strings (as keyword arguments)) – executable Python code in strings
constants = {'m.pi': 'math.pi', 'm.e': 'math.e'}
expr(name='_0', subs={}, **subs2)

Construct a PFA expression for one of the expressions in this Transformation.

Parameters:
  • name (string) – expression to choose
  • subs (dict of substitutions) – subsititusions to apply
  • subs2 (dict of substitutions) – more subsititusions to apply
Return type:

Pythonized JSON

Returns:

PFA “let” expression

static findFields(x)

Find the symbols (variables) referenced in a PFA expression.

Parameters:x (Pythonized JSON) – the PFA expression
Return type:list of strings
Returns:all symbols found
functions = {'!': 'numpy.logical_not', 'm.cos': 'numpy.cos', '>=': 'numpy.greater_equal', '^^': 'numpy.logical_xor', '%%': 'numpy.remainder', '||': 'numpy.logical_or', '<=': 'numpy.less_equal', 'm.acos': 'numpy.arccos', 'm.hypot': 'numpy.hypot', 'm.tan': 'numpy.tan', 'm.exp': 'numpy.exp', '**': 'numpy.power', 'm.floor': 'numpy.floor', 'm.log10': 'numpy.log10', 'm.rint': 'numpy.rint', 'm.round': 'numpy.round', '!=': 'numpy.not_equal', 'm.atan': 'numpy.arctan', '&&': 'numpy.logical_and', 'm.tanh': 'numpy.tanh', 'm.expm1': 'numpy.expm1', '%': 'numpy.mod', 'm.cosh': 'numpy.cosh', '+': 'numpy.add', '*': 'numpy.multiply', '-': 'numpy.subtract', 'm.asin': 'numpy.arcsin', '/': 'numpy.true_divide', 'm.ln': 'numpy.log', 'u-': 'numpy.negative', '==': 'numpy.equal', 'm.abs': 'numpy.absolute', 'm.ceil': 'numpy.ceil', 'm.sinh': 'numpy.sinh', 'm.sqrt': 'numpy.sqrt', 'm.sin': 'numpy.sin', 'm.copysign': 'numpy.copysign', 'm.atan2': 'numpy.arctan2', '<': 'numpy.less', '>': 'numpy.greater'}
static interpret(x)

Interpret expression from a PFA abstract syntax tree or PrettyPFA string.

Parameters:x (titus.datatype.Ast, PrettyPFA string) – input PFA
Return type:Pythonized JSON
Returns:the PFA as Pythonized JSON
let(subs={}, **subs2)

Construct a PFA “let” expression for this transformation.

Parameters:
  • subs (dict of substitutions) – subsititusions to apply
  • subs2 (dict of substitutions) – more subsititusions to apply
Return type:

Pythonized JSON

Returns:

PFA “let” expression

namespace = {'numpy': <module 'numpy' from '/usr/lib/python2.7/dist-packages/numpy/__init__.pyc'>, 'math': <module 'math' (built-in)>}
new(avroType, subs={}, **subs2)

Construct a PFA “new” expression for this transformation.

Parameters:
  • avroType (titus.datatype.AvroType) – type argument for the PFA “new” expression
  • subs (dict of substitutions) – subsititusions to apply
  • subs2 (dict of substitutions) – more subsititusions to apply
Return type:

Pythonized JSON

Returns:

PFA “new” expression

static replace(x, subs)

Apply replacements to a PFA expression.

Parameters:
  • x (Pythonized JSON) – the PFA expression
  • subs (dict from substitution identifier to its replacement value) – identifiers to replace
Return type:

Pythonized JSON

Returns:

the PFA with replacements

static toNumpyExpr(ast)

Convert a PFA abstract syntax tree into a Numpy expression.

Parameters:ast (titus.pfaast.Ast) – the PFA to convert
Return type:string
Returns:executable Python code in a string
transform(dataset, fieldNames=None)

Return a transformed Numpy dataset (leaving the original intact).

Parameters:dataset (Numpy record array, dict of 1-D Numpy arrays, or Numpy 2-d table) – input dataset to be transformed; the Numpy record names or dict keys must correspond to the keywords of the arguments used to construct this Transformation, or the column indexes of the 2-d table must correspond to the positions of the arguments used to construct this Transformation.
Return type:same as dataset
Returns:transformed dataset (operations are not performed in-place)