Entry point for computing one datum: call this after begin
.
Entry point for computing one datum: call this after begin
.
datum to compute; objects must be specialized to this class (see above).
if method
is map, returns the transformed input; if method
is emit, returns null
(provide a user-defined emit
callback to capture results!); if method
is fold, returns the current cumulative tally
.
Perform an analysis of a cell using a user-defined function.
Perform an analysis of a cell using a user-defined function.
the name of the cell
a function to perform some analysis of the cell to which it is applied; note that this function must not change the cell's state
whatever analysis
returns
Note that the analysis
function is called exactly once.
Perform an analysis of a pool using a user-defined function.
Perform an analysis of a pool using a user-defined function.
the name of the pool
a function to perform some analysis of each item in the pool; note that this function must not change the pool's state
a map from pool item name to whatever analysis
returns for that pool item
Note that the analysis
function is called as many times as there are items in the pool.
Create an Avro iterator (subclass of java.util.Iterator
) over Avro-serialized input data.
Create an Avro iterator (subclass of java.util.Iterator
) over Avro-serialized input data.
The objects produced by this iterator are suitable inputs to the action
method.
serialized data
unserialized data
Create an output stream for Avro-serializing scoring engine output.
Create an output stream for Avro-serializing scoring engine output.
Return values from the action
method (or outputs captured by an emit
callback) are suitable for writing to this stream.
a file that will be overwritten by output.
an output stream with an append
method for appending output data objects.
Create an output stream for Avro-serializing scoring engine output.
Create an output stream for Avro-serializing scoring engine output.
Return values from the action
method (or outputs captured by an emit
callback) are suitable for writing to this stream.
the raw output stream onto which Avro bytes will be written.
an output stream with an append
method for appending output data objects.
Entry point for starting up a scoring engine: call this first.
Determine call depth of a function by traversing the callGraph
.
Determine call depth of a function by traversing the callGraph
.
name of function to look up
set of functions to exclude
used by recursion to count
integral number representing call depth as a Double
, with positive infinity as a possible result
Graph of which functions can call which other functions in the engine.
Graph of which functions can call which other functions in the engine.
Map from function name (special forms in parentheses) to the set of all functions it calls. This map can be traversed as a graph by repeated application.
Determine which functions are called by fcnName
by traversing the callGraph
backward.
Determine which functions are called by fcnName
by traversing the callGraph
backward.
name of function to look up
set of functions to exclude
set of functions that can call fcnName
ClassLoader in which this scoring engine and its compiled types are compiled.
Abstract syntax tree that was used to generate this engine.
Create an iterator over CSV-serialized input data.
Create an iterator over CSV-serialized input data.
The objects produced by this iterator are suitable inputs to the action
method.
Note that only records of primitives can be read from CSV because of the nature of the CSV format.
serialized data
format description for Apache commons-csv
unserialized data
Create an output stream for CSV-serializing scoring engine output.
Create an output stream for CSV-serializing scoring engine output.
Return values from the action
method (or outputs captured by an emit
callback) are suitable for writing to this stream.
Note that only records of primitives can be written to CSV because of the nature of the CSV format.
the raw output stream onto which CSV bytes will be written.
format description for Apache commons-csv
if true
, write field names as the first line of the file.
an output stream with an append
method for appending output data objects.
Specialized Avro data reader for this PFAEngine
.
Specialized Avro data reader for this PFAEngine
.
Reads Avro streams into objects this engine can use (see the *Input
, *InputIterator
methods to prepare input objects).
Entry point for ending a scoring engine: call this after all action
calls are complete.
Entry point for ending a scoring engine: call this after all action
calls are complete.
If the input data stream is infinite, such a time may never happen.
Get a 0-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 0-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 1-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 1-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 10-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 10-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 11-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 11-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 12-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 12-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 13-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 13-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 14-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 14-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 15-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 15-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 16-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 16-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 17-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 17-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 18-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 18-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 19-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 19-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 2-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 2-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 20-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 20-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 21-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 21-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 22-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 22-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 3-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 3-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 4-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 4-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 5-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 5-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 6-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 6-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 7-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 7-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 8-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 8-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Get a 9-parameter user-defined function from the scoring engine as something that can be executed by an external agent.
Get a 9-parameter user-defined function from the scoring engine as something that can be executed by an external agent. If the PFA did not declare a user-defined function with this name or it has a different number of parameters, this will throw a java.util.NoSuchElementException
.
Translate data that might have been deserialized by Avro into objects suitable for this PFAEngine's action
.
Translate data that might have been deserialized by Avro into objects suitable for this PFAEngine's action
.
objects that may be Avro generic or Avro specific objects (note that Avro specific objects are subclasses of Avro generic objects)
objects that can be input for this PFAEngine's action
.
Translate data that might have come from any PFAEngine class (not necessarily this one) into objects suitable for this PFAEngine's action
.
Translate data that might have come from any PFAEngine class (not necessarily this one) into objects suitable for this PFAEngine's action
.
objects that may have been output from another type of PFAEngine's action
, snapshotCell
, or snapshotPool
.
objects that can be input for this PFAEngine's action
.
Determine if the call depth of a function is infinite.
Determine if the call depth of a function is infinite.
name of function to check
true
if the function can eventually call itself through a function that it calls, false
otherwise
Determine if a function modifies the scoring engine's persistent state.
Determine if a function modifies the scoring engine's persistent state.
name of function to check
true
if the function can eventually call (cell-to)
or (pool-to)
on any cell or pool.
Class object for the input type.
Get the AvroType of the input.
Instance number, non-zero if this engine is part of a collection of scoring engines made from the same PFA file.
Determine if a function is directly recursive.
Determine if a function is directly recursive.
name of function to check
true
if the function directly calls itself, false otherwise
Deserialize one JSON datum as suitable input to the action
method.
Deserialize one JSON datum as suitable input to the action
method.
Create an iterator over JSON-serialized input data.
Create an iterator over JSON-serialized input data.
The objects produced by this iterator are suitable inputs to the action
method.
serialized data
unserialized data
Create an iterator over JSON-serialized input data.
Create an iterator over JSON-serialized input data.
The objects produced by this iterator are suitable inputs to the action
method.
serialized data
unserialized data
Create an iterator over JSON-serialized input data.
Create an iterator over JSON-serialized input data.
The objects produced by this iterator are suitable inputs to the action
method.
serialized data
unserialized data
Serialize one datum from the action
method as JSON.
Create an output stream for JSON-serializing scoring engine output.
Create an output stream for JSON-serializing scoring engine output.
Writes one JSON object per line.
Return values from the action
method (or outputs captured by an emit
callback) are suitable for writing to this stream.
the name of a file that will be overwritten by output.
if true
, write an Avro schema as the first line of the file for interpreting the JSON objects.
an output stream with an append
method for appending output data objects.
Create an output stream for JSON-serializing scoring engine output.
Create an output stream for JSON-serializing scoring engine output.
Writes one JSON object per line.
Return values from the action
method (or outputs captured by an emit
callback) are suitable for writing to this stream.
a file that will be overwritten by output.
if true
, write an Avro schema as the first line of the file for interpreting the JSON objects.
an output stream with an append
method for appending output data objects.
Create an output stream for JSON-serializing scoring engine output.
Create an output stream for JSON-serializing scoring engine output.
Writes one JSON object per line.
Return values from the action
method (or outputs captured by an emit
callback) are suitable for writing to this stream.
the raw output stream onto which JSON bytes will be written.
if true
, write an Avro schema as the first line of the file for interpreting the JSON objects.
an output stream with an append
method for appending output data objects.
Externally supplied function for handling log output from PFA.
Externally supplied function for handling log output from PFA.
By default, prints to standard out.
Arguments:
Some(namespace)
for filtering log messages or None
Report whether this is a PFAMapEngine, PFAEmitEngine, or a PFAFoldEngine.
Report whether this is a PFAMapEngine, PFAEmitEngine, or a PFAFoldEngine.
Note that this information is also available in config.method
.
Get the AvroType of each compiled type.
Implementation-specific configuration options.
Class object for the output type.
Get the AvroType of the output.
The random number generator used by this particular scoring engine instance.
The random number generator used by this particular scoring engine instance.
Note that if a randseed
is given in the PFA file but a collection of scoring engines are generated from it, each scoring engine instance will have a different random generator seeded by a different seed.
Restore this scoring engine's original state as defined by the PFA file and sharedState
object it was derived from.
Restore this scoring engine's original state as defined by the PFA file and sharedState
object it was derived from.
same as the sharedState
passed to fromJson
, fromAst
, factoryFromJson
, etc.
Restore this scoring engine's original state as defined by the PFA file it was derived from.
Take a snapshot of the entire scoring engine (all cells and pools) and represent it as an abstract syntax tree that can be used to make new scoring engines.
Take a snapshot of the entire scoring engine (all cells and pools) and represent it as an abstract syntax tree that can be used to make new scoring engines.
Note that you can call toJson
on the EngineConfig
to get a string that can be written to a PFA file.
Take a snapshot of one cell and represent it using objects specialized to this class (see above).
Take a snapshot of one cell and represent it using objects specialized to this class (see above).
the name of the cell
an object that may contain internal PFA data, such as instances of classes that are only found in this engine's custom classLoader
.
Take a snapshot of one pool and represent it using objects specialized to this class (see above).
Take a snapshot of one pool and represent it using objects specialized to this class (see above).
the name of the pool
a Map from pool item name to objects that may contain internal PFA data, such as instances of classes that are only found in this engine's custom classLoader
.
Specialized Avro data model for this PFAEngine
.
Specialized Avro data model for this PFAEngine
.
Deserializes Avro data into objects this engine can use (see the *Input
, *InputIterator
methods to prepare input objects).
The parser used to interpret Avro types in the PFA document, which may be used to find compiled types used by this engine.
Interface for a Hadrian scoring engine.
Create instances using one of
PFAEngine
's "static" methods, then callbegin
once,action
once for each datum in the data stream, andend
once (if the stream ever ends). The rest of the functions are forconfig
, call graph),PFAEngines
, or the Avro library,Examples:
Load a PFA file as a scoring engine. Note the
.head
to extract the single scoring engine from theSeq
this function returns.Assuming (and verifying) that
method
is map, run it over an Avro data stream.Handle the case of
method
= emit engines (map and fold are the same).Take a snapshot of a changing model and write it as a new PFA file.
Take a snapshot of just one cell and write it as a JSON fragment.
Calling a PFA user-defined function from an external agent.
Specialized data:
Data passed to
action
,fcn*
or accepted fromaction
,fcn*
,analyzeCell
,analyzePool
,snapshotCell
,snapshotPool
has to satisfy a particular form. That form is:AnyRef
java.lang.Boolean
java.lang.Integer
java.lang.Long
java.lang.Float
java.lang.Double
String
Array[Byte]
PFAArray[X]
whereX
is an unboxed primitive if a primitivePFAMap[X]
whereX
is a boxed primitive if a primitivePFAEnumSymbol
that can only be found in this specificPFAEngine
's custom ClassLoader (cannot be created by external agent)PFAFixed
that can only be found in the custom ClassLoaderPFARecord
that can only be found in the custom ClassLoaderAnyRef
Compiled types, namely enum, fixed, and record, have to be converted using
fromPFAData
orfromGenericAvroData
.Although all of these types are immutable in PFA, bytes, fixed, and record are mutable in Java, but if you modify them, the behavior of the PFA engine is undefined and likely to be wrong. Do not change these objects in place!