com.opendatagroup.hadrian

data

package data

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. data
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. trait AnyPFAEnumSymbol extends AnyRef

    Interface for GenericPFAEnumSymbol all dynamically generated subclasses of PFAEnumSymbol.

  2. trait AnyPFAFixed extends AnyRef

    Interface for GenericPFAFixed all dynamically generated subclasses of PFAFixed.

  3. trait AnyPFARecord extends AnyRef

    Interface for GenericPFARecord all dynamically generated subclasses of PFARecord.

  4. class AvroDataTranslator extends AnyRef

    Translates data created by the Avro library (generic or specific) into a form usable by a PFAEngine.

  5. class AvroOutputDataStream extends DataFileWriter[AnyRef] with OutputDataStream

    Output stream for Avro files (including header).

  6. class Comparison extends (AnyRef, AnyRef) ⇒ Int

    Comparison function for two Avro objects, specialized to a given schema.

  7. class ComparisonMax extends AnyRef

    Max function for two Avro objects, specialized to a given schema.

  8. class ComparisonMin extends AnyRef

    Min function for two Avro objects, specialized to a given schema.

  9. class ComparisonOperator extends (AnyRef, AnyRef) ⇒ Boolean

    Comparison operator for two Avro objects, specialized to a given schema.

  10. class ComparisonOperatorLT extends (AnyRef, AnyRef) ⇒ Boolean

    Less than function for two Avro objects, specialized to a given schema.

  11. class CsvOutputDataStream extends OutputDataStream

    Output stream for CSV files.

  12. class GenericComparisonMax extends AnyRef

    Max function for two Avro objects, specialized to a given cmp.

  13. class GenericComparisonMin extends AnyRef

    Min function for two Avro objects, specialized to a given cmp.

  14. class GenericPFADatumReader[X] extends GenericDatumReader[X]

    Overrides Avro's data model with one designed for PFA.

  15. class GenericPFADatumWriter[X] extends GenericDatumWriter[X]

    Overrides Avro's data model with one designed for PFA.

  16. class GenericPFAEnumSymbol extends EnumSymbol with AnyPFAEnumSymbol

    PFA enum symbol with no dynamically generated code (used as an intermediate for data not assigned to any scoring engine).

  17. class GenericPFAFixed extends Fixed with AnyPFAFixed

    PFA fixed with no dynamically generated code (used as an intermediate for data not assigned to any scoring engine).

  18. class GenericPFARecord extends Record with AnyPFARecord

    PFA record with no dynamically generated code (used as an intermediate for data not assigned to any scoring engine).

  19. class JsonOutputDataStream extends OutputDataStream

    Output stream for JSON files (each line of text is one JSON object).

  20. trait OutputDataStream extends AnyRef

    Output for a stream of data objects.

  21. class PFAArray[X] extends List[X]

    Represents all arrays in PFA (generic or specific).

  22. class PFADataTranslator extends AnyRef

    Translates data created by one PFAEngine into a form usable by another PFAEngine.

  23. class PFADatumReader[X] extends SpecificDatumReader[X]

    Overrides Avro's data model with one designed for PFA.

  24. class PFADatumWriter[X] extends SpecificDatumWriter[X]

    Overrides Avro's data model with one designed for PFA.

  25. abstract class PFAEnumSymbol extends AnyPFAEnumSymbol

    Abstract superclass for all dynamically generated enum classes.

  26. abstract class PFAFixed extends SpecificFixed with AnyPFAFixed

    Abstract superclass for all dynamically generated fixed classes.

  27. class PFAGenericData extends GenericData

    Overrides Avro's data model with one designed for PFA.

  28. class PFAMap[X <: AnyRef] extends Map[String, X]

    Represents all maps in PFA (generic or specific).

  29. abstract class PFARecord extends SpecificRecordBase with AnyPFARecord

    Abstract superclass for all dynamically generated record classes.

  30. class PFASpecificData extends SpecificData

    Overrides Avro's data model with one designed for PFA with classes customized for each PFA engine.

  31. class ScalaDataTranslator[X] extends AnyRef

    Translates Scala data into a form usable by a PFAEngine.

Value Members

  1. object AvroDataTranslator

  2. object NumericalComparison

    Numerical comparison function for two Avro numbers; applies the correct Avro rules for NaN.

  3. object NumericalEQ

    Numerical equality function for two Avro numbers; applies the correct Avro rules for NaN.

  4. object NumericalGE

    Numerical greater than or equal function for two Avro numbers; applies the correct Avro rules for NaN.

  5. object NumericalGT

    Numerical greater than function for two Avro numbers; applies the correct Avro rules for NaN.

  6. object NumericalLE

    Numerical less than or equal function for two Avro numbers; applies the correct Avro rules for NaN.

  7. object NumericalLT

    Numerical less than function for two Avro numbers; applies the correct Avro rules for NaN.

  8. object NumericalMax

    Numerical maximum function for two Avro numbers; applies the correct Avro rules for NaN.

  9. object NumericalMin

    Numerical minimum function for two Avro numbers; applies the correct Avro rules for NaN.

  10. object NumericalNE

    Numerical not equal function for two Avro numbers; applies the correct Avro rules for NaN.

  11. object PFAArray

  12. object PFADataTranslator

  13. object PFAMap

  14. object ScalaDataTranslator

  15. def avroInputIterator[X](inputStream: InputStream, inputType: AvroType): DataFileStream[X]

    Create an Avro iterator (subclass of java.util.Iterator) over Avro-serialized input data.

    Create an Avro iterator (subclass of java.util.Iterator) over Avro-serialized input data.

    The objects produced by this iterator need to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    inputStream

    serialized data

    inputType

    input type

    returns

    unserialized data

  16. def avroOutputDataStream(fileName: String, outputType: AvroType): AvroOutputDataStream

    Create an output stream to an Avro file (including header).

    Create an output stream to an Avro file (including header).

    fileName

    name of file to overwrite

    outputType

    datum type

  17. def avroOutputDataStream(file: File, outputType: AvroType): AvroOutputDataStream

    Create an output stream to an Avro file (including header).

    Create an output stream to an Avro file (including header).

    file

    file to overwrite

    outputType

    datum type

  18. def avroOutputDataStream(outputStream: OutputStream, outputType: AvroType): AvroOutputDataStream

    Create an output stream to an Avro file (including header).

    Create an output stream to an Avro file (including header).

    outputStream

    stream to write into

    outputType

    datum type

  19. def csvInputIterator[X](inputStream: InputStream, inputType: AvroType, csvFormat: CSVFormat = CSVFormat.DEFAULT.withHeader(), makeFieldReaders: Option[(AvroType, Map[String, Int]) ⇒ Seq[(Int, (String) ⇒ AnyRef)]] = None, makeRecord: Option[(Array[AnyRef]) ⇒ X] = None): Iterator[X]

    Create an iterator over CSV-serialized input data.

    Create an iterator over CSV-serialized input data.

    The objects produced by this iterator are need to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    Note that only records of primitives can be read from CSV because of the nature of the CSV format.

    inputStream

    serialized data

    inputType

    input type

    csvFormat

    format description for Apache commons-csv

    returns

    unserialized data

  20. def csvOutputDataStream(outputStream: OutputStream, outputType: AvroType, csvFormat: CSVFormat = ..., writeHeader: Boolean = true): CsvOutputDataStream

    Create an output stream for CSV-serializing scoring engine output.

    Create an output stream for CSV-serializing scoring engine output.

    Return values from the action method (or outputs captured by an emit callback) are suitable for writing to this stream.

    Note that only records of primitives can be written to CSV because of the nature of the CSV format.

    outputStream

    the raw output stream onto which CSV bytes will be written.

    csvFormat

    format description for Apache commons-csv

  21. def fromAvro(avro: Array[Byte], avroType: AvroType): AnyRef

    Convert data from Avro to a generic PFA object.

    Convert data from Avro to a generic PFA object.

    Needs to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    avro

    Avro data

    avroType

    data type

    returns

    PFA data

  22. def fromAvro(avro: Array[Byte], schema: Schema): AnyRef

    Convert data from Avro to a generic PFA object.

    Convert data from Avro to a generic PFA object.

    Needs to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    avro

    Avro data

    schema

    Avro schema

    returns

    PFA data

  23. def fromJson(json: String, avroType: AvroType): AnyRef

    Convert data from JSON to a generic PFA object.

    Convert data from JSON to a generic PFA object.

    Needs to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    json

    JSON data

    avroType

    data type

    returns

    PFA data

  24. def fromJson(json: String, schema: Schema): AnyRef

    Convert data from JSON to a generic PFA object.

    Convert data from JSON to a generic PFA object.

    Needs to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    json

    JSON data

    schema

    Avro schema

    returns

    PFA data

  25. val genericData: PFAGenericData

  26. def jsonInputIterator[X](inputIterator: Iterator[String], inputType: AvroType): Iterator[X]

    Create an iterator over JSON-serialized input data.

    Create an iterator over JSON-serialized input data.

    The objects produced by this iterator need to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    inputIterator

    iterator of Strings, each containing a JSON object

    inputType

    input type

    returns

    unserialized data

  27. def jsonInputIterator[X](inputIterator: Iterator[String], inputType: AvroType): Iterator[X]

    Create an iterator over JSON-serialized input data.

    Create an iterator over JSON-serialized input data.

    The objects produced by this iterator need to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    inputIterator

    iterator of Strings, each containing a JSON object

    inputType

    input type

    returns

    unserialized data

  28. def jsonInputIterator[X](inputStream: InputStream, inputType: AvroType): Iterator[X]

    Create an iterator over JSON-serialized input data.

    Create an iterator over JSON-serialized input data.

    The objects produced by this iterator need to be translated to a specific engine with a PFADataTranslator or the engine's fromPFAData method.

    inputStream

    serialized data

    inputType

    input type

    returns

    unserialized data

  29. def jsonOutputDataStream(fileName: String, outputType: AvroType, writeSchema: Boolean): JsonOutputDataStream

    Create an output stream to a JSON file (each line of text is one JSON object).

    Create an output stream to a JSON file (each line of text is one JSON object).

    fileName

    name of file to overwrite

    outputType

    datum type

    writeSchema

    if true, write the Avro schema as the first line of the file; if false, write only data

  30. def jsonOutputDataStream(file: File, outputType: AvroType, writeSchema: Boolean): JsonOutputDataStream

    Create an output stream to a JSON file (each line of text is one JSON object).

    Create an output stream to a JSON file (each line of text is one JSON object).

    file

    file to overwrite

    outputType

    datum type

    writeSchema

    if true, write the Avro schema as the first line of the file; if false, write only data

  31. def jsonOutputDataStream(outputStream: OutputStream, outputType: AvroType, writeSchema: Boolean): JsonOutputDataStream

    Create an output stream to a JSON file (each line of text is one JSON object).

    Create an output stream to a JSON file (each line of text is one JSON object).

    outputStream

    stream to write into

    outputType

    datum type

    writeSchema

    if true, write the Avro schema as the first line of the file; if false, write only data

  32. def toAvro(obj: AnyRef, avroType: AvroType): Array[Byte]

    Convert data to Avro.

    Convert data to Avro.

    obj

    object reference

    returns

    Avro bytes

  33. def toAvro(obj: AnyRef, schema: Schema): Array[Byte]

    Convert data to Avro.

    Convert data to Avro.

    obj

    object reference

    schema

    Avro schema

    returns

    Avro bytes

  34. def toJson(obj: AnyRef, avroType: AvroType): String

    Convert data to JSON.

    Convert data to JSON.

    obj

    object reference

    returns

    JSON string

  35. def toJson(obj: AnyRef, schema: Schema): String

    Convert data to JSON.

    Convert data to JSON.

    obj

    object reference

    schema

    Avro schema

    returns

    JSON string

  36. def toJsonDom(obj: AnyRef, avroType: AvroType): JsonDom

    Convert a PFA object into an in-house JsonDom.

    Convert a PFA object into an in-house JsonDom.

    obj

    the object

    avroType

    its type

    returns

    the JSON DOM that can be used in Scala pattern matching

  37. def toJsonNode(obj: AnyRef, avroType: AvroType): JsonNode

    Convert a PFA object into a Jackson JsonNode.

    Convert a PFA object into a Jackson JsonNode.

    obj

    the object

    avroType

    its type

    returns

    the JSON DOM used by Jackson

Inherited from AnyRef

Inherited from Any

Ungrouped