Hadrian performance

The sortable table below presents mean and standard deviation (stdev) times spent computing each PFA function in isolation. The "(nothing)" special case is a PFA engine that simply returns null and the "(pass-through)" special case is a PFA engine that simply outputs what it is given. These represent the overhead of the PFA action itself, while the tests of each function should be understood as being relative to this baseline (within a standard deviation).

Each mean and standard deviation is based on all the conformance tests, which cover a broad spectrum of input types (for functions with wildcards) and input values, and each input value was tested hundreds of times. The effect of HotSpot just-in-time optimizations was avoided by instituting a burn-in period, and the effect of the Java garbage collector was avoided by invoking the garbage collector when the stopwatch is off (when the heap memory used reaches 5 GB in a JVM with a maximum of 8 GB).

Means and standard deviations are split into two cases: ones which result in a value (no errors) and ones that raised an exception.

All tests were performed on an AWS m3.xlarge instance (4 vCPU Intel Xeon E5-2670 v2 processors, 15 GB SSD-based RAM) running Ubuntu 14.04 (Trusty Tahr) and Oracle Java version 1.8.0_40 (64-bit build 25.40-b25) with -Xmx8g -Xmn8g to ensure the fixed memory allocation that allows us to control garbage collector calls.

Returning value
Raising exception
Function
Mean (μs)
Stdev (μs)
Mean (μs)
Stdev (μs)
Distribution
(nothing)1.690.15n/an/a
(pass-through)1.660.06n/an/a
!1.520.04n/an/a
!!!1.520.05n/an/a
!=1.660.22n/an/a
%1.570.096.640.07
%%1.540.046.670.15
&1.500.01n/an/a
&&1.520.03n/an/a
&&&1.540.12n/an/a
*1.520.086.550.06
**1.670.267.030.66
+1.740.1323.976.41
-1.650.1315.414.89
/1.510.03n/an/a
//1.540.036.730.11
<1.670.19n/an/a
<=1.650.19n/an/a
==1.650.20n/an/a
>1.650.22n/an/a
>=1.640.21n/an/a
^1.510.06n/an/a
^^1.520.03n/an/a
a.all1.590.09n/an/a
a.any1.540.05n/an/a
a.append1.550.02n/an/a
a.argmax3.763.438.742.07
a.argmaxLT3.312.878.872.55
a.argmaxN3.464.429.202.47
a.argmaxNLT3.082.408.832.54
a.argmin3.853.659.573.44
a.argminLT3.303.699.082.90
a.argminN3.434.488.852.42
a.argminNLT3.112.438.832.43
a.combinations2.092.728.812.37
a.concat2.020.35n/an/a
a.contains1.731.13n/an/a
a.corresponds1.610.22n/an/a
a.correspondsWithIndex1.590.27n/an/a
a.count1.760.78n/an/a
a.cycle1.590.098.160.91
a.diff2.684.30n/an/a
a.disjoint3.062.91n/an/a
a.distinct3.033.16n/an/a
a.dropWhile1.700.24n/an/a
a.endswith1.600.23n/an/a
a.filter1.720.29n/an/a
a.filterMap1.921.5015.3010.86
a.filterMapWithIndex2.181.8115.748.22
a.filterWithIndex1.990.60n/an/a
a.flatMap2.402.4626.3515.62
a.flatMapWithIndex2.522.5523.6013.78
a.flatten1.910.66n/an/a
a.fold1.740.6511.291.99
a.foldRight1.680.309.330.23
a.geomean2.640.39n/an/a
a.groupby3.975.4336.9514.57
a.head1.530.058.030.44
a.index1.680.55n/an/a
a.init1.590.108.411.49
a.insert2.280.478.270.64
a.intersection3.465.71n/an/a
a.last1.540.088.311.27
a.len1.520.02n/an/a
a.lnsum2.210.75n/an/a
a.logsumexp2.673.83n/an/a
a.map2.131.6716.3511.87
a.mapWithIndex2.321.8815.7710.00
a.max2.592.398.240.69
a.maxLT2.451.568.311.61
a.maxN2.933.788.701.09
a.maxNLT2.451.938.391.32
a.mean2.150.54n/an/a
a.median2.473.888.691.39
a.min2.572.368.602.06
a.minLT2.391.528.220.70
a.minN3.043.989.162.96
a.minNLT2.421.898.450.89
a.mode5.415.768.561.14
a.ntile2.533.998.641.54
a.permutations3.372.03n/an/a
a.product1.850.319.931.25
a.reduce1.841.208.851.99
a.reduceRight1.871.239.112.57
a.remove1.800.248.250.85
a.replace1.620.038.240.36
a.reverse1.690.24n/an/a
a.rindex1.710.66n/an/a
a.rotate1.910.398.541.38
a.seteq3.153.94n/an/a
a.shuffle1.890.41n/an/a
a.slidingWindow2.331.208.280.86
a.sort2.183.36n/an/a
a.sortLT1.870.91n/an/a
a.startswith1.580.22n/an/a
a.subseq1.550.07n/an/a
a.subseqto1.920.34n/an/a
a.subset2.812.44n/an/a
a.sum1.890.3710.012.07
a.symdiff3.535.01n/an/a
a.tail1.580.138.271.69
a.takeWhile1.650.26n/an/a
a.union4.066.18n/an/a
a.zipmap2.401.258.553.13
a.zipmapWithIndex2.561.418.623.20
bytes.decodeAscii1.710.1310.010.08
bytes.decodeLatin11.750.21n/an/a
bytes.decodeUtf161.650.0110.300.13
bytes.decodeUtf16be1.650.0110.550.08
bytes.decodeUtf16le1.660.0110.520.14
bytes.decodeUtf81.680.1310.841.61
bytes.encodeAscii1.950.4711.823.65
bytes.encodeLatin11.740.0310.650.09
bytes.encodeUtf161.860.10n/an/a
bytes.encodeUtf16be1.800.08n/an/a
bytes.encodeUtf16le1.820.08n/an/a
bytes.encodeUtf81.770.07n/an/a
bytes.fromBase642.140.2516.982.24
bytes.isAscii1.911.01n/an/a
bytes.isLatin11.780.71n/an/a
bytes.isUtf162.251.11n/an/a
bytes.isUtf16be2.110.89n/an/a
bytes.isUtf16le2.120.90n/an/a
bytes.isUtf81.760.50n/an/a
bytes.len1.530.01n/an/a
bytes.subseq1.550.06n/an/a
bytes.subseqto1.610.03n/an/a
bytes.toBase641.860.03n/an/a
cast.avro2.140.28n/an/a
cast.double1.520.05n/an/a
cast.fanoutBoolean1.730.3613.496.18
cast.fanoutDouble1.720.3512.154.60
cast.fanoutFloat1.730.359.961.17
cast.fanoutInt1.720.369.961.56
cast.fanoutLong1.730.3410.441.08
cast.float1.510.01n/an/a
cast.int1.600.188.861.11
cast.json7.732.73n/an/a
cast.long1.560.128.830.90
cast.signed1.650.168.931.22
cast.unsigned1.560.0111.750.12
cmp1.660.28n/an/a
enum.numSymbols1.530.01n/an/a
enum.toInt1.520.01n/an/a
enum.toString1.520.01n/an/a
fixed.fromBytes1.770.19n/an/a
fixed.toBytes1.530.01n/an/a
impute.defaultOnNonNum1.540.07n/an/a
impute.defaultOnNull1.530.04n/an/a
impute.errorOnNonNum1.610.108.811.54
impute.errorOnNull1.560.158.101.10
impute.isinf1.550.17n/an/a
impute.isnan1.520.01n/an/a
impute.isnum1.570.12n/an/a
interp.bin1.560.058.050.52
interp.linear6.654.409.201.71
interp.linearFlat6.444.988.971.28
interp.linearMissing6.344.878.981.36
interp.nearest2.253.328.121.36
la.add6.609.6114.316.49
la.det5.018.8111.384.15
la.dot15.779.6511.936.08
la.eigenBasis49.5431.5512.225.59
la.inverse27.0410.0415.824.44
la.map3.322.60n/an/a
la.scale2.110.79n/an/a
la.sub6.327.4812.796.23
la.symmetric3.962.8312.525.05
la.trace2.541.3410.571.33
la.transpose12.529.209.744.18
la.truncate6.442.6114.114.38
la.zipmap18.0931.9523.064.22
m.abs1.540.086.990.11
m.acos1.620.18n/an/a
m.asin1.630.18n/an/a
m.atan1.550.02n/an/a
m.atan21.590.08n/an/a
m.ceil1.570.08n/an/a
m.copysign1.510.03n/an/a
m.cos1.550.03n/an/a
m.cosh1.590.04n/an/a
m.e1.500.01n/an/a
m.exp1.640.27n/an/a
m.expm11.570.03n/an/a
m.floor1.500.01n/an/a
m.hypot1.800.22n/an/a
m.kernel.linear7.501.8213.064.44
m.kernel.poly2.600.947.710.61
m.kernel.rbf2.511.017.431.24
m.kernel.sigmoid2.290.557.710.41
m.link.cauchit1.950.74n/an/a
m.link.cloglog2.221.90n/an/a
m.link.logit2.161.61n/an/a
m.link.loglog2.292.02n/an/a
m.link.probit2.261.64n/an/a
m.link.relu1.890.52n/an/a
m.link.softmax6.7512.308.952.48
m.link.softplus2.141.48n/an/a
m.link.tanh2.020.89n/an/a
m.ln1.540.03n/an/a
m.ln1p1.560.03n/an/a
m.log1.600.067.250.51
m.log101.550.02n/an/a
m.pi1.590.17n/an/a
m.rint1.510.04n/an/a
m.round1.630.197.821.10
m.signum1.530.06n/an/a
m.sin1.550.02n/an/a
m.sinh1.600.04n/an/a
m.special.erf2.090.47n/an/a
m.special.erfc2.040.49n/an/a
m.special.lnBeta1.830.567.120.48
m.special.lnGamma2.370.5410.263.42
m.special.nChooseK1.930.457.902.09
m.sqrt1.510.02n/an/a
m.tan1.570.04n/an/a
m.tanh1.590.05n/an/a
map.add3.953.31n/an/a
map.argmax3.193.279.162.93
map.argmaxLT2.963.249.403.81
map.argmaxN3.315.619.213.42
map.argmaxNLT3.042.749.013.34
map.argmin3.403.739.784.01
map.argminLT3.394.1110.234.34
map.argminN2.995.388.682.89
map.argminNLT2.812.528.873.16
map.containsKey1.660.21n/an/a
map.containsValue1.800.62n/an/a
map.corresponds1.720.44n/an/a
map.correspondsWithKey1.740.50n/an/a
map.diff1.570.23n/an/a
map.disjoint2.201.11n/an/a
map.except1.590.35n/an/a
map.filter1.820.68n/an/a
map.filterMap2.051.8811.571.90
map.filterMapWithKey2.041.8411.152.62
map.filterWithKey1.770.43n/an/a
map.flatMap3.388.6224.426.94
map.flatMapWithKey3.7413.5945.7623.18
map.fromset3.820.74n/an/a
map.in10.170.78n/an/a
map.intersection1.560.19n/an/a
map.join2.251.45n/an/a
map.keys1.860.35n/an/a
map.len1.540.08n/an/a
map.map2.181.8711.941.99
map.mapWithKey2.201.9911.663.34
map.only1.740.09n/an/a
map.remove1.580.29n/an/a
map.split2.241.00n/an/a
map.subset1.750.41n/an/a
map.symdiff4.043.03n/an/a
map.toset8.6310.02n/an/a
map.union1.910.83n/an/a
map.update1.961.15n/an/a
map.values1.980.59n/an/a
map.zipmap2.991.819.711.63
map.zipmapWithKey3.121.979.651.65
max1.670.21n/an/a
metric.absDiff1.500.01n/an/a
metric.binarySimilarity1.920.227.900.63
metric.chebyshev1.710.187.612.06
metric.euclidean1.780.477.550.89
metric.gaussianSimilarity1.840.39n/an/a
metric.jaccard2.110.597.791.07
metric.minkowski2.381.317.540.97
metric.simpleEuclidean3.370.8513.914.78
metric.simpleMatching2.170.658.001.09
metric.squaredEuclidean2.020.747.801.00
metric.tanimoto2.110.657.811.08
metric.taxicab1.950.707.701.67
min1.640.20n/an/a
model.cluster.closest1.720.358.140.81
model.cluster.closestN3.8810.298.090.70
model.cluster.kmeansIteration3.882.558.841.79
model.cluster.randomSeeds8.975.929.875.61
model.cluster.updateMean3.481.069.561.54
model.naive.bernoulli2.891.6214.456.49
model.naive.gaussian2.160.7310.905.57
model.naive.multinomial3.291.8212.185.29
model.neighbor.ballR1.780.37n/an/a
model.neighbor.mean2.781.089.772.73
model.neighbor.nearestK2.531.308.120.64
model.neural.simpleLayers3.472.4110.804.74
model.reg.gaussianProcess2.676.319.263.69
model.reg.linear2.462.368.020.46
model.reg.linearVariance7.9918.8710.012.37
model.svm.score1.790.329.421.22
model.tree.compoundTest1.800.568.060.25
model.tree.missingTest1.580.058.340.62
model.tree.missingWalk1.610.19n/an/a
model.tree.simpleTest1.580.078.220.60
model.tree.simpleTree1.780.408.600.08
model.tree.simpleWalk1.590.13n/an/a
model.tree.surrogateTest1.630.238.350.90
parse.double7.410.6316.217.66
parse.float10.443.2220.922.39
parse.int4.551.3814.589.35
parse.long3.961.6011.973.91
prob.dist.betaCDF2.053.058.280.50
prob.dist.betaPDF2.063.028.500.56
prob.dist.betaQF4.868.878.070.68
prob.dist.binomialCDF2.143.268.430.68
prob.dist.binomialPDF2.103.078.090.50
prob.dist.binomialQF4.747.018.360.81
prob.dist.cauchyCDF19.670.438.580.76
prob.dist.cauchyPDF19.580.307.980.44
prob.dist.cauchyQF4.606.768.360.62
prob.dist.chi2CDF7.9413.968.640.97
prob.dist.chi2PDF2.576.609.863.52
prob.dist.chi2QF4.0510.318.390.54
prob.dist.exponentialCDF1.600.158.661.46
prob.dist.exponentialPDF1.630.298.821.61
prob.dist.exponentialQF1.680.328.500.84
prob.dist.fCDF20.040.688.380.75
prob.dist.fPDF2.804.638.130.76
prob.dist.fQF8.6512.768.170.91
prob.dist.gammaCDF2.554.268.230.33
prob.dist.gammaPDF2.413.928.410.34
prob.dist.gammaQF4.748.088.060.61
prob.dist.gaussianCDF1.840.327.970.65
prob.dist.gaussianLL1.570.047.950.54
prob.dist.gaussianQF1.620.108.070.58
prob.dist.geometricCDF19.630.228.771.11
prob.dist.geometricPDF19.700.518.371.07
prob.dist.geometricQF1.690.328.101.08
prob.dist.hypergeometricCDF5.347.388.640.48
prob.dist.hypergeometricPDF5.397.398.370.61
prob.dist.hypergeometricQF20.175.928.540.76
prob.dist.lognormalCDF19.840.508.640.66
prob.dist.lognormalPDF19.710.538.190.69
prob.dist.lognormalQF8.2216.868.220.72
prob.dist.negativeBinomialCDF2.644.358.280.61
prob.dist.negativeBinomialPDF1.740.628.390.89
prob.dist.negativeBinomialQF4.918.698.130.73
prob.dist.poissonCDF9.699.388.782.02
prob.dist.poissonPDF2.864.738.580.69
prob.dist.poissonQF4.416.878.420.71
prob.dist.tCDF20.571.198.661.69
prob.dist.tPDF20.131.149.352.76
prob.dist.tQF6.6411.208.821.73
prob.dist.uniformCDF19.600.528.290.69
prob.dist.uniformPDF19.610.528.550.78
prob.dist.uniformQF19.650.708.120.60
prob.dist.weibullCDF3.024.968.160.51
prob.dist.weibullPDF3.024.958.130.55
prob.dist.weibullQF19.730.588.150.43
rand.bytes1.950.617.581.12
rand.choice1.540.047.340.41
rand.choices1.700.457.310.34
rand.double1.570.098.161.26
rand.float1.540.067.951.19
rand.gaussian1.600.01n/an/a
rand.histogram2.360.929.381.16
rand.int1.610.2210.734.75
rand.long1.550.059.963.97
rand.sample2.210.427.811.92
rand.string2.361.457.221.06
rand.uuid42.720.11n/an/a
re.contains5.013.2912.250.37
re.count3.881.8212.260.38
re.findall4.172.6912.070.46
re.findfirst2.970.6612.220.57
re.findgroupsall5.623.9012.220.25
re.findgroupsfirst4.412.1312.310.37
re.groups3.962.7112.220.49
re.groupsall5.563.8112.340.28
re.index22.5419.9612.310.82
re.indexall3.891.9915.464.96
re.replaceall3.501.8916.897.95
re.replacefirst3.151.0015.274.67
re.replacelast3.190.9112.380.70
re.rindex3.110.7712.540.48
re.split4.893.9612.110.35
s.concat1.570.02n/an/a
s.contains1.540.09n/an/a
s.count1.610.23n/an/a
s.endswith1.500.01n/an/a
s.hex4.252.899.104.00
s.index1.510.02n/an/a
s.int3.351.068.321.64
s.join1.720.07n/an/a
s.len1.500.01n/an/a
s.lower1.550.03n/an/a
s.lstrip1.940.49n/an/a
s.number2.551.819.081.97
s.repeat1.810.60n/an/a
s.replaceall2.300.83n/an/a
s.replacefirst1.560.06n/an/a
s.replacelast1.570.08n/an/a
s.rindex1.770.36n/an/a
s.rstrip1.660.16n/an/a
s.split1.770.30n/an/a
s.startswith1.520.01n/an/a
s.strip1.740.25n/an/a
s.substr1.550.10n/an/a
s.substrto1.610.02n/an/a
s.translate1.610.09n/an/a
s.upper1.590.16n/an/a
stat.change.updateCUSUM1.540.04n/an/a
stat.change.updateTrigger1.640.087.910.50
stat.change.zValue1.540.04n/an/a
stat.sample.fillCounter1.820.46n/an/a
stat.sample.fillHistogram3.353.959.584.45
stat.sample.fillHistogram2d2.990.4612.162.66
stat.sample.forecast1HoltWinters1.560.058.140.44
stat.sample.forecastHoltWinters1.870.448.060.48
stat.sample.topN1.940.53n/an/a
stat.sample.update1.570.04n/an/a
stat.sample.updateCovariance37.46160.089.070.68
stat.sample.updateEWMA1.580.058.240.66
stat.sample.updateHoltWinters1.610.0610.041.60
stat.sample.updateHoltWintersPeriodic1.780.128.280.65
stat.sample.updateWindow5.018.049.301.70
stat.test.chi2Prob5.1610.958.301.40
stat.test.kolmogorov6.6715.15n/an/a
stat.test.mahalanobis8.084.2010.511.58
stat.test.pull1.520.0810.314.93
stat.test.reducedChi21.530.04n/an/a
stat.test.residual1.540.2712.805.78
stat.test.updateChi21.690.29n/an/a
time.dayOfMonth1.890.118.260.70
time.dayOfWeek1.860.139.673.12
time.dayOfYear1.840.058.090.75
time.hourOfDay1.860.058.300.81
time.isDayOfMonth1.880.148.440.54
time.isDayOfWeek1.860.218.040.62
time.isDayOfYear1.840.188.460.74
time.isHourOfDay1.890.258.080.79
time.isMinuteOfHour1.890.128.260.58
time.isMonthOfYear1.880.217.990.65
time.isSecondOfMinute1.900.138.200.55
time.isWeekend1.940.289.523.44
time.isWorkHours1.950.269.403.24
time.makeTimestamp1.870.2112.461.31
time.minuteOfHour1.960.058.200.73
time.monthOfYear1.940.319.353.21
time.secondOfMinute1.940.068.520.81
time.year4.441.8311.134.36
u-1.550.126.770.06
|1.510.04n/an/a
||1.510.02n/an/a
|||1.530.08n/an/a
~1.510.01n/an/a