blob: 8650b55072e8b9d9460d30f562994d28e4dfb726 [file] [log] [blame]
= Stream Evaluator Reference
:toclevels: 1
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
Stream evaluators are different then stream sources or stream decorators. Both
stream sources and stream decorators return streams of tuples. Stream evaluators are more like
a traditional function that evaluates its parameters and
returns an result. That result can be a single value, array, map or other structure.
Stream evaluators can be nested so that the output of an evaluator becomes the input
for another evaluator.
Stream evaluators can be called in different contexts. For example a stream evaluator
can be called on its own or it can be called within the context of a streaming expression.
== abs
The `abs` function will return the absolute value of the provided single parameter. The `abs` function will fail to execute if the value is non-numeric. If a null value is found then null will be returned as the result.
=== abs Parameters
* `Field Name | Raw Number | Number Evaluator`
=== abs Syntax
The expressions below show the various ways in which you can use the `abs` evaluator. Only one parameter is accepted. Returns a numeric value.
[source,text]
----
abs(1) // 1, not really a good use case for it
abs(-1) // 1, not really a good use case for it
abs(add(fieldA,fieldB)) // absolute value of fieldA + fieldB
abs(fieldA) // absolute value of fieldA
----
== acos
The `acos` function returns the trigonometric arccosine of a number.
=== acos Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the arccosine of.
=== acos Syntax
[source,text]
----
acos(100.4) // returns the arccosine of 100.4
acos(fieldA) // returns the arccosine for fieldA.
if(gt(fieldA,fieldB),sin(fieldA),sin(fieldB)) // if fieldA > fieldB then return the arccosine of fieldA, else return the arccosine of fieldB
----
== add
The `add` function will take 2 or more numeric values and add them together. The `add` function will fail to execute if any of the values are non-numeric. If a null value is found then null will be returned as the result.
=== add Parameters
* `Field Name | Raw Number | Number Evaluator`
* `Field Name | Raw Number | Number Evaluator`
* `......`
* `Field Name | Raw Number | Number Evaluator`
=== add Syntax
The expressions below show the various ways in which you can use the `add` evaluator. The number and order of these parameters do not matter and is not limited except that at least two parameters are required. Returns a numeric value.
[source,text]
----
add(1,2,3,4) // 1 + 2 + 3 + 4 == 10
add(1,fieldA) // 1 + value of fieldA
add(fieldA,1.4) // value of fieldA + 1.4
add(fieldA,fieldB,fieldC) // value of fieldA + value of fieldB + value of fieldC
add(fieldA,div(fieldA,fieldB)) // value of fieldA + (value of fieldA / value of fieldB)
add(fieldA,if(gt(fieldA,fieldB),fieldA,fieldB)) // if fieldA > fieldB then fieldA + fieldA, else fieldA + fieldB
----
== analyze
The `analyze` function analyzes text using a Lucene/Solr analyzer and returns a list of tokens
emitted by the analyzer. The `analyze` function can be called on its own or within the
`<<stream-decorator-reference.adoc#select,select>>` and `<<stream-decorator-reference.adoc#cartesianproduct,cartesianProduct>>` streaming expressions.
=== analyze Parameters
* `Field Name` | `Raw Text`: Either the field in a tuple or the raw text to be analyzed.
* `Analyzer Field Name`: The field name of the analyzer to use to analyze the text.
=== analyze Syntax
The expressions below show the various ways in which you can use the `analyze` evaluator.
* Analyze the raw text: `analyze("hello world", analyzerField)`
* Analyze a text field within a `select` expression. This will annotate tuples with the output of the analyzer: `select(expr, analyze(textField, analyzerField) as outField)`
* Analyze a text field with a `cartesianProduct` expression. This will stream each token emitted by the analyzer in its own tuple: `cartesianProduct(expr, analyze(textField, analyzer) as outField)`
== and
The `and` function will return the logical AND of at least 2 boolean parameters. The function will fail to execute if any parameters are non-boolean or null. Returns a boolean value.
=== and Parameters
* `Field Name | Raw Boolean | Boolean Evaluator`
* `Field Name | Raw Boolean | Boolean Evaluator`
* `......`
* `Field Name | Raw Boolean | Boolean Evaluator`
=== and Syntax
The expressions below show the various ways in which you can use the `and` evaluator. At least two parameters are required, but there is no limit to how many you can use.
[source,text]
----
and(true,fieldA) // true && fieldA
and(fieldA,fieldB) // fieldA && fieldB
and(or(fieldA,fieldB),fieldC) // (fieldA || fieldB) && fieldC
and(fieldA,fieldB,fieldC,or(fieldD,fieldE),fieldF)
----
== anova
The `anova` function calculates the https://en.wikipedia.org/wiki/Analysis_of_variance[analysis of variance] for two or more numeric arrays.
=== anova Parameters
//TODO fill in details of Parameters
* `numeric array` ... (two or more)
=== anova Syntax
[source,text]
anova(numericArray1, numericArray2) // calculates ANOVA for two numeric arrays
anova(numericArray1, numericArray2, numericArray2) // calculates ANOVA for three numeric arrays
== array
The `array` function returns an array of numerics or other objects including other arrays.
=== array Parameters
//TODO fill in details of Parameters
* `numeric` | `array` ...
=== array Syntax
[source,text]
array(1, 2, 3) // Array of numerics
array(array(1,2,3), array(4,5,6)) // Array of arrays
== asin
The `asin` function returns the trigonometric arcsine of a number.
=== asin Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the arcsine of.
=== asin Syntax
[source,text]
----
asin(100.4) // returns the sine of 100.4
asine(fieldA) // returns the sine for fieldA.
if(gt(fieldA,fieldB),asin(fieldA),asin(fieldB)) // if fieldA > fieldB then return the asine of fieldA, else return the asine of fieldB
----
== atan
The `atan` function returns the trigonometric arctangent of a number.
=== atan Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the arctangent of.
=== atan Syntax
[source,text]
----
atan(100.4) // returns the arctangent of 100.4
atan(fieldA) // returns the arctangent for fieldA.
if(gt(fieldA,fieldB),atan(fieldA),atan(fieldB)) // if fieldA > fieldB then return the arctanget of fieldA, else return the arctangent of fieldB
----
== betaDistribution
The `betaDistribution` function returns a https://en.wikipedia.org/wiki/Beta_distribution[beta probability distribution]
based on its parameters. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>`, `<<kolmogorovSmirnov>>` and `<<cumulativeProbability>>` functions.
=== betaDistribution Parameters
* `double`: shape1
* `double`: shape2
=== betaDistribution Returns
A probability distribution function.
=== betaDistribution Syntax
[source,text]
betaDistribution(1, 5)
== binomialCoefficient
The `binomialCoefficient` function returns a https://en.wikipedia.org/wiki/Binomial_coefficient[Binomial Coefficient], the number of k-element subsets that can
be selected from an n-element set.
=== binomialCoefficient Parameters
* `integer`: [n] set
* `integer`: [k] subset
=== binomialCoefficient Returns
A long value: The number of k-element subsets that can be selected from an n-element set.
=== binomialCoefficient Syntax
[source,text]
binomialCoefficient(8, 3) // Returns the number of 3 element subsets from an 8 element set.
== binomialDistribution
The `binomialDistribution` function returns a https://en.wikipedia.org/wiki/Binomial_distribution[binomial probability distribution]
based on its parameters. This function is part of the probability distribution framework and is designed to
work with the `<<sample>>`, `<<probability>>` and `<<cumulativeProbability>>` functions.
=== binomialDistribution Parameters
* `integer`: number of trials
* `double`: probability of success
=== binomialDistribution Returns
A probability distribution function.
=== binomialDistribution Syntax
[source,text]
binomialDistribution(1000, .5)
== cbrt
The `cbrt` function returns the trigonometric cube root of a number.
=== cbrt Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the cube root of.
=== cbrt Syntax
[source,text]
----
cbrt(100.4) // returns the square root of 100.4
cbrt(fieldA) // returns the square root for fieldA.
if(gt(fieldA,fieldB),cbrt(fieldA),cbrt(fieldB)) // if fieldA > fieldB then return the cbrt of fieldA, else return the cbrt of fieldB
----
== ceil
The `ceil` function rounds a decimal value to the next highest whole number.
=== ceil Parameters
* `Field Name | Raw Number | Number Evaluator`: The decimal to round up.
=== ceil Syntax
The expressions below show the various ways in which you can use the `ceil` evaluator.
[source,text]
----
ceil(100.4) // returns 101.
ceil(fieldA) // returns the next highest whole number for fieldA.
if(gt(fieldA,fieldB),ceil(fieldA),ceil(fieldB)) // if fieldA > fieldB then return the ceil of fieldA, else return the ceil of fieldB.
----
== col
The `col` function returns a numeric array from a list of tuples. The `col`
function is used to create numeric arrays from stream sources.
=== col Parameters
//TODO fill in details of Parameters
* `list of Tuples`
* `field name`: The field to create the array from.
=== col Syntax
[source,text]
col(tupleList, fieldName)
== colAt
The `colAt` function returns the column of a matrix at a specific index as a numeric array.
=== colAt Parameters
* `matrix`: the matrix to operate on
* `integer`: the index of the column to return
=== colAt Syntax
[source,text]
colAt(matrix, 10)
=== colAt Returns
numeric array: the column of the matrix
== columnCount
The `columnCount` function returns the number of columns in a `matrix`.
=== columnCount Parameters
* `matrix`: the matrix to operate on
=== columnCount Syntax
[source,text]
columnCount(matrix)
=== columnCount Returns
integer: number columns in the matrix.
== constantDistribution
The `constantDistribution` function returns a constant probability distribution based on its parameter.
This function is part of the probability distribution framework and is designed to
work with the `<<sample>>` and `<<cumulativeProbability>>` functions.
When sampled the constant distribution always returns its constant value.
=== constantDistribution Parameters
* `double`: constant value
=== constantDistribution Returns
A probability distribution function.
=== constantDistribution Syntax
[source,text]
constantDistribution(constantValue)
== conv
The `conv` function returns the https://en.wikipedia.org/wiki/Convolution[convolution] of two numeric arrays.
=== conv Parameters
* `numeric array`
* `numeric array`
=== conv Syntax
[source,text]
conv(numericArray1, numericArray2)
== copyOf
The `copyOf` function creates a copy of a numeric array.
=== copyOf Parameters
* `numeric array`
* `length`: The length of the copied array. The returned array will be right padded with zeros if the length parameter exceeds the size of the original array.
=== copyOf Syntax
[source,text]
copyOf(numericArray, length)
== copyOfRange
The `copyOfRange` function creates a copy of a range of a numeric array.
=== copyOfRange Parameters
//TODO fill in details of Parameters
* `numeric array`
* `start index`
* `end index`
=== copyOfRange Syntax
[source,text]
copyOfRange(numericArray, startIndex, endIndex)
== corr
The `corr` function returns the correlation of two numeric arrays or the correlation matrix for a matrix.
The `corr` function support Pearson's, Kendall's and Spearman's correlations.
=== corr Positional Parameters
* `numeric array`: The first numeric array
* `numeric array`: The second numeric array
OR
* `matrix`: The matrix to compute the correlation matrix for. Note that correlation is computed between the `columns` in the matrix.
=== corr Named Parameters
* `type`: (Optional) The type of correlation. Possible values are `pearsons`, `kendalls`, or `spearmans`. The default is `pearsons`.
=== corr Syntax
[source,text]
corr(numericArray1, numericArray2) // Compute the Pearsons correlation for two numeric arrays
corr(numericArray1, numericArray2, type=kendalls) // Compute the Kendalls correlation for two numeric arrays
corr(matrix) // Compute the Pearsons correlation matrix for a matrix
corr(matrix, type=spearmans) // Compute the Spearmans correlation matrix for a matrix
=== corr Returns
number | matrix: Either the correlation or correlation matrix.
== cos
The `cos` function returns the trigonometric cosine of a number.
=== cos Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the hyperbolic cosine of.
=== cos Syntax
[source,text]
----
cos(100.4) // returns the arccosine of 100.4
cos(fieldA) // returns the arccosine for fieldA.
if(gt(fieldA,fieldB),cos(fieldA),cos(fieldB)) // if fieldA > fieldB then return the arccosine of fieldA, else return the cosine of fieldB
----
== cosineSimilarity
The `cosineSimilarity` function returns the https://en.wikipedia.org/wiki/Cosine_similarity[cosine similarity] of two numeric arrays.
=== cosineSimilarity Parameters
* `numeric array`
* `numeric array`
=== cosineSimilarity Returns
A numeric.
=== cosineSimilarity Syntax
[source,text]
----
cosineSimilarity(numericArray, numericArray)
----
== cov
The `cov` function returns the covariance of two numeric array or the covariance matrix for matrix.
=== cov Parameters
* `numeric array`: The first numeric array
* `numeric array`: The second numeric array
OR
* `matrix`: The matrix to compute the covariance matrix from. Note that covariance is computed between the `columns` in the matrix.
=== cov Syntax
[source,text]
cov(numericArray, numericArray) // Computes the covariance of a two numeric arrays
cov(matrix) // Computes the covariance matrix for the matrix.
=== cov Returns
number | matrix: Either the covariance or covariance matrix.
== cumulativeProbability
The `cumulativeProbability` function returns the cumulative probability of a random variable within a
probability distribution. The cumulative probability is the total probability of
all random variables less then or equal to a random variable.
=== cumulativeProbability Parameters
* `probability distribution`
* `number`: Value to compute the probability for.
=== cumulativeProbability Returns
A double: the cumulative probability.
=== cumulativeProbability Syntax
[source,text]
cumulativeProbability(normalDistribution(500, 25), 502) // Returns the cumulative probability of the random sample 502 in a normal distribution with a mean of 500 and standard deviation of 25.
== derivative
The `derivative` function returns the https://en.wikipedia.org/wiki/Derivative[derivative] of a function. The derivative function
can compute the derivative of the <<spline>> function and the <<loess>> function. The derivative can also
take the derivative of a derivative.
=== derivative Parameters
* `spline` | `loess` | `akima` | `lerp` | `derivative`: The functions to compute the derivative for.
=== derivative Syntax
[source,text]
derivative(spline(...))
derivative(loess(...))
derivative(derivative(...))
=== derivative Returns
function: The function can be treated as both a `numeric array` and `function`.
== describe
The `describe` function returns a tuple containing the descriptive statistics for an array.
=== describe Parameters
* `numeric array`
=== describe Syntax
[source,text]
describe(numericArray)
== diff
The `diff` functions performs https://www.otexts.org/fpp/8/1[time series differencing].
Time series differencing is often used to make a time series stationary before further analysis.
=== diff Parameters
* `numeric array`: The time series data.
* `integer`: (Optional) The lag. Defaults to 1.
=== diff Syntax
[source,text]
diff(numericArray1) // Perform time series differencing with a default lag of 1.
diff(numericArray1, 30) // Perform time series differencing with a lag of 30.
=== diff Returns
numeric array: The differenced time series data. The size of the array will be equal to (original array size - lag).
== distance
The `distance` function computes the distance of two numeric arrays or the distance matrix for a matrix.
=== distance Positional Parameters
* `numeric array`: The first numeric array
* `numeric array`: The second numeric array
OR
* `matrix`: The matrix to compute the distance matrix for. Note that distance is computed between the `columns` in the matrix.
=== distance Named Parameters
* `type`: (Optional) The distance type. Possible values are `euclidean`, `manhattan`, `canberra`, or `earthMovers`. The default is `euclidean`.
=== distance Syntax
[source,text]
distance(numericArray1, numericArray2) // Computes the euclidean distance for two numeric arrays.
distance(numericArray1, numericArray2, type=manhattan) // Computes the manhattan distance for two numeric arrays.
distance(matrix) // Computes the euclidean distance matrix for a matrix.
distance(matrix, type=canberra) // Computes the canberra distance matrix for a matrix.
=== distance Returns
number | matrix: Either the distance or distance matrix.
== div
The `div` function will take two numeric values and divide them. The function will fail to execute if any of the values are non-numeric or null, or the 2nd value is 0. Returns a numeric value.
=== div Parameters
* `Field Name | Raw Number | Number Evaluator`
* `Field Name | Raw Number | Number Evaluator`
=== div Syntax
The expressions below show the various ways in which you can use the `div` evaluator. The first value will be divided by the second and as such the second cannot be 0.
[source,text]
----
div(1,2) // 1 / 2
div(1,fieldA) // 1 / fieldA
div(fieldA,1.4) // fieldA / 1.4
div(fieldA,add(fieldA,fieldB)) // fieldA / (fieldA + fieldB)
----
== dotProduct
The `dotProduct` function returns the https://en.wikipedia.org/wiki/Dot_product[dotproduct] of two numeric arrays.
=== dotProduct Parameters
* `numeric array`
* `numeric array`
=== dotProduct Returns
A number.
=== dotProduct Syntax
[source,text]
dotProduct(numericArray, numericArray)
== ebeAdd
The `ebeAdd` function performs an element-by-element addition of two numeric arrays.
=== ebeAdd Parameters
* `numeric array`
* `numeric array`
=== ebeAdd Returns
A numeric array.
=== ebeAdd Syntax
[source,text]
ebeAdd(numericArray, numericArray)
== ebeDivide
The `ebeDivide` function performs an element-by-element division of two numeric arrays.
=== ebeDivide Parameters
* `numeric array`
* `numeric array`
=== ebeDivide Returns
A numeric array.
=== ebeDivide Syntax
[source,text]
ebeDivide(numericArray, numericArray)
== ebeMultiple
The `ebeMultiply` function performs an element-by-element multiplication of two numeric arrays.
=== ebeMultiply Parameters
* `numeric array`
* `numeric array`
=== ebeMultiply Returns
A numeric array.
=== ebeMultiply Syntax
[source,text]
ebeMultiply(numericArray, numericArray)
== ebeSubtract
The `ebeSubtract` function performs an element-by-element subtraction of two numeric arrays.
=== ebeSubtract Parameters
* `numeric array`
* `numeric array`
=== ebeSubtract Returns
A numeric array.
=== ebeSubtract Syntax
[source,text]
ebeSubtract(numericArray, numericArray)
== empiricalDistribution
The `empiricalDistribution` function returns https://en.wikipedia.org/wiki/Empirical_distribution_function[empirical distribution function], a continuous probability distribution function based
on an actual data set. This function is part of the probability distribution framework and is designed to work with the `<<sample>>`, `<<kolmogorovSmirnov>>` and `<<cumulativeProbability>>` functions.
This function is designed to work with continuous data. To build a distribution from
a discrete data set use the `<<enumeratedDistribution>>`.
=== empiricalDistribution Parameters
* `numeric array`: empirical observations
=== empiricalDistribution Returns
A probability distribution function.
=== empiricalDistribution Syntax
[source,text]
empiricalDistribution(numericArray)
== enumeratedDistribution
The `enumeratedDistribution` function returns a discrete probability distribution function based
on an actual data set or a pre-defined set of data and probabilities.
This function is part of the probability distribution framework and is designed to
work with the `<<sample>>`, `<<probability>>` and `<<cumulativeProbability>>` functions.
The enumeratedDistribution can be called in two different scenarios:
1) Single array of discrete values. This works like an empirical distribution for
discrete data.
2) An array of singleton discrete values and an array of double values representing
the probabilities of the discrete values.
This function is designed to work with discrete data. To build a distribution from
a continuous data set use the `<<empiricalDistribution>>`.
=== enumeratedDistribution Parameters
* `integer array`: discrete observations or singleton discrete values.
* `double array`: (Optional) values representing the probabilities of the singleton discrete values.
=== enumeratedDistribution Returns
A probability distribution function.
=== enumeratedDistribution Syntax
[source,text]
enumeratedDistribution(integerArray) // This creates an enumerated distribution from the observations in the numeric array.
enumeratedDistribution(array(1,2,3,4), array(.25,.25,.25,.25)) // This creates an enumerated distribution with four discrete values (1,2,3,4) each with a probability of .25.
== eor
The `eor` function will return the logical exclusive or of at least two boolean parameters. The function will fail to execute if any parameters are non-boolean or null. Returns a boolean value.
=== eor Parameters
* `Field Name | Raw Boolean | Boolean Evaluator`
* `Field Name | Raw Boolean | Boolean Evaluator`
* `......`
* `Field Name | Raw Boolean | Boolean Evaluator`
=== eor Syntax
The expressions below show the various ways in which you can use the `eor` evaluator. At least two parameters are required, but there is no limit to how many you can use.
[source,text]
----
eor(true,fieldA) // true iff fieldA is false
eor(fieldA,fieldB) // true iff either fieldA or fieldB is true but not both
eor(eq(fieldA,fieldB),eq(fieldC,fieldD)) // true iff either fieldA == fieldB or fieldC == fieldD but not both
----
== eq
The `eq` function will return whether all the parameters are equal, as per Java's standard `equals(...)` function. The function accepts parameters of any type, but will fail to execute if all the parameters are not of the same type. That is, all are Boolean, all are String, or all are Numeric. If any any parameters are null and there is at least one parameter that is not null then false will be returned. Returns a boolean value.
=== eq Parameters
* `Field Name | Raw Value | Evaluator`
* `Field Name | Raw Value | Evaluator`
* `......`
* `Field Name | Raw Value | Evaluator`
=== eq Syntax
The expressions below show the various ways in which you can use the `eq` evaluator.
[source,text]
----
eq(1,2) // 1 == 2
eq(1,fieldA) // 1 == fieldA
eq(fieldA,val(foo)) fieldA == "foo"
eq(add(fieldA,fieldB),6) // fieldA + fieldB == 6
----
== expMovingAge
The `expMovingAverage` function computes an https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average[exponential moving average] for a numeric array.
=== expMovingAge Parameters
* `numeric array`: The array to compute the exponential moving average from.
* `integer`: window size
=== expMovingAvg Returns
A numeric array. The first element of the returned array will start from the windowSize-1 index of the original array.
=== expMovingAvg Syntax
[source,text]
----
expMovingAvg(numericArray, 5) //Computes an exponential moving average with a window size of 5.
----
== factorial
The `factorial` function returns the https://en.wikipedia.org/wiki/Factorial[factorial] of its parameter.
=== factorial Parameters
* `integer`: The value to compute the factorial for. The largest supported value of this parameter is 170.
=== factorial Returns
A double.
=== factorial Syntax
[source,text]
----
factorial(100) //Computes the factorial of 100
----
== finddelay
The `finddelay` function performs a cross-correlation between two numeric arrays and returns the delay.
=== finddelay Parameters
* `numeric array`
* `numeric array`
=== finddelay Syntax
[source,text]
finddelay(numericArray1, numericArray2)
== floor
The `floor` function rounds a decimal value to the next lowest whole number.
=== floor Parameters
* `Field Name | Raw Number | Number Evaluator`: The decimal to round down.
=== floor Syntax
The expressions below show the various ways in which you can use the `floor` evaluator.
[source,text]
----
floor(100.4) // returns 100.
ceil(fieldA) // returns the next lowestt whole number for fieldA.
if(gt(fieldA,fieldB),floor(fieldA),floor(fieldB)) // if fieldA > fieldB then return the floor of fieldA, else return the floor of fieldB.
----
== freqTable
The `freqTable` function returns a https://en.wikipedia.org/wiki/Frequency_distribution[frequency distribution] from
an array of discrete values.
This function is designed to work with discrete values. To work with continuous data
use the `<<hist>>` function.
=== freqTable Parameters
* `integer array`: The values to build the frequency distribution from.
=== freqTable Returns
A list of tuples containing the frequency information for each discrete value.
=== freqTable Syntax
[source,text]
----
freqTable(integerArray)
----
== gammaDistribution
The `gammaDistribution` function returns a https://en.wikipedia.org/wiki/Gamma_distribution[gamma probability distribution] based on its parameters. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>`, `<<kolmogorovSmirnov>>` and `<<cumulativeProbability>>` functions.
=== gammaDistribution Parameters
* `double`: shape
* `double`: scale
=== gammaDistribution Returns
A probability distribution function,
=== gammaDistribution Syntax
[source,text]
gammaDistribution(1, 10)
== geometricDistribution
The `geometricDistribution` function returns a https://en.wikipedia.org/wiki/Geometric_distribution[geometric probability distribution] based on its parameters. This function is part of the
probability distribution framework and is designed to work with the <<sample>>, <<probability>> and <<cumulativeProbability>> functions.
=== geometricDistribution Parameters
* `double`: probability
=== geometricDistribution Syntax
[source,text]
geometricDistribution(.5) // Creates a geometric distribution with probability of .5
=== geometricDistribution Returns
A probability distribution function
== getAttribute
The `getAttribute` function returns an attribute from a `matrix` by its key. Any function that returns a `matrix` can
also set attributes on the `matrix` with additional information. The `setAttribute` function can also be used
to set attributes on a `matrix`. The key to an attribute is always a string. The value of attribute can be any object
including numerics, arrays, maps, matrixes, etc.
=== getAttribute Parameters
* `matrix`: The matrix to set the attribute on
* `string`: The key for the attribute
=== getAttribute Syntax
[source,text]
getAttribute(matrix, key)
=== getAttribute Returns
object: any object
== getAttributes
The `getAttributes` function returns the attribute map from matrix. See the `getAttribute` function for more details
on attributes.
=== getAttributes Parameters
* `matrix`: The matrix to retrieve the attribute map from.
=== getAttributes Syntax:
[source,text]
getAttributes(matrix)
=== getAttributes Returns
map: The map of attributes.
== getColumnLabels
The `getColumnLabels` function returns the columns labels of a matrix. The column labels can be optionally
set by any function that returns a matrix. The column labels can also be set via the `setColumnLabels` function.
=== getColumnLabels Parameters
* `matrix`: The matrix to return the column labels of.
=== getColumnLabels Syntax
getColumnLabels(matrix)
=== getColumnLabels Returns
string array: The labels for each column in the matrix
== getRowLabels
The `getRowLabels` function returns the row labels of a matrix. The row labels can be optionally
set by any function that returns a matrix. The row labels can also be set via the `setRowLabels` function.
=== getRowLabels Parameters
* `matrix`: The matrix to return the row labels from.
=== getRowLabels Syntax
getRowLabels(matrix)
=== getRowLabels Returns
string array: The labels for each row in the matrix
== getValue
The `getValue` function returns the value of a single tuple entry by key.
=== getValue Parameters
* `tuple`: The tuple to return the entry from.
* `key`: The key of the entry to return the value for.
=== getValue Syntax
getValue(tuple, key)
=== getValue Returns
object: Returns an object of the same type as the tuple entry.
== grandSum
The `grandSum` function sums all the values in a matrix.
=== grandSum Parameters
* `matrix`: The matrix to operate on.
=== grandSum Syntax
[source,text]
grandSum(matrix)
=== grandSum Returns
number: the sum of all the values in the matrix.
== gt
The `gt` function will return whether the first parameter is greater than the second parameter. The function accepts numeric or string parameters, but will fail to execute if all the parameters are not of the same type. That is, all are String or all are Numeric. If any any parameters are null then an error will be raised. Returns a boolean value.
=== gt Parameters
* `Field Name | Raw Value | Evaluator`
* `Field Name | Raw Value | Evaluator`
=== gt Syntax
The expressions below show the various ways in which you can use the `gt` evaluator.
[source,text]
----
gt(1,2) // 1 > 2
gt(1,fieldA) // 1 > fieldA
gt(fieldA,val(foo)) // fieldA > "foo"
gt(add(fieldA,fieldB),6) // fieldA + fieldB > 6
----
== gteq
The `gteq` function will return whether the first parameter is greater than or equal to the second parameter. The function accepts numeric and string parameters, but will fail to execute if all the parameters are not of the same type. That is, all are String or all are Numeric. If any any parameters are null then an error will be raised. Returns a boolean value.
=== gteq Parameters
* `Field Name | Raw Value | Evaluator`
* `Field Name | Raw Value | Evaluator`
=== gteq Syntax
The expressions below show the various ways in which you can use the `gteq` evaluator.
[source,text]
----
gteq(1,2) // 1 >= 2
gteq(1,fieldA) // 1 >= fieldA
gteq(fieldA,val(foo)) fieldA >= "foo"
gteq(add(fieldA,fieldB),6) // fieldA + fieldB >= 6
----
== hist
The `hist` function creates a histogram from a numeric array. The hist function is designed
to work with continuous variables.
=== hist Parameters
//TODO fill in details of Parameters
* `numeric array`
* `bins`: The number of bins in the histogram. Each returned tuple contains
summary statistics for the observations that were within the bin.
=== hist Syntax
[source,text]
hist(numericArray, bins)
== hsin
The `hsin` function returns the trigonometric hyperbolic sine of a number.
=== hsin Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the hyperbolic sine of.
=== hsin Syntax
[source,text]
----
hsin(100.4) // returns the hsine of 100.4
hsin(fieldA) // returns the hsine for fieldA.
if(gt(fieldA,fieldB),sin(fieldA),sin(fieldB)) // if fieldA > fieldB then return the hsine of fieldA, else return the hsine of fieldB
----
== if
The `if` function works like a standard conditional if/then statement. If the first parameter is true, then the second parameter will be returned, else the third parameter will be returned. The function accepts a boolean as the first parameter and anything as the second and third parameters. An error will occur if the first parameter is not a boolean or is null.
=== if Parameters
* `Field Name | Raw Value | Boolean Evaluator`
* `Field Name | Raw Value | Evaluator`
* `Field Name | Raw Value | Evaluator`
=== if Syntax
The expressions below show the various ways in which you can use the `if` evaluator.
[source,text]
----
if(fieldA,fieldB,fieldC) // if fieldA is true then fieldB else fieldC
if(gt(fieldA,5), fieldA, 5) // if fieldA > 5 then fieldA else 5
if(eq(fieldB,null), null, div(fieldA,fieldB)) // if fieldB is null then null else fieldA / fieldB
----
== indexOf
The `indexOf` function returns the index of a string in an array of strings.
=== indexOf Parameters
* `string array`: The array to operate on.
* `string`: The string to search for in the array.
=== indexOf Syntax
[source,text]
indexOf(stringArray, string)
=== indexOf Returns
integer: The index of the string in the array or -1 if the string is not found.
== integrate
The `integrate` function computes the integral of an interpolation function for a specific range of the curve.
=== integrate Parameters
* `spline` | `akima` | `lerp` | `loess`: The interpolation function to compute the integral for.
* `numeric`: start of integral range
* `numeric`: end of integral range
=== integrate Syntax
[source,text]
integrate(function, start, end)
=== integrate Returns
numeric: The integral
== length
The `length` function returns the length of a numeric array.
=== length Parameters
//TODO fill in details of Parameters
* `numeric array`
=== length Syntax
[source,text]
length(numericArray)
== lerp (TOD0)
== loess
The `leoss` function is a smoothing curve fitter which uses a https://en.wikipedia.org/wiki/Local_regression[local regression] algorithm.
Unlike the <<spline>> function which touches each control point, the `loess` function puts a smooth curve through
the control points without having to touch the control points. The `loess` result can be used by the <<derivative>> function to produce smooth derivatives from
data that is not smooth.
=== loess Positional Parameters
* `numeric array`: (Optional) x values. If omitted a sequence will be created for the x values.
* `numeric array`: y values
=== loess Named Parameters
* `bandwidth`: (Optional) The percent of the data points to use when drawing the local regression line, defaults to .25. Decreasing the bandwidth increases the number of curves that loess can fit.
* `robustIterations`: (Optional) The number of iterations used to smooth outliers, defaults to 2.
=== loess Syntax
[source,text]
loess(yValues) // This creates the xValues automatically and fits a smooth curve through the data points.
loess(xValues, yValues) // This will fit a smooth curve through the data points.
loess(xValues, yValues, bandwidth=.15) // This will fit a smooth curve through the data points using 15 percent of the data points for each local regression line.
=== loess Returns
function: The function can be treated as both a `numeric array` of the smoothed data points and `function`.
== log
The `log` function will return the natural log of the provided single parameter. The `log` function will fail to execute if the value is non-numeric. If a null value is found, then null will be returned as the result.
=== log Parameters
* `Field Name | Raw Number | Number Evaluator`
=== log Syntax
The expressions below show the various ways in which you can use the `log` evaluator. Only one parameter is accepted. Returns a numeric value.
[source,text]
----
log(100)
log(add(fieldA,fieldB))
log(fieldA)
----
== logNormalDistribution
The `logNormalDistribution` function returns a https://en.wikipedia.org/wiki/Log-normal_distribution[log normal probability distribution] based on its parameters. This function is part of the probability distribution framework and is designed to
work with the `<<sample>>`, `<<kolmogorovSmirnov>>` and `<<cumulativeProbability>>` functions.
=== logNormalDistribution Parameters
* `double`: shape
* `double`: scale
=== logNormalDistribution Returns
A probability distribution function.
=== logNormalDistribution Syntax
[source,text]
logNormalDistribution(.3, .0)
== kolmogorovSmirnov
The `kolmogorovSmirnov` function performs a https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test[Kolmogorov Smirnov test],
between a reference continuous probability distribution and a sample set.
The supported distribution functions are: `<<empiricalDistribution>>`, `<<normalDistribution>>`, `<<logNormalDistribution>>`, `<<weibullDistribution>>`, `<<gammaDistribution>>`, and `<<betaDistribution>>`.
=== kolmogorovSmirnov Parameters
* `continuous probability distribution`: Reference distribution
* `numeric array`: sample set
=== kolmogorovSmirnov Returns
A result tuple: A tuple containing the p-value and d-statistic for the test result.
=== kolmogorovSmirnov Syntax
[source,text]
kolmogorovSmirnov(normalDistribution(10, 2), sampleSet)
== lt
The `lt` function will return whether the first parameter is less than the second parameter. The function accepts numeric or string parameters, but will fail to execute if all the parameters are not of the same type. That is, all are String or all are Numeric. If any any parameters are null then an error will be raised. Returns a boolean value.
=== lt Parameters
* `Field Name | Raw Value | Evaluator`
* `Field Name | Raw Value | Evaluator`
=== lt Syntax
The expressions below show the various ways in which you can use the `lt` evaluator.
[source,text]
----
lt(1,2) // 1 < 2
lt(1,fieldA) // 1 < fieldA
lt(fieldA,val(foo)) fieldA < "foo"
lt(add(fieldA,fieldB),6) // fieldA + fieldB < 6
----
== lteq
The `lteq` function will return whether the first parameter is less than or equal to the second parameter. The function accepts numeric and string parameters, but will fail to execute if all the parameters are not of the same type. That is, all are String or all are Numeric. If any any parameters are null then an error will be raised. Returns a boolean value.
=== lteq Parameters
* `Field Name | Raw Value | Evaluator`
* `Field Name | Raw Value | Evaluator`
=== lteq Syntax
The expressions below show the various ways in which you can use the `lteq` evaluator.
[source,text]
----
lteq(1,2) // 1 <= 2
lteq(1,fieldA) // 1 <= fieldA
lteq(fieldA,val(foo)) fieldA <= "foo"
lteq(add(fieldA,fieldB),6) // fieldA + fieldB <= 6
----
== markovChain
The `markovChain` function can be used to perform https://en.wikipedia.org/wiki/Markov_chain[Markov Chain] simulations.
The `markovChain` function takes as its parameter a https://en.wikipedia.org/wiki/Stochastic_matrix[transition matrix] and
returns a mathematical model that can be sampled using the <<sample>> function. Each sample taken
from the Markov Chain represents the current state of system.
=== markovChain Parameters
* `matrix`: Transition matrix
=== markovChain Syntax
[source,text]
sample(markovChain(transitionMatrix), 5) // This creates a Markov Chain given a specific transition matrix. The sample function takes 5 samples from the Markov Chain, representing the next five states of the system.
=== markovChain Returns
Markov Chain model: The Markoff Chain model can be used with <<sample>> function.
== matrix
The matrix function returns a https://en.wikipedia.org/wiki/Matrix_(mathematics)[matrix] which can be operated on by functions that support matrix operations.
=== matrix Parameters
* `numeric array` ...: One or more numeric arrays that will be the rows of the matrix.
=== matrix Syntax
[source,text]
matrix(numericArray1, numericArray2, numericArray3) // Returns a matrix with three rows of data: numericaArray1, numericArray2, numericArray3
=== matrix Returns
matrix
== meanDifference
The `meanDifference` function calculates the mean of the differences following the element-by-element subtraction between two numeric arrays.
=== meanDifference Parameters
* `numeric array`
* `numeric array`
=== meanDifference Returns
A numeric.
=== meanDifference Syntax
[source,text]
----
meanDifference(numericArray, numericArray)
----
== minMaxScale
The `minMaxScale` function scales numeric arrays within a minimum and maximum value.
By default `minMaxScale` scales between 0 and 1. The `minMaxScale` function can operate on
both numeric arrays and matrices.
When operating on a matrix the `minMaxScale` function operates on each row of the matrix.
=== minMaxScale Parameters
* `numeric array` | `matrix`: The array or matrix to scale
* `double`: (Optional) The min value. Defaults to 0.
* `double`: (Optional) The max value. Defaults to 1.
=== minMaxScale Syntax
[source,text]
minMaxScale(numericArray) // scale a numeric array between 0 and 1
minMaxScale(numericArray, 0, 100) // scale a numeric array between 1 and 100
minMaxScale(matrix) // Scale each row in a matrix between 0 and 1
minMaxScale(matrix, 0, 100) // Scale each row in a matrix between 0 and 100
=== minMaxScale Returns
A numeric array or matrix
== mod
The `mod` function returns the remainder (modulo) of the first parameter divided by the second parameter.
=== mod Parameters
* `Field Name | Raw Number | Number Evaluator`: Parameter 1
* `Field Name | Raw Number | Number Evaluator`: Parameter 2
=== mod Syntax
The expressions below show the various ways in which you can use the `mod` evaluator.
[source,text]
----
mod(100,3) // returns the remainder of 100 / 3 .
mod(100,fieldA) // returns the remainder of 100 divided by the value of fieldA.
mod(fieldA,1.4) // returns the remainder of fieldA divided by 1.4.
if(gt(fieldA,fieldB),mod(fieldA,fieldB),mod(fieldB,fieldA)) // if fieldA > fieldB then return the remainder of fieldA/fieldB, else return the remainder of fieldB/fieldA.
----
== monteCarlo
The `monteCarlo` function performs a https://en.wikipedia.org/wiki/Monte_Carlo_method[Monte Carlo simulation]
based on its parameters. The `monteCarlo` function runs another function a specified number of times and returns the results.
The function being run typically has one or more variables that are drawn from probability
distributions on each run. The `<<sample>>` function is used in the function to draw the samples.
The simulation's result array can then be treated as an empirical distribution to understand
the probabilities of the simulation results.
=== monteCarlo Parameters
* `numeric function`: The function being run by the simulation, which must return a numeric value.
* `integer`: The number of times to run the function.
=== monteCarlo Returns
A numeric array: The results of simulation runs.
=== monteCarlo Syntax
[source,text]
let(a=uniformIntegerDistribution(1, 6),
b=uniformIntegerDistribution(1, 6),
c=monteCarlo(add(sample(a), sample(b)), 1000))
In the expression above the `monteCarlo` function is running the function `add(sample(a), sample(b))`
1000 times and returning the result. Each time the function is run samples are drawn from the
probability distributions stored in variables `a` and `b`.
== movingAvg
The `movingAvg` function calculates a https://en.wikipedia.org/wiki/Moving_average[moving average] over an array of numbers.
=== movingAvg Parameters
* `numeric array`
* `window size`
=== movingAvg Returns
A numeric array. The first element of the returned array will start from the windowSize-1 index of the original array.
=== movingAvg Syntax
[source,text]
movingAverage(numericArray, 30)
== movingMedian
The `movingMedian` function calculates a moving median over an array of numbers.
=== movingMedian Parameters
* `numeric array`
* `window size`
=== movingMedian Returns
A numeric array. The first element of the returned array will start from the windowSize-1 index of the original array.
=== movingMedian Syntax
[source,text]
movingMedian(numericArray, 30)
== mult
The `mult` function will take two or more numeric values and multiply them together. The `mult` function will fail to execute if any of the values are non-numeric. If a null value is found then null will be returned as the result.
=== mult Parameters
* `Field Name | Raw Number | Number Evaluator`
* `Field Name | Raw Number | Number Evaluator`
* `......`
* `Field Name | Raw Number | Number Evaluator`
=== mult Syntax
The expressions below show the various ways in which you can use the `mult` evaluator. The number and order of these parameters do not matter and is not limited except that at least two parameters are required. Returns a numeric value.
[source,text]
----
mult(1,2,3,4) // 1 * 2 * 3 * 4
mult(1,fieldA) // 1 * value of fieldA
mult(fieldA,1.4) // value of fieldA * 1.4
mult(fieldA,fieldB,fieldC) // value of fieldA * value of fieldB * value of fieldC
mult(fieldA,div(fieldA,fieldB)) // value of fieldA * (value of fieldA / value of fieldB)
mult(fieldA,if(gt(fieldA,fieldB),fieldA,fieldB)) // if fieldA > fieldB then fieldA * fieldA, else fieldA * fieldB
----
== normalDistribution
The `normalDistribution` function returns a https://en.wikipedia.org/wiki/Normal_distribution[normal probability distribution]
based on its parameters. This function is part of the probability distribution framework and is designed to
work with the `<<sample>>`, `<<kolmogorovSmirnov>>` and `<<cumulativeProbability>>` functions.
=== normalDistribution Parameters
* `double`: mean
* `double`: standard deviation
=== normalDistribution Returns
A probability distribution function.
=== normalDistribution Syntax
[source,text]
normalDistribution(mean, stddev)
== normalizeSum
The `normalizeSum` function scales numeric arrays so that they sum to 1.
The `normalizeSum` function can operate on both numeric arrays and matrices.
When operating on a matrix the `normalizeSum` function operates on each row of the matrix.
=== normalizeSum Parameters
* `numeric array` | `matrix`
=== normalizeSum Syntax
[source,text]
normalizeSum(numericArray)
normalizeSum(matrix)
=== normalizeSum Returns
numeric array | matrix
== not
The `not` function will return the logical NOT of a single boolean parameter. The function will fail to execute if the parameter is non-boolean or null. Returns a boolean value.
=== not Parameters
* `Field Name | Raw Boolean | Boolean Evaluator`
=== not Syntax
The expressions below show the various ways in which you can use the `not` evaluator. Only one parameter is allowed.
[source,text]
----
not(true) // false
not(fieldA) // true if fieldA is false else false
not(eq(fieldA,fieldB)) // true if fieldA != fieldB
----
== olsRegress
The `olsRegress` function performs https://en.wikipedia.org/wiki/Ordinary_least_squares[ordinary least squares], multivariate, linear regression.
The `olsRegress` function returns a single tuple containing the regression model with estimated regression parameters, RSquared and regression diagnostics.
The output of `olsRegress` can be used with the <<predict>> function to predict values based on the regression model.
=== olsRegress Parameters
* `matrix`: The regressor observation matrix. Each row in the matrix represents a single multi-variate regressor observation. Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an intercept term, this column will be added automatically.
* `numeric array`: The outcomes array which matches up with each row in the regressor observation matrix.
=== olsRegress Syntax
[source,text]
olsRegress(matrix, numericArray) // This performs the olsRegression analysis on given regressor matrix and outcome array.
=== olsRegress Returns
Tuple: The regression model including the estimated regression parameters and diagnostics.
== or
The `or` function will return the logical OR of at least 2 boolean parameters. The function will fail to execute if any parameters are non-boolean or null. Returns a boolean value.
=== or Parameters
* `Field Name | Raw Boolean | Boolean Evaluator`
* `Field Name | Raw Boolean | Boolean Evaluator`
* `......`
* `Field Name | Raw Boolean | Boolean Evaluator`
=== or Syntax
The expressions below show the various ways in which you can use the `or` evaluator. At least two parameters are required, but there is no limit to how many you can use.
[source,text]
----
or(true,fieldA) // true || fieldA
or(fieldA,fieldB) // fieldA || fieldB
or(and(fieldA,fieldB),fieldC) // (fieldA && fieldB) || fieldC
or(fieldA,fieldB,fieldC,and(fieldD,fieldE),fieldF)
----
== poissonDistribution
The `poissonDistribution` function returns a https://en.wikipedia.org/wiki/Poisson_distribution[poisson probability distribution]
based on its parameter. This function is part of the probability distribution framework and is designed to
work with the `<<sample>>`, `<<probability>>` and `<<cumulativeProbability>>` functions.
=== poissonDistribution Parameters
* `double`: mean
=== poissonDistribution Returns
A probability distribution function.
=== poissonDistribution Syntax
[source,text]
poissonDistribution(mean)
== polyFit
The `polyFit` function performs https://en.wikipedia.org/wiki/Curve_fitting#Fitting_lines_and_polynomial_functions_to_data_points[polynomial curve fitting].
=== polyFit Parameters
* `numeric array`: (Optional) x values. If omitted a sequence will be created for the x values.
* `numeric array`: y values
* `integer`: (Optional) polynomial degree. Defaults to 3.
=== polyFit Returns
A numeric array: curve that was fit to the data points.
=== polyFit Syntax
[source,text]
polyFit(yValues) // This creates the xValues automatically and fits a curve through the data points using the default 3 degree polynomial.
polyFit(yValues, 5) // This creates the xValues automatically and fits a curve through the data points using a 5 degree polynomial.
polyFit(xValues, yValues, 5) // This will fit a curve through the data points using a 5 degree polynomial.
== pow
The `pow` function returns the value of its first parameter raised to the power of its second parameter.
=== pow Parameters
* `Field Name` | `Raw Number` | `Number Evaluator`: Parameter 1
* `Field Name` | `Raw Number` | `Number Evaluator`: Parameter 2
=== pow Syntax
The expressions below show the various ways in which you can use the `pow` evaluator.
[source,text]
----
pow(2,3) // returns 2 raised to the 3rd power.
pow(4,fieldA) // returns 4 raised by the value of fieldA.
pow(fieldA,1.4) // returns the value of fieldA raised by 1.4.
if(gt(fieldA,fieldB),pow(fieldA,fieldB),pow(fieldB,fieldA)) // if fieldA > fieldB then raise fieldA by fieldB, else raise fieldB by fieldA.
----
== predict
The `predict` function predicts the value of dependent variables based on regression models or functions.
The `predict` function can predict values based on the output of the following functions: <<spline>>, <<loess>>, <<regress>>, <<olsRegress>>.
=== predict Parameters
* `regression model` | `function`: The model or function used for the prediction
* `number` | `numeric array` | `matrix`: Depending on the regression model or function used, the predictor variable can be a number, numeric array or matrix.
=== predict Syntax
[source,text]
----
predict(regressModel, number) // predict using the output of the <<regress>> function and single numeric predictor. This will return a single numeric prediction.
predict(regressModel, numericArray) // predict using the output of the <<regress>> function and a numeric array of predictors. This will return a numeric array of predictions.
predict(splineFunc, number) // predict using the output of the <<spline>> function and single numeric predictor. This will return a single numeric prediction.
predict(splineFunc, numericArray) // predict using the output of the <<spline>> function and a numeric array of predictors. This will return a numeric array of predictions.
predict(olsRegressModel, numericArray) // predict using the output of the <<olsRegress>> function and a numeric array containing one multi-variate predictor. This will return a single numeric prediction.
predict(olsRegressModel, matrix) // predict using the output of the <<olsRegress>> function and a matrix containing rows of multi-variate predictor arrays. This will return a numeric array of predictions.
----
== primes
The `primes` function returns an array of prime numbers starting from a specified number.
=== primes Parameters
* `integer`: The number of primes to return in the list
* `integer`: The starting point for returning the primes
=== primes Returns
A numeric array.
=== primes Syntax
[source,text]
----
primes(100, 2000) // returns 100 primes starting from 2000
----
== probability
The `probability` function returns the probability of a random variable within a probability distribution.
The `probability` function computes the probability between random variable ranges for both https://en.wikipedia.org/wiki/Probability_distribution#Continuous_probability_distribution[continuous] and
https://en.wikipedia.org/wiki/Probability_distribution#Discrete_probability_distribution[discrete] probability distributions.
The `probability` function can compute probabilities for a specific random variable for
discrete probability distributions only.
The supported continuous distribution functions are:
<<normalDistribution>>, <<logNormalDistribution>>, <<betaDistribution>>, <<gammaDistribution>>,
<<empiricalDistribution>>, <<triangularDistribution>>, <<weibullDistribution>>,
<<uniformDistribution>>, <<constantDistribution>>
The supported discreet distributions are:
<<poissonDistribution>>, <<binomialDistribution>>, <<enumeratedDistribution>>, <<zipFDistribution>>,
<<geometricDistribution>>, <<uniformIntegerDistribution>>
=== probability Parameters
* `probability distribution`: the probability distribution to compute the probability from.
* `number`: low value of the range.
* `number`: (Optional for discrete probability distributions) high value of the range. If the high range is omitted then the probability function will compute a probability for the low range value.
=== probability Syntax
[source,text]
----
probability(poissonDistribution(10), 7) // Returns the probability of a random sample of 7 in a poisson distribution with a mean of 10.
probability(normalDistribution(10, 2), 7.5, 8.5) // Returns the probability between the range of 7.5 to 8.5 for a normal distribution with a mean of 10 and standard deviation of 2.
----
=== probability Returns
double: probability
== rank
The `rank` performs a rank transformation on a numeric array.
=== rank Parameters
//TODO fill in details of Parameters
* `numeric array`
=== rank Syntax
[source,text]
rank(numericArray)
== raw
The `raw` function will return whatever raw value is the parameter. This is useful for cases where you want to use a string as part of another evaluator.
=== raw Parameters
* `Raw Value`
=== raw Syntax
The expressions below show the various ways in which you can use the `raw` evaluator. Whatever is inside will be returned as-is. Internal evaluators are considered strings and are not evaluated.
[source,text]
----
raw(foo) // "foo"
raw(count(*)) // "count(*)"
raw(45) // 45
raw(true) // "true" (note: this returns the string "true" and not the boolean true)
eq(raw(fieldA), fieldA) // true if the value of fieldA equals the string "fieldA"
----
== regress
The `regress` function performs a simple regression of two numeric arrays.
The result of this expression is also used by the `<<predict>>` function.
=== regress Parameters
//TODO fill in details of Parameters
* `numeric array`
* `numeric array`
=== regress Syntax
[source,text]
regress(numericArray1, numericArray2)
== rev
The `rev` function reverses the order of a numeric array.
=== rev Parameters
* `numeric array`
=== rev Syntax
[source,text]
rev(numericArray)
== round
The `round` function returns the closest whole number to the argument.
=== round Parameters
* `Field Name` | `Raw Number` | `Number Evaluator`: The value to return the square root of.
=== round Syntax
[source,text]
----
round(100.4)
round(fieldA)
if(gt(fieldA,fieldB),sqrt(fieldA),sqrt(fieldB)) // if fieldA > fieldB then return the round of fieldA, else return the round of fieldB
----
== rowAt
The `rowAt` function returns the row of a matrix at a specific index as a numeric array.
=== rowAt Parameters
* `matrix`: the matrix to operate on
* `integer`: the index of the row to return
=== rowAt Syntax
[source,text]
rowAt(matrix, 10)
=== rowAt Returns
numeric array: the row of the matrix
== rowCount
The `rowCount` function returns the number of rows in a `matrix`.
=== rowCount Parameters
* `matrix`: the matrix to operate on
=== rowCount Syntax
[source,text]
rowCount(matrix)
=== rowCount Returns
integer: number rows in the matrix.
== sample
The `sample` function can be used to draw random samples from a probability distribution or Markov Chain.
=== sample Parameters
* `probability distribution` | `Markov Chain`: The distribution or Markov Chain to sample.
* `integer`: (Optional) Sample size. Defaults to 1.
=== sample Returns
Either a single numeric random sample, or a numeric array depending on the sample size parameter.
=== sample Syntax
[source,text]
sample(poissonDistribution(5)) // Returns a single random sample from a poissonDistribution with mean of 5.
sample(poissonDistribution(5), 1000) // Returns 1000 random samples from poissonDistribution with a mean of 5.
sample(markovChain(transitionMatrix), 1000) // Returns 1000 random samples from a Markov Chain.
== scalarAdd
The `scalarAdd` function adds a scalar value to every value in a numeric array or matrix.
When working with numeric arrays, `scalarAdd` returns a new array with the new values. When working
with a matrix, `scalarAdd` returns a new matrix with new values.
=== scalarAdd Parameters
`number`: value to add
`numeric array` | `matrix`: the numeric array or matrix to add the value to.
=== scalarAdd Syntax
[source,text]
scalarAdd(number, numericArray) // Adds the number to each element in the number in the array.
scalarAdd(number, matrix) // Adds the number to each value in a matrix
=== scalarAdd Returns
numericArray | matrix: Depending on what is being operated on.
== scalarDivide
The `scalarDivide` function divides each number in numeric array or matrix by a scalar value.
When working with numeric arrays, `scalarDivide` returns a new array with the new values. When working
with a matrix, `scalarDivide` returns a new matrix with new values.
=== scalarDivide Parameters
`number`: value to divide by
`numeric array` | `matrix`: the numeric array or matrix to divide by the value to.
=== scalarDivide Syntax
[source,text]
scalarDivide(number, numericArray) // Divides each element in the numeric array by the number.
scalarDivide(number, matrix) // Divides each element in the matrix by the number.
=== scalarDivide Returns
numericArray | matrix: depending on what is being operated on.
== scalarMultiply
The `scalarMultiply` function multiplies each element in a numeric array or matrix by a
scalar value. When working with numeric arrays, `scalarMultiply` returns a new array with the new values. When working
with a matrix, `scalarMultiply` returns a new matrix with new values.
=== scalarMultiply Parameters
`number`: value to divide by
`numeric array` | `matrix`: the numeric array or matrix to divide by the value to.
=== scalarMultiply Syntax
[source,text]
scalarMultiply(number, numericArray) // Multiplies each element in the numeric array by the number.
scalarMultiply(number, matrix) // Multiplies each element in the matrix by the number.
=== scalarMultiply Returns
numericArray | matrix: depending on what is being operated on
== scalarSubtract
The `scalarSubtract` function subtracts a scalar value from every value in a numeric array or matrix.
When working with numeric arrays, `scalarSubtract` returns a new array with the new values. When working
with a matrix, `scalarSubtract` returns a new matrix with new values.
=== scalarSubtract Parameters
`number`: value to add
`numeric array` | `matrix`: the numeric array or matrix to subtract the value from.
=== scalarSubtract Syntax
[source,text]
scalarSubtract(number, numericArray) // Subtracts the number from each element in the number in the array.
scalarSubtract(number, matrix) // Subtracts the number from each value in a matrix
=== scalarSubtract Returns
numericArray | matrix: depending on what is being operated on.
== scale
The `scale` function multiplies all the elements of an array by a number.
=== scale Parameters
//TODO fill in details of Parameters
* `number`
* `numeric array`
=== scale Syntax
[source,text]
scale(number, numericArray)
== sequence
The `sequence` function returns an array of numbers based on its parameters.
=== sequence Parameters
//TODO fill in details of Parameters
* `length`
* `start`
* `stride`
=== sequence Syntax
[source,text]
sequence(100, 0, 1) // Returns a sequence of length 100, starting from 0 with a stride of 1.
== setAttributes
The `setAttributes` function sets an attributes map of a `matrix`.
=== setAttributes Parameters
* `matrix`: The matrix to set the attributes map to.
* `map`: The map of attributes to set on the matrix.
=== setAttributes Syntax
[source,text]
setAttributes(matrix, map)
=== setAttributes Returns
matrix: The matrix with the attributes set.
== setColumnLabels
The `setColumnLabels` function sets the columns labels of a matrix.
=== setColumnLabels Parameters
* `matrix`: The matrix to set the column labels to.
* `string array`: The column labels to set the matrix
=== setColumnLabels Syntax
setColumnLabels(matrix, labels)
=== setColumnLabels Returns
matrix: The matrix with the labels set.
== setRowLabels
The `setRowLabels` function sets the row labels of a matrix.
=== setRowLabels Parameters
* `matrix`: The matrix to set the row labels to.
* `string array`: The row labels to set to the matrix
=== setRowLabels Syntax
setRowLabels(matrix, labels)
=== setRowLabels Returns
matrix: The matrix with the labels set.
== setValue
The `setValue` function sets a new value for a tuple entry.
=== setValue Parameters
* `tuple`: The tuple to return the entry from.
* `key`: The key of the entry to set.
* `value`: The value to set.
=== setValue Syntax
setValue(tuple, key, value)
=== setValue Returns
tuple: Returns the new modified tuple
== sin
The `sin` function returns the trigonometric sine of a number.
=== sin Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the sine of.
=== sin Syntax
[source,text]
----
sin(100.4) // returns the sine of 100.4
sine(fieldA) // returns the sine for fieldA.
if(gt(fieldA,fieldB),sin(fieldA),sin(fieldB)) // if fieldA > fieldB then return the sine of fieldA, else return the sine of fieldB
----
== spline
The `spline` function performs a cubic spline interpolation (https://en.wikiversity.org/wiki/Cubic_Spline_Interpolation) of a curve
given a set of x,y coordinates. The return value of the spline function is an
interpolation function which can be used to <<predict>> values along the curve and generate a <<derivative>> of
the curve.
=== spline Parameters
* `numeric array`: (Optional) x values. If omitted a sequence will be created for the x values.
* `numeric array`: y values
=== spline Syntax
[source,text]
spline(yValues) // This creates the xValues automatically and fits a spline through the data points.
spline(xValues, yValues) // This will fit a spline through the data points.
=== spline Returns
function: the function can be treated as both a `numeric array` and `function`.
== sqrt
The `sqrt` function returns the trigonometric square root of a number.
=== sqrt Parameters
* `Field Name | Raw Number | Number Evaluator`: The value to return the square root of.
=== sqrt Syntax
[source,text]
----
sqrt(100.4) // returns the square root of 100.4
sqrt(fieldA) // returns the square root for fieldA.
if(gt(fieldA,fieldB),sqrt(fieldA),sqrt(fieldB)) // if fieldA > fieldB then return the sqrt of fieldA, else return the sqrt of fieldB
----
== standardize
The `standardize` function standardizes a numeric array so that values within the array
have a mean of 0 and standard deviation of 1.
=== standardize Parameters
* `numeric array`: the array to standardize
=== standardize Syntax
[source,text]
standardize(numericArray)
=== standardize Returns
numeric array: the standardized values
== sub
The `sub` function will take 2 or more numeric values and subtract them, from left to right. The `sub` function will fail to execute if any of the values are non-numeric. If a null value is found then `null` will be returned as the result.
=== sub Parameters
* `Field Name | Raw Number | Number Evaluator`
* `Field Name | Raw Number | Number Evaluator`
* `......`
* `Field Name | Raw Number | Number Evaluator`
=== sub Syntax
The expressions below show the various ways in which you can use the `sub` evaluator. The number of these parameters does not matter and is not limited except that at least two parameters are required. Returns a numeric value.
[source,text]
----
sub(1,2,3,4) // 1 - 2 - 3 - 4
sub(1,fieldA) // 1 - value of fieldA
sub(fieldA,1.4) // value of fieldA - 1.4
sub(fieldA,fieldB,fieldC) // value of fieldA - value of fieldB - value of fieldC
sub(fieldA,div(fieldA,fieldB)) // value of fieldA - (value of fieldA / value of fieldB)
if(gt(fieldA,fieldB),sub(fieldA,fieldB),sub(fieldB,fieldA)) // if fieldA > fieldB then fieldA - fieldB, else fieldB - field
----
== sumDifference
The `sumDifference` function calculates the sum of the differences following an element-by-element subtraction between two numeric arrays.
=== sumDifference Parameters
* `numeric array`
* `numeric array`
=== sumDifference Returns
A numeric.
=== sumDifference Syntax
[source,text]
----
sumDifference(numericArray, numericArray)
----
== sumColumns
The `sumColumns` function sums the columns in a matrix and returns a numeric array with the result.
=== sumColumns Parameters
* `matrix`: the matrix to operate on
=== sumColumns Syntax
[source,text]
sumColumns(matrix)
=== sumColumns Returns
numeric array: the sum of the columns
== sumRows
The `sumRows` function sums the rows in a matrix and returns a numeric array with the result.
=== sumRows Parameters
* `matrix`: the matrix to operate on
=== sumRows Syntax
[source,text]
sumRows(matrix)
=== sumRows Returns
numeric array: sum of the rows.
== sumSq
The `sumSq` function returns the sum-of-squares of the values in a numeric array.
=== sumSq Parameters
* `numeric array`: The numeric array to compute the sumSq of.
=== sumSq Syntax
[source,text]
sumSq(numericArray)
=== sumSq Returns
numeric: result of the sumSq calculation
== transpose
The `transpose` function https://en.wikipedia.org/wiki/Transpose[transposes] a matrix.
=== transpose Parameters
* `matrix`: the matrix to transpose
=== transpose Syntax
[source,text]
transpose(matrix)
=== transpose Returns
matrix: the transposed matrix
== triangularDistribution
The `triangularDistribution` function returns a https://en.wikipedia.org/wiki/Triangular_distribution[triangular probability distribution]
based on its parameters. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>`, `<<probability>>` and `<<cumulativeProbability>>` functions.
=== triangularDistribution Parameters
* `double`: low value
* `double`: most likely value
* `double`: high value
=== triangularDistribution Syntax
[source,text]
triangularDistribution(10, 15, 20) // A triangular distribution with a low value of 10, most likely value of 15 and high value of 20.
=== triangularDistribution Returns
Probability distribution function
== uniformDistribution
The `uniformDistribution` function returns a https://en.wikipedia.org/wiki/Uniform_distribution_(continuous)[continuous uniform probability distribution]
based on its parameters. See the `<<uniformIntegerDistribution>>` to work with discrete uniform distributions. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>` and `<<cumulativeProbability>>` functions.
=== uniforDistribution Parameters
* `double`: start
* `double`: end
=== uniformDistribution Returns
Probability distribution function.
=== uniformDistribution Syntax
[source,text]
uniformDistribution(0.0, 100.0)
== uniformIntegerDistribution
The `uniformIntegerDistribution` function returns a https://en.wikipedia.org/wiki/Discrete_uniform_distribution[discrete uniform probability distribution]
based on its parameters. See the `<<uniformDistribution>>` to work with continuous uniform distributions. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>`, `<<probability>>` and `<<cumulativeProbability>>` functions.
=== uniformIntegerDistribution Parameters
* `integer`: start
* `integer`: end
=== uniformIntegerDistribution Returns
A probability distribution function.
=== uniformIntegerDistribution Syntax
[source,text]
uniformDistribution(1, 6)
== unitize
The `unitize` function scales numeric arrays to a magnitude of 1, often called https://en.wikipedia.org/wiki/Unit_vector[unit vectors].
The unitize function can operate on both numeric arrays and matrices.
When operating on a matrix the unitize function unitizes each row of the matrix.
=== unitize Parameters
* `numeric array` | `matrix`: The array or matrix to unitize
=== unitize Syntax
[source,text]
unitize(numericArray) // Unitize a numeric array
unitize(matrix) // Unitize each row in a matrix
=== unitize Returns
numeric array | matrix
== weibullDistribution
The `weibullDistribution` function returns a https://en.wikipedia.org/wiki/Weibull_distribution[Weibull probability distribution]
based on its parameters. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>`, `<<kolmogorovSmirnov>>` and `<<cumulativeProbability>>` functions.
=== weibullDistribution Parameters
* `double`: shape
* `double`: scale
=== weibullDistribution Returns
A probability distribution function.
=== weibullDistribution Syntax
[source,text]
weibullDistribution(.5, 10)
== zipFDistribution
The `zipFDistribution` function returns a https://en.wikipedia.org/wiki/Zeta_distribution[ZipF distribution]
based on its parameters. This function is part of the
probability distribution framework and is designed to work with the `<<sample>>`,
`<<probability>>` and `<<cumulativeProbability>>` functions.
=== zipFDistribution Parameters
* `integer`: size
* `double`: exponent
=== zipFDistribution Returns
A probability distribution function.
=== zipFDistribution Syntax
[source,text]
zipFDistribution(5000, 1.0)