| # Licensed to the Apache Software Foundation (ASF) under one or more |
| # contributor license agreements. See the NOTICE file distributed with |
| # this work for additional information regarding copyright ownership. |
| # The ASF licenses this file to You under the Apache License, Version 2.0 |
| # (the "License"); you may not use this file except in compliance with |
| # the License. You may obtain a copy of the License at |
| # |
| # http://www.apache.org/licenses/LICENSE-2.0 |
| # |
| # Unless required by applicable law or agreed to in writing, software |
| # distributed under the License is distributed on an "AS IS" BASIS, |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| # See the License for the specific language governing permissions and |
| # limitations under the License. |
| # |
| #------------------------------------------------------------------------------ |
| |
| INTRODUCTION |
| |
| The purpose of the R programs included in this directory is to validate |
| the target values used in Apache commons math unit tests. Success running the |
| R and commons-math tests on a platform (OS and R version) means that R and |
| commons-math give results for the test cases that are close in value. The |
| tests include configurable tolerance levels; but care must be taken in changing |
| these, since in most cases the pre-set tolerance is close to the number of |
| decimal digits used in expressing the expected values (both here and in the |
| corresponding commons-math unit tests). |
| |
| Of course it is always possible that both R and commons-math give incorrect |
| values for test cases, so these tests should not be interpreted as definitive |
| in any absolute sense. The value of developing and running the tests is really |
| to generate questions (and answers!) when the two systems give different |
| results. |
| |
| Contributions of additional test cases (both R and Junit code) or just |
| R programs to validate commons-math tests that are not covered here would be |
| greatly appreciated. |
| |
| SETUP |
| |
| 0) Download and install R. You can get R here |
| http://www.r-project.org/ |
| Follow the install instructions and make sure that you can launch R from this |
| (i.e., either explitly add R to your OS path or let the install package do it |
| for you). |
| |
| 1) Launch R from this directory and type |
| > source("testAll") |
| to an R prompt. This should produce output to the console similar to this: |
| |
| Binomial test cases |
| Density test n = 10, p = 0.7...........................................SUCCEEDED |
| Distribution test n = 10, p = 0.7......................................SUCCEEDED |
| Inverse Distribution test n = 10, p = 0.7..............................SUCCEEDED |
| Density test n = 5, p = 0..............................................SUCCEEDED |
| Distribution test n = 5, p = 0.........................................SUCCEEDED |
| Density test n = 5, p = 1..............................................SUCCEEDED |
| Distribution test n = 5, p = 1.........................................SUCCEEDED |
| -------------------------------------------------------------------------------- |
| Normal test cases |
| Distribution test mu = 2.1, sigma = 1.4................................SUCCEEDED |
| Distribution test mu = 2.1, sigma = 1.4................................SUCCEEDED |
| Distribution test mu = 0, sigma = 1....................................SUCCEEDED |
| Distribution test mu = 0, sigma = 0.1..................................SUCCEEDED |
| -------------------------------------------------------------------------------- |
| ... |
| <more test reports> |
| |
| |
| WORKING WITH THE TESTS |
| |
| The R distribution comes with online manuals that you can view by launching |
| a browser instance and then entering |
| |
| > help.start() |
| |
| at an R prompt. Poking about in the test case files and the online docs should |
| bring you up to speed fairly quickly. Here are some basic things to get |
| you started. I should note at this point that I am by no means an expert R |
| programmer, so some things may not be implemented in the the nicest way. |
| Comments / suggestions for improvement are welcome! |
| |
| All of the test cases use some basic functions and global constants (screen |
| width and success / failure strings) defined in "testFunctions." The |
| R "source" function is used to "import" these functions into each of the test |
| programs. The "testAll" program pulls together and executes all of the |
| individual test programs. You can execute any one of them by just entering |
| |
| > source(<program-name>). |
| |
| The "assertEquals" function in the testFunctions file mimics the similarly |
| named function used by Junit: |
| |
| assertEquals <- function(expected, observed, tol, message) { |
| if(any(abs(expected - observed) > tol)) { |
| cat("FAILURE: ",message,"\n") |
| cat("EXPECTED: ",expected,"\n") |
| cat("OBSERVED: ",observed,"\n") |
| return(0) |
| } else { |
| return(1) |
| } |
| } |
| |
| The <expected> and <observed> arguments can be scalar values, vectors or |
| matrices. If the arguments are vectors or matrices, corresponding entries |
| are compared. |
| |
| The standard pattern used throughout the tests looks like this (from |
| binomialTestCases): |
| |
| Start by defining a "verification function" -- in this example a function to |
| verify computation of binomial probabilities. The <points> argument is a vector |
| of integer values to feed into the density function, <expected> is a vector of |
| the computed probabilies from the commons-math Junit tests, <n> and <p> are |
| parameters of the distribution and <tol> is the error tolerance of the test. |
| The function computes the probabilities using R and compares the values that |
| R produces with those in the <expected> vector. |
| |
| verifyDensity <- function(points, expected, n, p, tol) { |
| rDensityValues <- rep(0, length(points)) |
| i <- 0 |
| for (point in points) { |
| i <- i + 1 |
| rDensityValues[i] <- dbinom(point, n, p, log = FALSE) |
| } |
| output <- c("Density test n = ", n, ", p = ", p) |
| if (assertEquals(expected,rDensityValues,tol,"Density Values")) { |
| displayPadded(output, SUCCEEDED, WIDTH) |
| } else { |
| displayPadded(output, FAILED, WIDTH) |
| } |
| } |
| |
| The displayPadded function just displays its first and second arguments with |
| enough dots in between to make the whole string WIDTH characters long. It is |
| defined in testFunctions. |
| |
| Then call this function with different parameters corresponding to the different |
| Junit test cases: |
| |
| size <- 10.0 |
| probability <- 0.70 |
| |
| densityPoints <- c(-1,0,1,2,3,4,5,6,7,8,9,10,11) |
| densityValues <- c(0, 0.0000, 0.0001, 0.0014, 0.0090, 0.0368, 0.1029, |
| 0.2001, 0.2668, 0.2335, 0.1211, 0.0282, 0) |
| ... |
| verifyDensity(densityPoints, densityValues, size, probability, tol) |
| |
| If the values computed by R match the target values in densityValues, this will |
| produce one line of output to the console: |
| |
| Density test n = 10, p = 0.7...........................................SUCCEEDED |
| |
| If you modify the value of tol set at the top of binomialTestCases to make the |
| test more sensitive than the number of digits specified in the densityValues |
| vector, it will fail, producing the following output, showing the failure and |
| the expected and observed values: |
| |
| FAILURE: Density Values |
| EXPECTED: 0 0 1e-04 0.0014 0.009 0.0368 0.1029 0.2001 0.2668 0.2335 0.1211 / |
| 0.0282 0 |
| OBSERVED: 0 5.9049e-06 0.000137781 0.0014467005 0.009001692 0.036756909 / |
| 0.1029193452 0.200120949 0.266827932 0.2334744405 0.121060821 0.0282475249 0 |
| Density test n = 10, p = 0.7..............................................FAILED |
| |
| |