Add 'stresso/' from commit 'f246b630c4cd2eb8426af2f0c79a4c8509a1fb91'
git-subtree-dir: stresso
git-subtree-mainline: 6b2b93da132372906763a1ea28f047a947c1e467
git-subtree-split: f246b630c4cd2eb8426af2f0c79a4c8509a1fb91
diff --git a/LICENSE b/LICENSE
index 37ec93a..d645695 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,180 +1,191 @@
-Apache License
-Version 2.0, January 2004
-http://www.apache.org/licenses/
-TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
-1. Definitions.
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-"License" shall mean the terms and conditions for use, reproduction, and
-distribution as defined by Sections 1 through 9 of this document.
+ 1. Definitions.
-"Licensor" shall mean the copyright owner or entity authorized by the copyright
-owner that is granting the License.
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
-"Legal Entity" shall mean the union of the acting entity and all other entities
-that control, are controlled by, or are under common control with that entity.
-For the purposes of this definition, "control" means (i) the power, direct or
-indirect, to cause the direction or management of such entity, whether by
-contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
-outstanding shares, or (iii) beneficial ownership of such entity.
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
-"You" (or "Your") shall mean an individual or Legal Entity exercising
-permissions granted by this License.
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
-"Source" form shall mean the preferred form for making modifications, including
-but not limited to software source code, documentation source, and configuration
-files.
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
-"Object" form shall mean any form resulting from mechanical transformation or
-translation of a Source form, including but not limited to compiled object code,
-generated documentation, and conversions to other media types.
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
-"Work" shall mean the work of authorship, whether in Source or Object form, made
-available under the License, as indicated by a copyright notice that is included
-in or attached to the work (an example is provided in the Appendix below).
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
-"Derivative Works" shall mean any work, whether in Source or Object form, that
-is based on (or derived from) the Work and for which the editorial revisions,
-annotations, elaborations, or other modifications represent, as a whole, an
-original work of authorship. For the purposes of this License, Derivative Works
-shall not include works that remain separable from, or merely link (or bind by
-name) to the interfaces of, the Work and Derivative Works thereof.
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
-"Contribution" shall mean any work of authorship, including the original version
-of the Work and any modifications or additions to that Work or Derivative Works
-thereof, that is intentionally submitted to Licensor for inclusion in the Work
-by the copyright owner or by an individual or Legal Entity authorized to submit
-on behalf of the copyright owner. For the purposes of this definition,
-"submitted" means any form of electronic, verbal, or written communication sent
-to the Licensor or its representatives, including but not limited to
-communication on electronic mailing lists, source code control systems, and
-issue tracking systems that are managed by, or on behalf of, the Licensor for
-the purpose of discussing and improving the Work, but excluding communication
-that is conspicuously marked or otherwise designated in writing by the copyright
-owner as "Not a Contribution."
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
-"Contributor" shall mean Licensor and any individual or Legal Entity on behalf
-of whom a Contribution has been received by Licensor and subsequently
-incorporated within the Work.
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
-2. Grant of Copyright License.
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
-Subject to the terms and conditions of this License, each Contributor hereby
-grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
-irrevocable copyright license to reproduce, prepare Derivative Works of,
-publicly display, publicly perform, sublicense, and distribute the Work and such
-Derivative Works in Source or Object form.
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
-3. Grant of Patent License.
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
-Subject to the terms and conditions of this License, each Contributor hereby
-grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
-irrevocable (except as stated in this section) patent license to make, have
-made, use, offer to sell, sell, import, and otherwise transfer the Work, where
-such license applies only to those patent claims licensable by such Contributor
-that are necessarily infringed by their Contribution(s) alone or by combination
-of their Contribution(s) with the Work to which such Contribution(s) was
-submitted. If You institute patent litigation against any entity (including a
-cross-claim or counterclaim in a lawsuit) alleging that the Work or a
-Contribution incorporated within the Work constitutes direct or contributory
-patent infringement, then any patent licenses granted to You under this License
-for that Work shall terminate as of the date such litigation is filed.
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
-4. Redistribution.
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
-You may reproduce and distribute copies of the Work or Derivative Works thereof
-in any medium, with or without modifications, and in Source or Object form,
-provided that You meet the following conditions:
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
-You must give any other recipients of the Work or Derivative Works a copy of
-this License; and
-You must cause any modified files to carry prominent notices stating that You
-changed the files; and
-You must retain, in the Source form of any Derivative Works that You distribute,
-all copyright, patent, trademark, and attribution notices from the Source form
-of the Work, excluding those notices that do not pertain to any part of the
-Derivative Works; and
-If the Work includes a "NOTICE" text file as part of its distribution, then any
-Derivative Works that You distribute must include a readable copy of the
-attribution notices contained within such NOTICE file, excluding those notices
-that do not pertain to any part of the Derivative Works, in at least one of the
-following places: within a NOTICE text file distributed as part of the
-Derivative Works; within the Source form or documentation, if provided along
-with the Derivative Works; or, within a display generated by the Derivative
-Works, if and wherever such third-party notices normally appear. The contents of
-the NOTICE file are for informational purposes only and do not modify the
-License. You may add Your own attribution notices within Derivative Works that
-You distribute, alongside or as an addendum to the NOTICE text from the Work,
-provided that such additional attribution notices cannot be construed as
-modifying the License.
-You may add Your own copyright statement to Your modifications and may provide
-additional or different license terms and conditions for use, reproduction, or
-distribution of Your modifications, or for any such Derivative Works as a whole,
-provided Your use, reproduction, and distribution of the Work otherwise complies
-with the conditions stated in this License.
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
-5. Submission of Contributions.
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
-Unless You explicitly state otherwise, any Contribution intentionally submitted
-for inclusion in the Work by You to the Licensor shall be under the terms and
-conditions of this License, without any additional terms or conditions.
-Notwithstanding the above, nothing herein shall supersede or modify the terms of
-any separate license agreement you may have executed with Licensor regarding
-such Contributions.
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
-6. Trademarks.
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
-This License does not grant permission to use the trade names, trademarks,
-service marks, or product names of the Licensor, except as required for
-reasonable and customary use in describing the origin of the Work and
-reproducing the content of the NOTICE file.
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
-7. Disclaimer of Warranty.
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
-Unless required by applicable law or agreed to in writing, Licensor provides the
-Work (and each Contributor provides its Contributions) on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
-including, without limitation, any warranties or conditions of TITLE,
-NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
-solely responsible for determining the appropriateness of using or
-redistributing the Work and assume any risks associated with Your exercise of
-permissions under this License.
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
-8. Limitation of Liability.
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
-In no event and under no legal theory, whether in tort (including negligence),
-contract, or otherwise, unless required by applicable law (such as deliberate
-and grossly negligent acts) or agreed to in writing, shall any Contributor be
-liable to You for damages, including any direct, indirect, special, incidental,
-or consequential damages of any character arising as a result of this License or
-out of the use or inability to use the Work (including but not limited to
-damages for loss of goodwill, work stoppage, computer failure or malfunction, or
-any and all other commercial damages or losses), even if such Contributor has
-been advised of the possibility of such damages.
+ END OF TERMS AND CONDITIONS
-9. Accepting Warranty or Additional Liability.
+ APPENDIX: How to apply the Apache License to your work.
-While redistributing the Work or Derivative Works thereof, You may choose to
-offer, and charge a fee for, acceptance of support, warranty, indemnity, or
-other liability obligations and/or rights consistent with this License. However,
-in accepting such obligations, You may act only on Your own behalf and on Your
-sole responsibility, not on behalf of any other Contributor, and only if You
-agree to indemnify, defend, and hold each Contributor harmless for any liability
-incurred by, or claims asserted against, such Contributor by reason of your
-accepting any such warranty or additional liability.
-
-END OF TERMS AND CONDITIONS
-
-APPENDIX: How to apply the Apache License to your work
-
-To apply the Apache License to your work, attach the following boilerplate
-notice, with the fields enclosed by brackets "[]" replaced with your own
-identifying information. (Don't include the brackets!) The text should be
-enclosed in the appropriate comment syntax for the file format. We also
-recommend that a file or class name and description of purpose be included on
-the same "printed page" as the copyright notice for easier identification within
-third-party archives.
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
@@ -182,7 +193,7 @@
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
+ http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
diff --git a/README.md b/README.md
index d3c2577..6a9cfba 100644
--- a/README.md
+++ b/README.md
@@ -1,192 +1 @@
-
-# Stresso
-
-[](https://travis-ci.org/astralway/stresso)
-
-An example application designed to stress Apache Fluo. This Fluo application computes the
-number of unique integers through the process of building a bitwise trie. New numbers
-are added to the trie as leaf nodes. Observers watch all nodes in the trie to create
-parents and percolate counts up to the root nodes such that each node in the trie keeps
-track of the number of leaf nodes below it. The count at the root nodes should equal
-the total number of leaf nodes. This makes it easy to verify if the test ran correctly.
-The test stresses Apache Fluo in that multiple transactions can operate on the same data
-as counts are percolated up the trie.
-
-## Concepts and definitions
-
-This test has the following set of configurable parameters.
-
- * **nodeSize** : The number of bits chopped off the end each time a number is
- percolated up. Must choose a nodeSize such that `64 % nodeSize == 0`
- * **stopLevel** : The number of levels in the tree is a function of the
- nodeSize. The deepest possible level is `64 / nodeSize`. Levels are
- decremented going up the tree. Setting the stop level determines how far up
- to percolate. The lower the stop level, the more root nodes there are.
- Having more root nodes means less collisions, but all roots need to be
- scanned to get the count of unique numbers. Having ~64k root nodes is a
- good choice.
- * **max** : Random numbers are generated modulo the max.
-
-Setting the stop level such that you have ~64k root nodes is dependent on the
-max and nodeSize. For example assume we choose a max of 10<sup>12</sup> and a
-node size of 8. The following table shows information about each level in the
-tree using this configuration. So for a max of 10<sup>12</sup> choosing a stop
-level of 5 would result in 59,604 root nodes. With this many root nodes there
-would not be many collisions and scanning 59,604 nodes to compute the unique
-number of intergers is a quick operation.
-
-|Level|Max Node |Number of possible Nodes|
-|:---:|---------------------|-----------------------:|
-| 0 |`0xXXXXXXXXXXXXXXXX` | 1 |
-| 1 |`0x00XXXXXXXXXXXXXX` | 1 |
-| 2 |`0x0000XXXXXXXXXXXX` | 1 |
-| 3 |`0x000000XXXXXXXXXX` | 1 |
-| 4 |`0x000000E8XXXXXXXX` | 232 |
-| 5 |`0x000000E8D4XXXXXX` | 59,604 |
-| 6 |`0x000000E8D4A5XXXX` | 15,258,789 |
-| 7 |`0x000000E8D4A510XX` | 3,906,250,000 |
-| 8 |`0x000000E8D4A51000` | 1,000,000,000,000 |
-
-In the table above, X indicates nibbles that are always zeroed out for every
-node at that level. You can easily view nodes at a level using a row prefix
-with the fluo scan command. For example `fluo scan -p 05` shows all nodes at
-level 5.
-
-For small scale test a max of 10<sup>9</sup> and a stop level of 6 is a good
-choice.
-
-## Building Stresso
-
-```
-mvn package
-```
-
-This will create a jar and shaded jar in target:
-
-```
-$ ls target/stresso-*
-target/stresso-0.0.1-SNAPSHOT.jar target/stresso-0.0.1-SNAPSHOT-shaded.jar
-```
-
-## Run Stresso using MiniFluo
-
-There are several integration tests that run Stresso on a MiniFluo instance.
-These tests can be run using `mvn verify`.
-
-## Run Stresso on cluster
-
-The [bin directory](/bin) contains a set of scripts to help run this test on a
-cluster. These scripts make the following assumpitions.
-
- * `FLUO_HOME` environment variable is set. If not set, then set it in `conf/env.sh`.
- * Hadoop `yarn` command is on path.
- * Hadoop `hadoop` command is on path.
- * Accumulo `accumulo` command is on path.
-
-Before running any of the scipts, copy [conf/env.sh.example](/conf/env.sh.example)
-to `conf/env.sh`, then inspect and modify the file.
-
-Next, execute the [run-test.sh](/bin/run-test.sh) script. This script will create a
-new Apache Fluo app called `stresso` (which can be changed by `FLUO_APP_NAME` in your env.sh).
-It will modify the application's fluo.properties, copy the stresso jar to the `lib/`
-directory of the app and set the following in fluo.properties:
-
-```
-fluo.observer.0=stresso.trie.NodeObserver
-fluo.app.trie.nodeSize=X
-fluo.app.trie.stopLevel=Y
-```
-
-The `run-test.sh` script will then initialize and start the Stresso application.
-It will load a lot of data directly into Accumulo without transactions and then
-incrementally load smaller amounts of data using transactions. After incrementally
-loading some data, it computes the expected number of unique integers using map reduce.
-It then prints the number of unique integers computed by Apache Fluo.
-
-## Additional Scripts
-
-The script [generate.sh](/bin/generate.sh) starts a map reduce job to generate
-random integers.
-
-```
-generate.sh <num files> <num per file> <max> <out dir>
-
-where:
-
-num files = Number of files to generate (and number of map task)
-numPerMap = Number of random numbers to generate per file
-max = Generate random numbers between 0 and max
-out dir = Output directory
-```
-
-The script [split.sh](/bin/split.sh) pre-splits the Accumulo table used by Apache
-Fluo. Consider running this command before loading data.
-
-```
-split.sh <num tablets> <max>
-
-where:
-
-num tablets = Num tablets to create for lowest level of tree. Will create less tablets for higher levels based on the max.
-```
-After generating random numbers, load them into Apache Fluo with one of the following
-commands. The script [init.sh](/bin/init.sh) intializes any empty table using
-map reduce. This simulates the case where a user has a lot of initial data to
-load into Fluo. This command should only be run when the table is empty
-because it writes directly to the Fluo table w/o using transactions.
-
-```
-init.sh <input dir> <tmp dir> <num reducers>
-
-where:
-
-input dir = A directory with file created by stresso.trie.Generate
-node size = Size of node in bits which must be a divisor of 32/64
-tmp dir = This command runs two map reduce jobs and needs an intermediate directory to store data.
-num reducers = Number of reduce task map reuduce job should run
-```
-
-Run the [load.sh](/bin/load.sh) script on a table with existing data. It starts
-a map reduce job that executes load transactions. Loading the same directory
-multiple times should not result in incorrect counts.
-
-```
-load.sh <input dir>
-```
-
-After loading data, run the [print.sh](/bin/print.sh) script to check the
-status of the computation of the number of unique integers within Apache Fluo. This
-command will print two numbers, the sum of the root nodes and number of root
-nodes. If there are outstanding notification to process, this count may not be
-accurate.
-
-```
-print.sh
-```
-
-In order to know how many unique numbers are expected, run the [unique.sh](/bin/unique.sh)
-script. This scrpt runs a map reduce job that calculates the number of
-unique integers. This script can take a list of directories created by
-multiple runs of [generate.sh](/bin/generate.sh)
-
-```
-unique.sh <num reducers> <input dir>{ <input dir>}
-```
-
-As transactions execute they leave a trail of history behind. The nodes in the
-lower levels of the tree are updated by many transactions and therefore have a
-long history trail. A long transactional history can slow down transactions.
-Forcing a compaction in Accumulo will clean up this history. However
-compacting the entire table is expensive. To avoid this expense, compact only the
-lower levels of the tree. The following command will compact levels of the
-tree with a maximum number of nodes less than the specified cutoff.
-
-```
-compact-ll.sh <max> <cutoff>
-```
-
-where:
-
-```
-cutoff = Any level of the tree with a maximum number of nodes that is less than this cutoff will be compacted.
-```
+Examples for Apache Fluo
diff --git a/.gitignore b/stresso/.gitignore
similarity index 100%
rename from .gitignore
rename to stresso/.gitignore
diff --git a/.travis.yml b/stresso/.travis.yml
similarity index 100%
rename from .travis.yml
rename to stresso/.travis.yml
diff --git a/AUTHORS b/stresso/AUTHORS
similarity index 100%
rename from AUTHORS
rename to stresso/AUTHORS
diff --git a/stresso/LICENSE b/stresso/LICENSE
new file mode 100644
index 0000000..37ec93a
--- /dev/null
+++ b/stresso/LICENSE
@@ -0,0 +1,191 @@
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction, and
+distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by the copyright
+owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all other entities
+that control, are controlled by, or are under common control with that entity.
+For the purposes of this definition, "control" means (i) the power, direct or
+indirect, to cause the direction or management of such entity, whether by
+contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity exercising
+permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications, including
+but not limited to software source code, documentation source, and configuration
+files.
+
+"Object" form shall mean any form resulting from mechanical transformation or
+translation of a Source form, including but not limited to compiled object code,
+generated documentation, and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or Object form, made
+available under the License, as indicated by a copyright notice that is included
+in or attached to the work (an example is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object form, that
+is based on (or derived from) the Work and for which the editorial revisions,
+annotations, elaborations, or other modifications represent, as a whole, an
+original work of authorship. For the purposes of this License, Derivative Works
+shall not include works that remain separable from, or merely link (or bind by
+name) to the interfaces of, the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including the original version
+of the Work and any modifications or additions to that Work or Derivative Works
+thereof, that is intentionally submitted to Licensor for inclusion in the Work
+by the copyright owner or by an individual or Legal Entity authorized to submit
+on behalf of the copyright owner. For the purposes of this definition,
+"submitted" means any form of electronic, verbal, or written communication sent
+to the Licensor or its representatives, including but not limited to
+communication on electronic mailing lists, source code control systems, and
+issue tracking systems that are managed by, or on behalf of, the Licensor for
+the purpose of discussing and improving the Work, but excluding communication
+that is conspicuously marked or otherwise designated in writing by the copyright
+owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity on behalf
+of whom a Contribution has been received by Licensor and subsequently
+incorporated within the Work.
+
+2. Grant of Copyright License.
+
+Subject to the terms and conditions of this License, each Contributor hereby
+grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+irrevocable copyright license to reproduce, prepare Derivative Works of,
+publicly display, publicly perform, sublicense, and distribute the Work and such
+Derivative Works in Source or Object form.
+
+3. Grant of Patent License.
+
+Subject to the terms and conditions of this License, each Contributor hereby
+grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
+irrevocable (except as stated in this section) patent license to make, have
+made, use, offer to sell, sell, import, and otherwise transfer the Work, where
+such license applies only to those patent claims licensable by such Contributor
+that are necessarily infringed by their Contribution(s) alone or by combination
+of their Contribution(s) with the Work to which such Contribution(s) was
+submitted. If You institute patent litigation against any entity (including a
+cross-claim or counterclaim in a lawsuit) alleging that the Work or a
+Contribution incorporated within the Work constitutes direct or contributory
+patent infringement, then any patent licenses granted to You under this License
+for that Work shall terminate as of the date such litigation is filed.
+
+4. Redistribution.
+
+You may reproduce and distribute copies of the Work or Derivative Works thereof
+in any medium, with or without modifications, and in Source or Object form,
+provided that You meet the following conditions:
+
+You must give any other recipients of the Work or Derivative Works a copy of
+this License; and
+You must cause any modified files to carry prominent notices stating that You
+changed the files; and
+You must retain, in the Source form of any Derivative Works that You distribute,
+all copyright, patent, trademark, and attribution notices from the Source form
+of the Work, excluding those notices that do not pertain to any part of the
+Derivative Works; and
+If the Work includes a "NOTICE" text file as part of its distribution, then any
+Derivative Works that You distribute must include a readable copy of the
+attribution notices contained within such NOTICE file, excluding those notices
+that do not pertain to any part of the Derivative Works, in at least one of the
+following places: within a NOTICE text file distributed as part of the
+Derivative Works; within the Source form or documentation, if provided along
+with the Derivative Works; or, within a display generated by the Derivative
+Works, if and wherever such third-party notices normally appear. The contents of
+the NOTICE file are for informational purposes only and do not modify the
+License. You may add Your own attribution notices within Derivative Works that
+You distribute, alongside or as an addendum to the NOTICE text from the Work,
+provided that such additional attribution notices cannot be construed as
+modifying the License.
+You may add Your own copyright statement to Your modifications and may provide
+additional or different license terms and conditions for use, reproduction, or
+distribution of Your modifications, or for any such Derivative Works as a whole,
+provided Your use, reproduction, and distribution of the Work otherwise complies
+with the conditions stated in this License.
+
+5. Submission of Contributions.
+
+Unless You explicitly state otherwise, any Contribution intentionally submitted
+for inclusion in the Work by You to the Licensor shall be under the terms and
+conditions of this License, without any additional terms or conditions.
+Notwithstanding the above, nothing herein shall supersede or modify the terms of
+any separate license agreement you may have executed with Licensor regarding
+such Contributions.
+
+6. Trademarks.
+
+This License does not grant permission to use the trade names, trademarks,
+service marks, or product names of the Licensor, except as required for
+reasonable and customary use in describing the origin of the Work and
+reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty.
+
+Unless required by applicable law or agreed to in writing, Licensor provides the
+Work (and each Contributor provides its Contributions) on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
+including, without limitation, any warranties or conditions of TITLE,
+NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
+solely responsible for determining the appropriateness of using or
+redistributing the Work and assume any risks associated with Your exercise of
+permissions under this License.
+
+8. Limitation of Liability.
+
+In no event and under no legal theory, whether in tort (including negligence),
+contract, or otherwise, unless required by applicable law (such as deliberate
+and grossly negligent acts) or agreed to in writing, shall any Contributor be
+liable to You for damages, including any direct, indirect, special, incidental,
+or consequential damages of any character arising as a result of this License or
+out of the use or inability to use the Work (including but not limited to
+damages for loss of goodwill, work stoppage, computer failure or malfunction, or
+any and all other commercial damages or losses), even if such Contributor has
+been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability.
+
+While redistributing the Work or Derivative Works thereof, You may choose to
+offer, and charge a fee for, acceptance of support, warranty, indemnity, or
+other liability obligations and/or rights consistent with this License. However,
+in accepting such obligations, You may act only on Your own behalf and on Your
+sole responsibility, not on behalf of any other Contributor, and only if You
+agree to indemnify, defend, and hold each Contributor harmless for any liability
+incurred by, or claims asserted against, such Contributor by reason of your
+accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work
+
+To apply the Apache License to your work, attach the following boilerplate
+notice, with the fields enclosed by brackets "[]" replaced with your own
+identifying information. (Don't include the brackets!) The text should be
+enclosed in the appropriate comment syntax for the file format. We also
+recommend that a file or class name and description of purpose be included on
+the same "printed page" as the copyright notice for easier identification within
+third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/stresso/README.md b/stresso/README.md
new file mode 100644
index 0000000..d3c2577
--- /dev/null
+++ b/stresso/README.md
@@ -0,0 +1,192 @@
+
+# Stresso
+
+[](https://travis-ci.org/astralway/stresso)
+
+An example application designed to stress Apache Fluo. This Fluo application computes the
+number of unique integers through the process of building a bitwise trie. New numbers
+are added to the trie as leaf nodes. Observers watch all nodes in the trie to create
+parents and percolate counts up to the root nodes such that each node in the trie keeps
+track of the number of leaf nodes below it. The count at the root nodes should equal
+the total number of leaf nodes. This makes it easy to verify if the test ran correctly.
+The test stresses Apache Fluo in that multiple transactions can operate on the same data
+as counts are percolated up the trie.
+
+## Concepts and definitions
+
+This test has the following set of configurable parameters.
+
+ * **nodeSize** : The number of bits chopped off the end each time a number is
+ percolated up. Must choose a nodeSize such that `64 % nodeSize == 0`
+ * **stopLevel** : The number of levels in the tree is a function of the
+ nodeSize. The deepest possible level is `64 / nodeSize`. Levels are
+ decremented going up the tree. Setting the stop level determines how far up
+ to percolate. The lower the stop level, the more root nodes there are.
+ Having more root nodes means less collisions, but all roots need to be
+ scanned to get the count of unique numbers. Having ~64k root nodes is a
+ good choice.
+ * **max** : Random numbers are generated modulo the max.
+
+Setting the stop level such that you have ~64k root nodes is dependent on the
+max and nodeSize. For example assume we choose a max of 10<sup>12</sup> and a
+node size of 8. The following table shows information about each level in the
+tree using this configuration. So for a max of 10<sup>12</sup> choosing a stop
+level of 5 would result in 59,604 root nodes. With this many root nodes there
+would not be many collisions and scanning 59,604 nodes to compute the unique
+number of intergers is a quick operation.
+
+|Level|Max Node |Number of possible Nodes|
+|:---:|---------------------|-----------------------:|
+| 0 |`0xXXXXXXXXXXXXXXXX` | 1 |
+| 1 |`0x00XXXXXXXXXXXXXX` | 1 |
+| 2 |`0x0000XXXXXXXXXXXX` | 1 |
+| 3 |`0x000000XXXXXXXXXX` | 1 |
+| 4 |`0x000000E8XXXXXXXX` | 232 |
+| 5 |`0x000000E8D4XXXXXX` | 59,604 |
+| 6 |`0x000000E8D4A5XXXX` | 15,258,789 |
+| 7 |`0x000000E8D4A510XX` | 3,906,250,000 |
+| 8 |`0x000000E8D4A51000` | 1,000,000,000,000 |
+
+In the table above, X indicates nibbles that are always zeroed out for every
+node at that level. You can easily view nodes at a level using a row prefix
+with the fluo scan command. For example `fluo scan -p 05` shows all nodes at
+level 5.
+
+For small scale test a max of 10<sup>9</sup> and a stop level of 6 is a good
+choice.
+
+## Building Stresso
+
+```
+mvn package
+```
+
+This will create a jar and shaded jar in target:
+
+```
+$ ls target/stresso-*
+target/stresso-0.0.1-SNAPSHOT.jar target/stresso-0.0.1-SNAPSHOT-shaded.jar
+```
+
+## Run Stresso using MiniFluo
+
+There are several integration tests that run Stresso on a MiniFluo instance.
+These tests can be run using `mvn verify`.
+
+## Run Stresso on cluster
+
+The [bin directory](/bin) contains a set of scripts to help run this test on a
+cluster. These scripts make the following assumpitions.
+
+ * `FLUO_HOME` environment variable is set. If not set, then set it in `conf/env.sh`.
+ * Hadoop `yarn` command is on path.
+ * Hadoop `hadoop` command is on path.
+ * Accumulo `accumulo` command is on path.
+
+Before running any of the scipts, copy [conf/env.sh.example](/conf/env.sh.example)
+to `conf/env.sh`, then inspect and modify the file.
+
+Next, execute the [run-test.sh](/bin/run-test.sh) script. This script will create a
+new Apache Fluo app called `stresso` (which can be changed by `FLUO_APP_NAME` in your env.sh).
+It will modify the application's fluo.properties, copy the stresso jar to the `lib/`
+directory of the app and set the following in fluo.properties:
+
+```
+fluo.observer.0=stresso.trie.NodeObserver
+fluo.app.trie.nodeSize=X
+fluo.app.trie.stopLevel=Y
+```
+
+The `run-test.sh` script will then initialize and start the Stresso application.
+It will load a lot of data directly into Accumulo without transactions and then
+incrementally load smaller amounts of data using transactions. After incrementally
+loading some data, it computes the expected number of unique integers using map reduce.
+It then prints the number of unique integers computed by Apache Fluo.
+
+## Additional Scripts
+
+The script [generate.sh](/bin/generate.sh) starts a map reduce job to generate
+random integers.
+
+```
+generate.sh <num files> <num per file> <max> <out dir>
+
+where:
+
+num files = Number of files to generate (and number of map task)
+numPerMap = Number of random numbers to generate per file
+max = Generate random numbers between 0 and max
+out dir = Output directory
+```
+
+The script [split.sh](/bin/split.sh) pre-splits the Accumulo table used by Apache
+Fluo. Consider running this command before loading data.
+
+```
+split.sh <num tablets> <max>
+
+where:
+
+num tablets = Num tablets to create for lowest level of tree. Will create less tablets for higher levels based on the max.
+```
+After generating random numbers, load them into Apache Fluo with one of the following
+commands. The script [init.sh](/bin/init.sh) intializes any empty table using
+map reduce. This simulates the case where a user has a lot of initial data to
+load into Fluo. This command should only be run when the table is empty
+because it writes directly to the Fluo table w/o using transactions.
+
+```
+init.sh <input dir> <tmp dir> <num reducers>
+
+where:
+
+input dir = A directory with file created by stresso.trie.Generate
+node size = Size of node in bits which must be a divisor of 32/64
+tmp dir = This command runs two map reduce jobs and needs an intermediate directory to store data.
+num reducers = Number of reduce task map reuduce job should run
+```
+
+Run the [load.sh](/bin/load.sh) script on a table with existing data. It starts
+a map reduce job that executes load transactions. Loading the same directory
+multiple times should not result in incorrect counts.
+
+```
+load.sh <input dir>
+```
+
+After loading data, run the [print.sh](/bin/print.sh) script to check the
+status of the computation of the number of unique integers within Apache Fluo. This
+command will print two numbers, the sum of the root nodes and number of root
+nodes. If there are outstanding notification to process, this count may not be
+accurate.
+
+```
+print.sh
+```
+
+In order to know how many unique numbers are expected, run the [unique.sh](/bin/unique.sh)
+script. This scrpt runs a map reduce job that calculates the number of
+unique integers. This script can take a list of directories created by
+multiple runs of [generate.sh](/bin/generate.sh)
+
+```
+unique.sh <num reducers> <input dir>{ <input dir>}
+```
+
+As transactions execute they leave a trail of history behind. The nodes in the
+lower levels of the tree are updated by many transactions and therefore have a
+long history trail. A long transactional history can slow down transactions.
+Forcing a compaction in Accumulo will clean up this history. However
+compacting the entire table is expensive. To avoid this expense, compact only the
+lower levels of the tree. The following command will compact levels of the
+tree with a maximum number of nodes less than the specified cutoff.
+
+```
+compact-ll.sh <max> <cutoff>
+```
+
+where:
+
+```
+cutoff = Any level of the tree with a maximum number of nodes that is less than this cutoff will be compacted.
+```
diff --git a/bin/compact-ll.sh b/stresso/bin/compact-ll.sh
similarity index 100%
rename from bin/compact-ll.sh
rename to stresso/bin/compact-ll.sh
diff --git a/bin/diff.sh b/stresso/bin/diff.sh
similarity index 100%
rename from bin/diff.sh
rename to stresso/bin/diff.sh
diff --git a/bin/generate.sh b/stresso/bin/generate.sh
similarity index 100%
rename from bin/generate.sh
rename to stresso/bin/generate.sh
diff --git a/bin/init.sh b/stresso/bin/init.sh
similarity index 100%
rename from bin/init.sh
rename to stresso/bin/init.sh
diff --git a/bin/load-env.sh b/stresso/bin/load-env.sh
similarity index 100%
rename from bin/load-env.sh
rename to stresso/bin/load-env.sh
diff --git a/bin/load.sh b/stresso/bin/load.sh
similarity index 100%
rename from bin/load.sh
rename to stresso/bin/load.sh
diff --git a/bin/print.sh b/stresso/bin/print.sh
similarity index 100%
rename from bin/print.sh
rename to stresso/bin/print.sh
diff --git a/bin/run-test.sh b/stresso/bin/run-test.sh
similarity index 100%
rename from bin/run-test.sh
rename to stresso/bin/run-test.sh
diff --git a/bin/split.sh b/stresso/bin/split.sh
similarity index 100%
rename from bin/split.sh
rename to stresso/bin/split.sh
diff --git a/bin/unique.sh b/stresso/bin/unique.sh
similarity index 100%
rename from bin/unique.sh
rename to stresso/bin/unique.sh
diff --git a/conf/.gitignore b/stresso/conf/.gitignore
similarity index 100%
rename from conf/.gitignore
rename to stresso/conf/.gitignore
diff --git a/conf/env.sh.example b/stresso/conf/env.sh.example
similarity index 100%
rename from conf/env.sh.example
rename to stresso/conf/env.sh.example
diff --git a/conf/log4j.xml b/stresso/conf/log4j.xml
similarity index 100%
rename from conf/log4j.xml
rename to stresso/conf/log4j.xml
diff --git a/pom.xml b/stresso/pom.xml
similarity index 100%
rename from pom.xml
rename to stresso/pom.xml
diff --git a/src/main/java/stresso/trie/CompactLL.java b/stresso/src/main/java/stresso/trie/CompactLL.java
similarity index 100%
rename from src/main/java/stresso/trie/CompactLL.java
rename to stresso/src/main/java/stresso/trie/CompactLL.java
diff --git a/src/main/java/stresso/trie/Constants.java b/stresso/src/main/java/stresso/trie/Constants.java
similarity index 100%
rename from src/main/java/stresso/trie/Constants.java
rename to stresso/src/main/java/stresso/trie/Constants.java
diff --git a/src/main/java/stresso/trie/Diff.java b/stresso/src/main/java/stresso/trie/Diff.java
similarity index 100%
rename from src/main/java/stresso/trie/Diff.java
rename to stresso/src/main/java/stresso/trie/Diff.java
diff --git a/src/main/java/stresso/trie/Generate.java b/stresso/src/main/java/stresso/trie/Generate.java
similarity index 100%
rename from src/main/java/stresso/trie/Generate.java
rename to stresso/src/main/java/stresso/trie/Generate.java
diff --git a/src/main/java/stresso/trie/Init.java b/stresso/src/main/java/stresso/trie/Init.java
similarity index 100%
rename from src/main/java/stresso/trie/Init.java
rename to stresso/src/main/java/stresso/trie/Init.java
diff --git a/src/main/java/stresso/trie/Load.java b/stresso/src/main/java/stresso/trie/Load.java
similarity index 100%
rename from src/main/java/stresso/trie/Load.java
rename to stresso/src/main/java/stresso/trie/Load.java
diff --git a/src/main/java/stresso/trie/Node.java b/stresso/src/main/java/stresso/trie/Node.java
similarity index 100%
rename from src/main/java/stresso/trie/Node.java
rename to stresso/src/main/java/stresso/trie/Node.java
diff --git a/src/main/java/stresso/trie/NodeObserver.java b/stresso/src/main/java/stresso/trie/NodeObserver.java
similarity index 100%
rename from src/main/java/stresso/trie/NodeObserver.java
rename to stresso/src/main/java/stresso/trie/NodeObserver.java
diff --git a/src/main/java/stresso/trie/NumberLoader.java b/stresso/src/main/java/stresso/trie/NumberLoader.java
similarity index 100%
rename from src/main/java/stresso/trie/NumberLoader.java
rename to stresso/src/main/java/stresso/trie/NumberLoader.java
diff --git a/src/main/java/stresso/trie/Print.java b/stresso/src/main/java/stresso/trie/Print.java
similarity index 100%
rename from src/main/java/stresso/trie/Print.java
rename to stresso/src/main/java/stresso/trie/Print.java
diff --git a/src/main/java/stresso/trie/Split.java b/stresso/src/main/java/stresso/trie/Split.java
similarity index 100%
rename from src/main/java/stresso/trie/Split.java
rename to stresso/src/main/java/stresso/trie/Split.java
diff --git a/src/main/java/stresso/trie/Unique.java b/stresso/src/main/java/stresso/trie/Unique.java
similarity index 100%
rename from src/main/java/stresso/trie/Unique.java
rename to stresso/src/main/java/stresso/trie/Unique.java
diff --git a/src/test/java/stresso/ITBase.java b/stresso/src/test/java/stresso/ITBase.java
similarity index 100%
rename from src/test/java/stresso/ITBase.java
rename to stresso/src/test/java/stresso/ITBase.java
diff --git a/src/test/java/stresso/TrieBasicIT.java b/stresso/src/test/java/stresso/TrieBasicIT.java
similarity index 100%
rename from src/test/java/stresso/TrieBasicIT.java
rename to stresso/src/test/java/stresso/TrieBasicIT.java
diff --git a/src/test/java/stresso/TrieMapRedIT.java b/stresso/src/test/java/stresso/TrieMapRedIT.java
similarity index 100%
rename from src/test/java/stresso/TrieMapRedIT.java
rename to stresso/src/test/java/stresso/TrieMapRedIT.java
diff --git a/src/test/java/stresso/TrieStopLevelIT.java b/stresso/src/test/java/stresso/TrieStopLevelIT.java
similarity index 100%
rename from src/test/java/stresso/TrieStopLevelIT.java
rename to stresso/src/test/java/stresso/TrieStopLevelIT.java
diff --git a/src/test/resources/log4j.properties b/stresso/src/test/resources/log4j.properties
similarity index 100%
rename from src/test/resources/log4j.properties
rename to stresso/src/test/resources/log4j.properties
