Avoid checking Accumulo table exists before creation (#74)
The initial focus of this ticket was to remove the check for table existence and just create and catch and then ignore TableExistsException instead.
After initial work (see previous comments), it was decided to update the code to work more realistically. Primarily, do not ignore the exceptions and at least alert the user that the table existed prior and let the user decide whether to remove the table and re-run the example.
Another concern was the fear of interfering with an existing table on a users system.
The primary changes then became to update table creation and provide feedback via logs to situations where prior tables already existed. In order to prevent table name collision, the examples were modified to make use of Accumulo namespaces. The classes and documentation were update to create an 'examples' namespace wherein all the table are created.
Along the way several other smaller tweaks were made also. Some of these are listed below.
* The process of table creation was re-factored into a <code>Commons</code> class. All examples now use a method in that class to create both the namespace and tablename. A couple of constants used throughout the example classes are defined there as well.
* The bloom classes now have a couple of methods that helped to remove some redundant code.
* An unneeded import was removed from the CharacterHistogram.java class.
* Most of the use of System.out.println was replace with logging instead.
* Update SequentialBatchWriter to exit if required table for scanning does not exist.
* A majority of the documentation was updated to included the creation of the necessary 'examples' namespace.
* The config command was updated to use the table.class.loader.context rather than the deprecated table.classpath.context.
* Update the constraints example to work with the new location of the contraints classes in Accumulo.
* Update filedata documentation to note that the ChunkCombiner class must be avaiable in the accumulo lib directory or on the classpath somewhere in order to scan the create examples.dataTable.
Closes #13
diff --git a/.gitignore b/.gitignore
index 614659a..fca81e8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,3 +6,4 @@
/.idea
/examples.conf
/conf/env.sh
+/conf/log4j.properties
diff --git a/docs/batch.md b/docs/batch.md
index 305a732..af9205e 100644
--- a/docs/batch.md
+++ b/docs/batch.md
@@ -30,8 +30,8 @@
Verify data was ingested by scanning the table using the Accumulo shell:
$ accumulo shell
- root@instance> table batch
- root@instance batch> scan
+ root@instance> table examples.batch
+ root@instance examples.batch> scan
Run `RandomBatchScanner` to perform 1000 random queries and verify the results.
diff --git a/docs/bloom.md b/docs/bloom.md
index 528bbb5..8a38df5 100644
--- a/docs/bloom.md
+++ b/docs/bloom.md
@@ -39,10 +39,10 @@
$ ./bin/runex bloom.BloomBatchScanner
- Scanning bloom_test1 with seed 7
+ Scanning example.bloom_test1 with seed 7
Scan finished! 282.49 lookups/sec, 1.77 secs, 500 results
All expected rows were scanned
- Scanning bloom_test2 with seed 7
+ Scanning examples.bloom_test2 with seed 7
Scan finished! 704.23 lookups/sec, 0.71 secs, 500 results
All expected rows were scanned
@@ -52,8 +52,8 @@
$ accumulo shell -u username -p password -e 'tables -l'
accumulo.metadata => !0
accumulo.root => +r
- bloom_test1 => 2
- bloom_test2 => 3
+ examples.bloom_test1 => 2
+ examples.bloom_test2 => 3
trace => 1
So the table id for bloom_test2 is 3. The command below shows what files this
@@ -103,10 +103,10 @@
Once the flush completes, 500 random queries are done against each table but with a different seed.
Even when nothing is found the lookups are faster against the table with the bloom filters.
- Writing data to bloom_test3 and bloom_test4 (bloom filters enabled)
- Scanning bloom_test3 with seed 8
+ Writing data to examples.bloom_test3 and examples.bloom_test4 (bloom filters enabled)
+ Scanning examples.bloom_test3 with seed 8
Scan finished! 780.03 lookups/sec, 0.64 secs, 0 results
Did not find 500
- Scanning bloom_test4 with seed 8
+ Scanning examples.bloom_test4 with seed 8
Scan finished! 1736.11 lookups/sec, 0.29 secs, 0 results
Did not find 500
diff --git a/docs/bulkIngest.md b/docs/bulkIngest.md
index 6edee37..28cd3c8 100644
--- a/docs/bulkIngest.md
+++ b/docs/bulkIngest.md
@@ -20,7 +20,7 @@
This tutorial uses the following Java classes.
- * [SetupTable.java] - creates the table, 'test_bulk', and sets two split points.
+ * [SetupTable.java] - creates the table, 'examples.test_bulk', and sets two split points.
* [BulkIngestExample.java] - creates some data to ingest and then ingests the data using mapReduce
* [VerifyIngest.java] - checks that the data was ingested
diff --git a/docs/classpath.md b/docs/classpath.md
index 37526eb..8e8cc28 100644
--- a/docs/classpath.md
+++ b/docs/classpath.md
@@ -33,40 +33,41 @@
Execute following in Accumulo shell to setup classpath context
- root@test15> config -s general.vfs.context.classpath.cx1=hdfs://<namenode host>:<namenode port>/user1/lib/[^.].*.jar
+ root@uno> config -s general.vfs.context.classpath.cx1=hdfs://<namenode host>:<namenode port>/user1/lib/[^.].*.jar
-Create a table
+Create a namespace and table
- root@test15> createtable nofoo
+ root@uno> createnamespace examples
+ root@uno> createtable examples.nofoo
The following command makes this table use the configured classpath context
- root@test15 nofoo> config -t nofoo -s table.classpath.context=cx1
+ root@uno examples.nofoo> config -t examples.nofoo -s table.class.loader.context=cx1
The following command configures an iterator that's in FooFilter.jar
- root@test15 nofoo> setiter -n foofilter -p 10 -scan -minc -majc -class org.apache.accumulo.test.FooFilter
+ root@uno examples.nofoo> setiter -n foofilter -p 10 -scan -minc -majc -class org.apache.accumulo.test.FooFilter
Filter accepts or rejects each Key/Value pair
----------> set FooFilter parameter negate, default false keeps k/v that pass accept method, true rejects k/v that pass accept method: false
The commands below show the filter is working.
- root@test15 nofoo> insert foo1 f1 q1 v1
- root@test15 nofoo> insert noo1 f1 q1 v2
- root@test15 nofoo> scan
+ root@uno examples.nofoo> insert foo1 f1 q1 v1
+ root@uno examples.nofoo> insert noo1 f1 q1 v2
+ root@uno examples.nofoo> scan
noo1 f1:q1 [] v2
- root@test15 nofoo>
+ root@uno examples.nofoo>
Below, an attempt is made to add the FooFilter to a table that's not configured
to use the classpath context cx1. This fails until the table is configured to
use cx1.
- root@test15 nofoo> createtable nofootwo
- root@test15 nofootwo> setiter -n foofilter -p 10 -scan -minc -majc -class org.apache.accumulo.test.FooFilter
+ root@uno examples.nofoo> createtable examples.nofootwo
+ root@uno examples.nofootwo> setiter -n foofilter -p 10 -scan -minc -majc -class org.apache.accumulo.test.FooFilter
2013-05-03 12:49:35,943 [shell.Shell] ERROR: org.apache.accumulo.shell.ShellCommandException: Command could
not be initialized (Unable to load org.apache.accumulo.test.FooFilter; class not found.)
- root@test15 nofootwo> config -t nofootwo -s table.classpath.context=cx1
- root@test15 nofootwo> setiter -n foofilter -p 10 -scan -minc -majc -class org.apache.accumulo.test.FooFilter
+ root@uno examples.nofootwo> config -t nofootwo -s table.class.loader.context=cx1
+ root@uno examples.nofootwo> setiter -n foofilter -p 10 -scan -minc -majc -class org.apache.accumulo.test.FooFilter
Filter accepts or rejects each Key/Value pair
----------> set FooFilter parameter negate, default false keeps k/v that pass accept method, true rejects k/v that pass accept method: false
diff --git a/docs/combiner.md b/docs/combiner.md
index 48dc7e3..7ee9cd1 100644
--- a/docs/combiner.md
+++ b/docs/combiner.md
@@ -29,43 +29,44 @@
Shell - Apache Accumulo Interactive Shell
-
- - version: 1.5.0
+ - version: 2.1.0-SNAPSHOT
- instance name: instance
- instance id: 00000000-0000-0000-0000-000000000000
-
- type 'help' for a list of available commands
-
- username@instance> createtable runners
- username@instance runners> setiter -t runners -p 10 -scan -minc -majc -n decStats -class org.apache.accumulo.examples.combiner.StatsCombiner
+ username@instance> createnamespace examples
+ username@instance> createtable examples.runners
+ username@instance exampoles.runners> setiter -t runners -p 10 -scan -minc -majc -n decStats -class org.apache.accumulo.examples.combiner.StatsCombiner
Combiner that keeps track of min, max, sum, and count
----------> set StatsCombiner parameter all, set to true to apply Combiner to every column, otherwise leave blank. if true, columns option will be ignored.:
----------> set StatsCombiner parameter columns, <col fam>[:<col qual>]{,<col fam>[:<col qual>]} escape non aplhanum chars using %<hex>.: stat
----------> set StatsCombiner parameter reduceOnFullCompactionOnly, If true, only reduce on full major compactions. Defaults to false. :
----------> set StatsCombiner parameter radix, radix/base of the numbers: 10
- username@instance runners> setiter -t runners -p 11 -scan -minc -majc -n hexStats -class org.apache.accumulo.examples.combiner.StatsCombiner
+ username@instance examples.runners> setiter -t runners -p 11 -scan -minc -majc -n hexStats -class org.apache.accumulo.examples.combiner.StatsCombiner
Combiner that keeps track of min, max, sum, and count
----------> set StatsCombiner parameter all, set to true to apply Combiner to every column, otherwise leave blank. if true, columns option will be ignored.:
----------> set StatsCombiner parameter columns, <col fam>[:<col qual>]{,<col fam>[:<col qual>]} escape non-alphanum chars using %<hex>.: hstat
----------> set StatsCombiner parameter reduceOnFullCompactionOnly, If true, only reduce on full major compactions. Defaults to false. :
----------> set StatsCombiner parameter radix, radix/base of the numbers: 16
- username@instance runners> insert 123456 name first Joe
- username@instance runners> insert 123456 stat marathon 240
- username@instance runners> scan
+ username@instance examples.runners> insert 123456 name first Joe
+ username@instance examples.runners> insert 123456 stat marathon 240
+ username@instance examples.runners> scan
123456 name:first [] Joe
123456 stat:marathon [] 240,240,240,1
- username@instance runners> insert 123456 stat marathon 230
- username@instance runners> insert 123456 stat marathon 220
- username@instance runners> scan
+ username@instance examples.runners> insert 123456 stat marathon 230
+ username@instance examples.runners> insert 123456 stat marathon 220
+ username@instance examples.runners> scan
123456 name:first [] Joe
123456 stat:marathon [] 220,240,690,3
- username@instance runners> insert 123456 hstat virtualMarathon 6a
- username@instance runners> insert 123456 hstat virtualMarathon 6b
- username@instance runners> scan
+ username@instance examples.runners> insert 123456 hstat virtualMarathon 6a
+ username@instance examples.runners> insert 123456 hstat virtualMarathon 6b
+ username@instance examples.runners> scan
123456 hstat:virtualMarathon [] 6a,6b,d5,2
123456 name:first [] Joe
123456 stat:marathon [] 220,240,690,3
-In this example a table is created and the example stats combiner is applied to
+In this example a table is created, and the example stats combiner is applied to
the column family stat and hstat. The stats combiner computes min,max,sum, and
count. It can be configured to use a different base or radix. In the example
above the column family stat is configured for base 10 and the column family
diff --git a/docs/compactionStrategy.md b/docs/compactionStrategy.md
index 8b63ae8..594b28d 100644
--- a/docs/compactionStrategy.md
+++ b/docs/compactionStrategy.md
@@ -34,7 +34,8 @@
To begin, run the command to create a table for testing.
```bash
-$ accumulo shell -u <username> -p <password> -e "createtable test1"
+$ accumulo shell -u <username> -p <password> -e "createnamespace examples"
+$ accumulo shell -u <username> -p <password> -e "createtable examples.test1"
```
The commands below will configure the BasicCompactionStrategy to:
@@ -43,24 +44,24 @@
- Compact files less than 100M using snappy.
```bash
- $ accumulo shell -u <username> -p <password> -e "config -t test1 -s table.file.compress.type=snappy"
- $ accumulo shell -u <username> -p <password> -e "config -t test1 -s table.majc.compaction.strategy=org.apache.accumulo.tserver.compaction.strategies.BasicCompactionStrategy"
- $ accumulo shell -u <username> -p <password> -e "config -t test1 -s table.majc.compaction.strategy.opts.filter.size=250M"
- $ accumulo shell -u <username> -p <password> -e "config -t test1 -s table.majc.compaction.strategy.opts.large.compress.threshold=100M"
- $ accumulo shell -u <username> -p <password> -e "config -t test1 -s table.majc.compaction.strategy.opts.large.compress.type=gz"
+ $ accumulo shell -u <username> -p <password> -e "config -t examples.test1 -s table.file.compress.type=snappy"
+ $ accumulo shell -u <username> -p <password> -e "config -t test1 -s examples.table.majc.compaction.strategy=org.apache.accumulo.tserver.compaction.strategies.BasicCompactionStrategy"
+ $ accumulo shell -u <username> -p <password> -e "config -t test1 -s examples.table.majc.compaction.strategy.opts.filter.size=250M"
+ $ accumulo shell -u <username> -p <password> -e "config -t test1 -s examples.table.majc.compaction.strategy.opts.large.compress.threshold=100M"
+ $ accumulo shell -u <username> -p <password> -e "config -t test1 -s examples.table.majc.compaction.strategy.opts.large.compress.type=gz"
```
Generate some data and files in order to test the strategy:
```bash
-$ ./bin/runex client.SequentialBatchWriter -t test1 --start 0 --num 10000 --size 50
-$ accumulo shell -u <username> -p <password> -e "flush -t test1"
-$ ./bin/runex client.SequentialBatchWriter -t test1 --start 0 --num 11000 --size 50
-$ accumulo shell -u <username> -p <password> -e "flush -t test1"
-$ ./bin/runex client.SequentialBatchWriter -t test1 --start 0 --num 12000 --size 50
-$ accumulo shell -u <username> -p <password> -e "flush -t test1"
-$ ./bin/runex client.SequentialBatchWriter -t test1 --start 0 --num 13000 --size 50
-$ accumulo shell -u <username> -p <password> -e "flush -t test1"
+$ ./bin/runex client.SequentialBatchWriter -t examples.test1 --start 0 --num 10000 --size 50
+$ accumulo shell -u <username> -p <password> -e "flush -t examples.test1"
+$ ./bin/runex client.SequentialBatchWriter -t examples.test1 --start 0 --num 11000 --size 50
+$ accumulo shell -u <username> -p <password> -e "flush -t examples.test1"
+$ ./bin/runex client.SequentialBatchWriter -t examples.test1 --start 0 --num 12000 --size 50
+$ accumulo shell -u <username> -p <password> -e "flush -t examples.test1"
+$ ./bin/runex client.SequentialBatchWriter -t examples.test1 --start 0 --num 13000 --size 50
+$ accumulo shell -u <username> -p <password> -e "flush -t examples.test1"
```
View the tserver log in <accumulo_home>/logs for the compaction and find the name of the <rfile> that was compacted for your table. Print info about this file using the PrintInfo tool:
diff --git a/docs/constraints.md b/docs/constraints.md
index 71803bc..8231518 100644
--- a/docs/constraints.md
+++ b/docs/constraints.md
@@ -22,8 +22,6 @@
* [NumericValueConstraint.java] - a constraint that requires numeric string values
* [MaxMutationSize.java] - a constraint that limits the size of mutations accepted into a table
-Remember to copy the accumulo-examples-\*.jar to Accumulo's 'lib/ext' directory.
-
AlphaNumKeyConstraint prevents insertion of keys with characters not between aA and zZ or 0 to 9.
NumericValueConstraint prevents insertion of values with characters not between 0 and 9. The examples create mutations
that violate the constraint, throwing an exception.
diff --git a/docs/deleteKeyValuePair.md b/docs/deleteKeyValuePair.md
index 87315a3..b96cd29 100644
--- a/docs/deleteKeyValuePair.md
+++ b/docs/deleteKeyValuePair.md
@@ -20,28 +20,29 @@
```
$ /path/to/accumulo shell -u username -p secret
-username@instance> createtable deleteKeyValuePair
-username@instance deleteKeyValuePair> insert 567890 name first Joe
-username@instance deleteKeyValuePair> insert 567890 name last Smith
-username@instance deleteKeyValuePair> insert 567890 address city Columbia
+username@instance> createnamespace examples
+username@instance> createtable examples.deleteKeyValuePair
+username@instance examples.deleteKeyValuePair> insert 567890 name first Joe
+username@instance examples.deleteKeyValuePair> insert 567890 name last Smith
+username@instance examples.deleteKeyValuePair> insert 567890 address city Columbia
```
```
-username@instance deleteKeyValuePair> scan
+username@instance examples.deleteKeyValuePair> scan
567890 address:city [] Columbia
567890 name:first [] Joe
567890 name:last [] Smith
```
```
-username@instance deleteKeyValuePair> flush -w
+username@instance examples.deleteKeyValuePair> flush -w
2019-04-18 11:01:47,444 [shell.Shell] INFO : Flush of table deleteKeyValuePair completed.
```
-Get the deleteKeyValuePair table id.
+Get the examples.deleteKeyValuePair table id.
```
-username@instance deleteKeyValuePair> tables -l
+username@instance examples.deleteKeyValuePair> tables -l
accumulo.metadata => !0
accumulo.replication => +rep
accumulo.root => +r
@@ -51,7 +52,7 @@
Scan accumulo.metadata table to see the list of RFiles Accumulo is currently using.
```
-username@instance deleteKeyValuePair> scan -t accumulo.metadata -c file -r 1t<
+username@instance examples.deleteKeyValuePair> scan -t accumulo.metadata -c file -r 1t<
1t< file:hdfs://localhost:8020/accumulo/tables/1t/default_tablet/F00007em.rf
```
@@ -59,6 +60,7 @@
```
$ /path/to/accumulo rfile-info -d hdfs://localhost/accumulo/tables/1t/default_tablet/F00007em.rf
+RFile Version : 8
Locality group : <DEFAULT>
Num blocks : 1
@@ -77,27 +79,29 @@
Raw size : 141 bytes
Compressed size : 97 bytes
Compression type : gz
-567890 address:city [] 1555418057811 false -> Columbia
-567890 name:first [] 1555418067848 false -> Joe
-567890 name:last [] 1555418063052 false -> Smith
+
+567890 address:city [] 1618581304723 false -> Columbia
+567890 name:first [] 1618581291527 false -> Joe
+567890 name:last [] 1618581297272 false -> Smith
```
Delete a key-value pair and view a newly created RFile to verify the deletion flag is true.
```
$ /path/to/accumulo shell -u username -p secret
-username@instance> table deleteKeyValuePair
-username@instance deleteKeyValuePair> delete 567890 name first
-username@instance deleteKeyValuePair> flush -w
+username@instance> table examples.deleteKeyValuePair
+username@instance examples.deleteKeyValuePair> delete 567890 name first
+username@instance examples.deleteKeyValuePair> flush -w
```
```
-username@instance deleteKeyValuePair> scan -t accumulo.metadata -c file -r 1t<
+username@instance examples.deleteKeyValuePair> scan -t accumulo.metadata -c file -r 1t<
1t< file:hdfs://localhost:8020/accumulo/tables/1t/default_tablet/F00007em.rf
1t< file:hdfs://localhost:8020/accumulo/tables/1t/default_tablet/F00007fq.rf
```
```
$ /path/to/accumulo rfile-info -d hdfs://localhost/accumulo/tables/1t/default_tablet/F00007fq.rf
+RFile Version : 8
Locality group : <DEFAULT>
Num blocks : 1
@@ -116,26 +120,28 @@
Raw size : 121 bytes
Compressed size : 68 bytes
Compression type : gz
-
-567890 name:first [] 1555419184531 true ->
+
+567890 name:first [] 1618581499491 true ->
```
Compact the RFiles and verify the key-value pair was removed. The new RFile will start with 'A'.
```
$ /path/to/accumulo shell -u username -p secret
-username@instance deleteKeyValuePair> compact -t deleteKeyValuePair -w
+username@instance> compact -t examples.deleteKeyValuePair -w
2019-04-17 08:17:15,468 [shell.Shell] INFO : Compacting table ...
-2019-04-17 08:17:16,143 [shell.Shell] INFO : Compaction of table deleteKeyValuePair completed for given range
+2019-04-17 08:17:16,143 [shell.Shell] INFO : Compaction of table examples.deleteKeyValuePair
+completed for given range
```
```
-username@instance deleteKeyValuePair> scan -t accumulo.metadata -c file -r 1t<
+username@instance> scan -t accumulo.metadata -c file -r 1t<
lt< file:hdfs://localhost:8020/accumulo/tables/1t/default_tablet/A00007g1.rf
```
```
$ /path/to/accumulo rfile-info -v hdfs://localhost/accumulo/tables/1t/default_tablet/A00007g1.rf
+RFile Version : 8
Locality group : <DEFAULT>
Num blocks : 1
@@ -154,7 +160,10 @@
Raw size : 141 bytes
Compressed size : 96 bytes
Compression type : gz
-
-567890 address:city [] 1555418057811 false -> Columbia
-567890 name:last [] 1555418063052 false -> Smith
+
+Locality Group: <DEFAULT>
+Visibility Number of keys Percent of keys Number of blocks Percent of blocks
+ 2 100.00% 1 100.00%
+Number of keys: 2
+
```
diff --git a/docs/dirlist.md b/docs/dirlist.md
index 2b653cf..159f7aa 100644
--- a/docs/dirlist.md
+++ b/docs/dirlist.md
@@ -43,26 +43,26 @@
then run the Viewer:
- $ ./bin/runex dirlist.Viewer -t dirTable --dataTable dataTable --auths exampleVis --path /local/username/workspace
+ $ ./bin/runex dirlist.Viewer -t examples.dirTable --dataTable examples.dataTable --auths exampleVis --path /local/username/workspace
To list the contents of specific directories, use QueryUtil.java.
- $ ./bin/runex dirlist.QueryUtil -t dirTable --auths exampleVis --path /local/username
- $ ./bin/runex dirlist.QueryUtil -t dirTable --auths exampleVis --path /local/username/workspace
+ $ ./bin/runex dirlist.QueryUtil -t examples.dirTable --auths exampleVis --path /local/username
+ $ ./bin/runex dirlist.QueryUtil -t examples.dirTable --auths exampleVis --path /local/username/workspace
To perform searches on file or directory names, also use QueryUtil.java. Search terms must contain no more than one wild card and cannot contain "/".
*Note* these queries run on the _indexTable_ table instead of the dirTable table.
- $ ./bin/runex dirlist.QueryUtil -t indexTable --auths exampleVis --path filename --search
- $ ./bin/runex dirlist.QueryUtil -t indexTable --auths exampleVis --path 'filename*' --search
- $ ./bin/runex dirlist.QueryUtil -t indexTable --auths exampleVis --path '*jar' --search
- $ ./bin/runex dirlist.QueryUtil -t indexTable --auths exampleVis --path 'filename*jar' --search
+ $ ./bin/runex dirlist.QueryUtil -t examples.indexTable --auths exampleVis --path filename --search
+ $ ./bin/runex dirlist.QueryUtil -t examples.indexTable --auths exampleVis --path 'filename*' --search
+ $ ./bin/runex dirlist.QueryUtil -t examples.indexTable --auths exampleVis --path '*jar' --search
+ $ ./bin/runex dirlist.QueryUtil -t examples.indexTable --auths exampleVis --path 'filename*jar' --search
To count the number of direct children (directories and files) and descendants (children and children's descendants, directories and files), run the FileCount over the dirTable table.
The results are written back to the same table. FileCount reads from and writes to Accumulo. This requires scan authorizations for the read and a visibility for the data written.
In this example, the authorizations and visibility are set to the same value, exampleVis. See the [visibility example][vis] for more information on visibility and authorizations.
- $ ./bin/runex dirlist.FileCount -t dirTable --auths exampleVis
+ $ ./bin/runex dirlist.FileCount -t examples.dirTable --auths exampleVis
## Directory Table
diff --git a/docs/export.md b/docs/export.md
index 6bab52d..87d5385 100644
--- a/docs/export.md
+++ b/docs/export.md
@@ -24,22 +24,23 @@
for the duration of the distcp. An easy way to take a table offline without
interuppting access to it is to clone it and take the clone offline.
- root@test15> createtable table1
- root@test15 table1> insert a cf1 cq1 v1
- root@test15 table1> insert h cf1 cq1 v2
- root@test15 table1> insert z cf1 cq1 v3
- root@test15 table1> insert z cf1 cq2 v4
- root@test15 table1> addsplits -t table1 b r
- root@test15 table1> scan
+ root@test15> createnamespace examples
+ root@test15> createtable examples.table1
+ root@test15 examples.table1> insert a cf1 cq1 v1
+ root@test15 examples.table1> insert h cf1 cq1 v2
+ root@test15 examples.table1> insert z cf1 cq1 v3
+ root@test15 examples.table1> insert z cf1 cq2 v4
+ root@test15 examples.table1> addsplits -t examples.table1 b r
+ root@test15 examples.table1> scan
a cf1:cq1 [] v1
h cf1:cq1 [] v2
z cf1:cq1 [] v3
z cf1:cq2 [] v4
- root@test15> config -t table1 -s table.split.threshold=100M
- root@test15 table1> clonetable table1 table1_exp
- root@test15 table1> offline table1_exp
- root@test15 table1> exporttable -t table1_exp /tmp/table1_export
- root@test15 table1> quit
+ root@test15 examples.table1> config -t examples.table1 -s table.split.threshold=100M
+ root@test15 examples.table1> clonetable examples.table1 examples.table1_exp
+ root@test15 examples.table1table1> offline examples.table1_exp
+ root@test15 examples.table1> exporttable -t examples.table1_exp /tmp/table1_export
+ root@test15 examples.table1> quit
After executing the export command, a few files are created in the hdfs dir.
One of the files is a list of files to distcp as shown below.
@@ -61,17 +62,17 @@
The data, splits, config, and logical time information for the table were
preserved.
- root@test15> importtable table1_copy /tmp/table1_export_dest
- root@test15> table table1_copy
- root@test15 table1_copy> scan
+ root@test15> importtable examples.table1_copy /tmp/table1_export_dest
+ root@test15> table examples.table1_copy
+ root@test15 examples.table1_copy> scan
a cf1:cq1 [] v1
h cf1:cq1 [] v2
z cf1:cq1 [] v3
z cf1:cq2 [] v4
- root@test15 table1_copy> getsplits -t table1_copy
+ root@test15 examples.table1_copy> getsplits -t examples.table1_copy
b
r
- root@test15> config -t table1_copy -f split
+ root@test15> config -t examples.table1_copy -f split
---------+--------------------------+-------------------------------------------
SCOPE | NAME | VALUE
---------+--------------------------+-------------------------------------------
@@ -83,7 +84,7 @@
accumulo.root => +r
table1_copy => 5
trace => 1
- root@test15 table1_copy> scan -t accumulo.metadata -b 5 -c srv:time
+ root@test15> scan -t accumulo.metadata -b 5 -c srv:time
5;b srv:time [] M1343224500467
5;r srv:time [] M1343224500467
5< srv:time [] M1343224500467
diff --git a/docs/filedata.md b/docs/filedata.md
index 7ee9d9c..a77a35d 100644
--- a/docs/filedata.md
+++ b/docs/filedata.md
@@ -32,18 +32,21 @@
If you haven't already run the [dirlist example][dirlist], ingest a file with FileDataIngest.
- $ ./bin/runex filedata.FileDataIngest -t dataTable --auths exampleVis --chunk 1000 /path/to/accumulo/README.md
+ $ ./bin/runex filedata.FileDataIngest -t examples.dataTable --auths exampleVis --chunk 1000 /path/to/accumulo/README.md
-Open the accumulo shell and look at the data. The row is the MD5 hash of the file, which you can verify by running a command such as 'md5sum' on the file.
+Open the accumulo shell and look at the data. The row is the MD5 hash of the file, which you can
+verify by running a command such as 'md5sum' on the file. Note that in order to scan the
+examples.dataTable the class, org.apache.accumulo.examples.filedata.ChunkCombiner, must be in
+your classpath, or the accumulo-examples-shaded.jar should be moved to the accumulo lib directory.
- > scan -t dataTable
+ > scan -t examples.dataTable
Run the CharacterHistogram MapReduce to add some information about the file.
- $ ./bin/runmr filedata.CharacterHistogram -t dataTable --auths exampleVis --vis exampleVis
+ $ ./bin/runmr filedata.CharacterHistogram -t examples.dataTable --auths exampleVis --vis exampleVis
Scan again to see the histogram stored in the 'info' column family.
- > scan -t dataTable
+ > scan -t examples.dataTable
[dirlist]: dirlist.md
diff --git a/docs/filter.md b/docs/filter.md
index 3f41265..ee34b33 100644
--- a/docs/filter.md
+++ b/docs/filter.md
@@ -26,22 +26,23 @@
return value of the accept method is negated, so that key/value pairs accepted
by the method are omitted by the Filter.
- username@instance> createtable filtertest
- username@instance filtertest> setiter -t filtertest -scan -p 10 -n myfilter -ageoff
+ username@instance> createnamespace examples
+ username@instance> createtable examples.filtertest
+ username@instance examples.filtertest> setiter -t examples.filtertest -scan -p 10 -n myfilter -ageoff
AgeOffFilter removes entries with timestamps more than <ttl> milliseconds old
----------> set AgeOffFilter parameter negate, default false keeps k/v that pass accept method, true rejects k/v that pass accept method:
----------> set AgeOffFilter parameter ttl, time to live (milliseconds): 30000
----------> set AgeOffFilter parameter currentTime, if set, use the given value as the absolute time in milliseconds as the current time of day:
- username@instance filtertest> scan
- username@instance filtertest> insert foo a b c
- username@instance filtertest> scan
+ username@instance examples.filtertest> scan
+ username@instance examples.filtertest> insert foo a b c
+ username@instance examples.filtertest> scan
foo a:b [] c
- username@instance filtertest>
+ username@instance examples.filtertest>
... wait 30 seconds ...
- username@instance filtertest> scan
- username@instance filtertest>
+ username@instance examples.filtertest> scan
+ username@instance examples.filtertest>
Note the absence of the entry inserted more than 30 seconds ago. Since the
scope was set to "scan", this means the entry is still in Accumulo, but is
@@ -59,21 +60,21 @@
following commands show how to enable the AgeOffFilter for the minc and majc
scopes using the -class flag, then flush and compact the table.
- username@instance filtertest> setiter -t filtertest -minc -majc -p 10 -n myfilter -class org.apache.accumulo.core.iterators.user.AgeOffFilter
+ username@instance examples.filtertest> setiter -t examples.filtertest -minc -majc -p 10 -n myfilter -class org.apache.accumulo.core.iterators.user.AgeOffFilter
AgeOffFilter removes entries with timestamps more than <ttl> milliseconds old
----------> set AgeOffFilter parameter negate, default false keeps k/v that pass accept method, true rejects k/v that pass accept method:
----------> set AgeOffFilter parameter ttl, time to live (milliseconds): 30000
----------> set AgeOffFilter parameter currentTime, if set, use the given value as the absolute time in milliseconds as the current time of day:
- username@instance filtertest> flush
- 06 10:42:24,806 [shell.Shell] INFO : Flush of table filtertest initiated...
- username@instance filtertest> compact
- 06 10:42:36,781 [shell.Shell] INFO : Compaction of table filtertest started for given range
- username@instance filtertest> flush -t filtertest -w
- 06 10:42:52,881 [shell.Shell] INFO : Flush of table filtertest completed.
- username@instance filtertest> compact -t filtertest -w
+ username@instance examples.filtertest> flush
+ 06 10:42:24,806 [shell.Shell] INFO : Flush of table examples.filtertest initiated...
+ username@instance examples.filtertest> compact
+ 06 10:42:36,781 [shell.Shell] INFO : Compaction of table examples.filtertest started for given range
+ username@instance examples.filtertest> flush -t examples.filtertest -w
+ 06 10:42:52,881 [shell.Shell] INFO : Flush of table examples.filtertest completed.
+ username@instance examples.filtertest> compact -t examples.filtertest -w
06 10:43:00,632 [shell.Shell] INFO : Compacting table ...
- 06 10:43:01,307 [shell.Shell] INFO : Compaction of table filtertest completed for given range
- username@instance filtertest>
+ 06 10:43:01,307 [shell.Shell] INFO : Compaction of table examples.filtertest completed for given range
+ username@instance examples.filtertest>
By default, flush and compact execute in the background, but with the -w flag
they will wait to return until the operation has completed. Both are
@@ -86,7 +87,7 @@
To see the iterator settings for a table, use config.
- username@instance filtertest> config -t filtertest -f iterator
+ username@instance examples.filtertest> config -t examples.filtertest -f iterator
---------+---------------------------------------------+---------------------------------------------------------------------------
SCOPE | NAME | VALUE
---------+---------------------------------------------+---------------------------------------------------------------------------
@@ -103,7 +104,7 @@
table | table.iterator.scan.vers .................. | 20,org.apache.accumulo.core.iterators.user.VersioningIterator
table | table.iterator.scan.vers.opt.maxVersions .. | 1
---------+---------------------------------------------+---------------------------------------------------------------------------
- username@instance filtertest>
+ username@instance examples.filtertest>
When setting new iterators, make sure to order their priority numbers
(specified with -p) in the order you would like the iterators to be applied.
diff --git a/docs/helloworld.md b/docs/helloworld.md
index 5666b77..b0d4381 100644
--- a/docs/helloworld.md
+++ b/docs/helloworld.md
@@ -31,8 +31,8 @@
To view the entries, use the shell (run `accumulo shell -u username -p password` to access it) to scan the table:
- username@instance> table hellotable
- username@instance hellotable> scan
+ username@instance> table examples.hellotable
+ username@instance examples.hellotable> scan
You can also use a Java class to scan the table:
diff --git a/docs/isolation.md b/docs/isolation.md
index a848af9..1a37567 100644
--- a/docs/isolation.md
+++ b/docs/isolation.md
@@ -30,7 +30,9 @@
Below, Interference Test is run without isolation enabled for 5000 iterations
and it reports problems.
- $ ./bin/runex isolation.InterferenceTest -t isotest --iterations 5000
+
+ $ accumulo shell -u <username> -p <password> -e 'createnamespace examples'
+ $ ./bin/runex isolation.InterferenceTest -t examples.isotest --iterations 50000
ERROR Columns in row 053 had multiple values [53, 4553]
ERROR Columns in row 061 had multiple values [561, 61]
ERROR Columns in row 070 had multiple values [570, 1070]
@@ -43,7 +45,7 @@
Below, Interference Test is run with isolation enabled for 5000 iterations and
it reports no problems.
- $ ./bin/runex isolation.InterferenceTest -t isotest --iterations 5000 --isolated
+ $ ./bin/runex isolation.InterferenceTest -t examples.isotest --iterations 50000 --isolated
finished
diff --git a/docs/regex.md b/docs/regex.md
index f9c4715..86c3b18 100644
--- a/docs/regex.md
+++ b/docs/regex.md
@@ -23,10 +23,11 @@
put a trivial amount of data into accumulo using the accumulo shell:
$ accumulo shell
- username@instance> createtable regex
- username@instance> insert dogrow dogcf dogcq dogvalue
- username@instance> insert catrow catcf catcq catvalue
- username@instance> quit
+ username@instance> createnamespace examples
+ username@instance> createtable examples.regex
+ username@instance examples.regex> insert dogrow dogcf dogcq dogvalue
+ username@instance examples.regex> insert catrow catcf catcq catvalue
+ username@instance examples.regex> quit
The RegexExample class sets an iterator on the scanner. This does pattern matching
against each key/value in accumulo, and only returns matching items. It will do this
@@ -34,12 +35,11 @@
The following will search for any rows in the input table that starts with "dog":
- $ ./bin/runmr mapreduce.RegexExample -t regex --rowRegex 'dog.*' --output /tmp/output
+ $ ./bin/runmr mapreduce.RegexExample -t examples.regex --rowRegex 'dog.*' --output /tmp/output
$ hdfs dfs -ls /tmp/output
Found 3 items
-rw-r--r-- 1 username supergroup 0 2013-01-10 14:11 /tmp/output/_SUCCESS
- drwxr-xr-x - username supergroup 0 2013-01-10 14:10 /tmp/output/_logs
-rw-r--r-- 1 username supergroup 51 2013-01-10 14:10 /tmp/output/part-m-00000
We can see the output of our little map-reduce job:
diff --git a/docs/reservations.md b/docs/reservations.md
index a987cfe..0b682ba 100644
--- a/docs/reservations.md
+++ b/docs/reservations.md
@@ -31,9 +31,10 @@
---------------------+--------------------------------------+-------------------------------
<instance name> | 9f8f2a97-432f-4e66-b153-861e2a1ca246 | localhost:9999
- $ /path/to/accumulo shell -u root -p secret -e "createtable ars"
+ $ /path/to/accumulo shell -u root -p secret -e "createnamespace examples"
+ $ /path/to/accumulo shell -u root -p secret -e "createtable examples.ars"
$ ./bin/runex reservations.ARS
- >connect <instance name> localhost root secret ars
+ >connect <instance name> localhost root secret examples.ars
connected
>
Commands :
@@ -59,8 +60,8 @@
Scanning the table in the Accumulo shell after running the example shows the
following:
- root@test16> table ars
- root@test16 ars> scan
+ root@test16> table examples.ars
+ root@test16 examples.ars> scan
room06:20140101 res:0001 [] mallory
room06:20140101 res:0003 [] trent
room06:20140101 res:0004 [] eve
diff --git a/docs/rgbalancer.md b/docs/rgbalancer.md
index daa1a3b..57d87eb 100644
--- a/docs/rgbalancer.md
+++ b/docs/rgbalancer.md
@@ -25,9 +25,10 @@
all of the tablets where the split point has the same two digits to be on
different tservers. This gives us four groups of tablets: 01, 02, 03, and 04.
- root@accumulo> createtable testRGB
- root@accumulo testRGB> addsplits -t testRGB 01b 01m 01r 01z 02b 02m 02r 02z 03b 03m 03r 03z 04a 04b 04c 04d 04e 04f 04g 04h 04i 04j 04k 04l 04m 04n 04o 04p
- root@accumulo testRGB> tables -l
+ root@accumulo> createnamespace examples
+ root@accumulo> createtable examples.testRGB
+ root@accumulo examples.testRGB> addsplits -t examples.testRGB 01b 01m 01r 01z 02b 02m 02r 02z 03b 03m 03r 03z 04a 04b 04c 04d 04e 04f 04g 04h 04i 04j 04k 04l 04m 04n 04o 04p
+ root@accumulo examples.testRGB> tables -l
accumulo.metadata => !0
accumulo.replication => +rep
accumulo.root => +r
@@ -36,7 +37,7 @@
After adding the splits we look at the locations in the metadata table.
- root@accumulo testRGB> scan -t accumulo.metadata -b 2; -e 2< -c loc
+ root@accumulo examples.testRGB> scan -t accumulo.metadata -b 2; -e 2< -c loc
2;01b loc:34a5f6e086b000c [] ip-10-1-2-25:9997
2;01m loc:34a5f6e086b000c [] ip-10-1-2-25:9997
2;01r loc:14a5f6e079d0011 [] ip-10-1-2-15:9997
@@ -95,13 +96,13 @@
from a tablets end row as the group id. Tablets that don't match and the
default tablet are configured to be in group 04.
- root@accumulo testRGB> config -t testRGB -s table.custom.balancer.group.regex.pattern=(\d\d).*
- root@accumulo testRGB> config -t testRGB -s table.custom.balancer.group.regex.default=04
- root@accumulo testRGB> config -t testRGB -s table.balancer=org.apache.accumulo.server.master.balancer.RegexGroupBalancer
+ root@accumulo examples.testRGB> config -t examples.testRGB -s table.custom.balancer.group.regex.pattern=(\d\d).*
+ root@accumulo examples.testRGB> config -t examples.testRGB -s table.custom.balancer.group.regex.default=04
+ root@accumulo examples.testRGB> config -t examples.testRGB -s table.balancer=org.apache.accumulo.server.master.balancer.RegexGroupBalancer
-After waiting a little bit, look at the tablet locations again and all is good.
+After waiting a bit, look at the tablet locations again and all is good.
- root@accumulo testRGB> scan -t accumulo.metadata -b 2; -e 2< -c loc
+ root@accumulo examples.testRGB> scan -t accumulo.metadata -b 2; -e 2< -c loc
2;01b loc:34a5f6e086b000a [] ip-10-1-2-18:9997
2;01m loc:34a5f6e086b000c [] ip-10-1-2-25:9997
2;01r loc:14a5f6e079d0011 [] ip-10-1-2-15:9997
diff --git a/docs/rowhash.md b/docs/rowhash.md
index 5bea7df..303c52b 100644
--- a/docs/rowhash.md
+++ b/docs/rowhash.md
@@ -23,20 +23,21 @@
put a trivial amount of data into accumulo using the accumulo shell:
$ accumulo shell
- username@instance> createtable rowhash
- username@instance> insert a-row cf cq value
- username@instance> insert b-row cf cq value
- username@instance> quit
+ username@instance> createnamespace examples
+ username@instance> createtable examples.rowhash
+ username@instance examples.rowhash> insert a-row cf cq value
+ username@instance examples.rowhash> insert b-row cf cq value
+ username@instance examples.rowhash> quit
The RowHash class will insert a hash for each row in the database if it contains a
specified colum. Here's how you run the map/reduce job
- $ ./bin/runmr mapreduce.RowHash -t rowhash --column cf:cq
+ $ ./bin/runmr mapreduce.RowHash -t examples.rowhash --column cf:cq
Now we can scan the table and see the hashes:
$ accumulo shell
- username@instance> scan -t rowhash
+ username@instance> scan -t examples.rowhash
a-row cf:cq [] value
a-row cf-HASHTYPE:cq-MD5BASE64 [] IGPBYI1uC6+AJJxC4r5YBA==
b-row cf:cq [] value
diff --git a/docs/sample.md b/docs/sample.md
index 931579c..e631f12 100644
--- a/docs/sample.md
+++ b/docs/sample.md
@@ -23,46 +23,47 @@
accessed by scanners. What data is included in the sample set is configurable.
Below, some data representing documents are inserted.
- root@instance sampex> createtable sampex
- root@instance sampex> insert 9255 doc content 'abcde'
- root@instance sampex> insert 9255 doc url file://foo.txt
- root@instance sampex> insert 8934 doc content 'accumulo scales'
- root@instance sampex> insert 8934 doc url file://accumulo_notes.txt
- root@instance sampex> insert 2317 doc content 'milk, eggs, bread, parmigiano-reggiano'
- root@instance sampex> insert 2317 doc url file://groceries/9.txt
- root@instance sampex> insert 3900 doc content 'EC2 ate my homework'
- root@instance sampex> insert 3900 doc uril file://final_project.txt
+ root@instance> createnamespace examples
+ root@instance> createtable examples.sampex
+ root@instance examples.sampex> insert 9255 doc content 'abcde'
+ root@instance examples.sampex> insert 9255 doc url file://foo.txt
+ root@instance examples.sampex> insert 8934 doc content 'accumulo scales'
+ root@instance examples.sampex> insert 8934 doc url file://accumulo_notes.txt
+ root@instance examples.sampex> insert 2317 doc content 'milk, eggs, bread, parmigiano-reggiano'
+ root@instance examples.sampex> insert 2317 doc url file://groceries/9.txt
+ root@instance examples.sampex> insert 3900 doc content 'EC2 ate my homework'
+ root@instance examples.sampex> insert 3900 doc uril file://final_project.txt
-Below the table sampex is configured to build a sample set. The configuration
+Below the table examples.sampex is configured to build a sample set. The configuration
causes Accumulo to include any row where `murmur3_32(row) % 3 ==0` in the
tables sample data.
- root@instance sampex> config -t sampex -s table.sampler.opt.hasher=murmur3_32
- root@instance sampex> config -t sampex -s table.sampler.opt.modulus=3
- root@instance sampex> config -t sampex -s table.sampler=org.apache.accumulo.core.client.sample.RowSampler
+ root@instance examples.sampex> config -t examples.sampex -s table.sampler.opt.hasher=murmur3_32
+ root@instance examples.sampex> config -t examples.sampex -s table.sampler.opt.modulus=3
+ root@instance examples.sampex> config -t examples.sampex -s table.sampler=org.apache.accumulo.core.client.sample.RowSampler
Below, attempting to scan the sample returns an error. This is because data
was inserted before the sample set was configured.
- root@instance sampex> scan --sample
+ root@instance examples.sampex> scan --sample
2015-09-09 12:21:50,643 [shell.Shell] ERROR: org.apache.accumulo.core.client.SampleNotPresentException: Table sampex(ID:2) does not have sampling configured or built
To remedy this problem, the following command will flush in memory data and
compact any files that do not contain the correct sample data.
- root@instance sampex> compact -t sampex --sf-no-sample
+ root@instance examples.sampex> compact -t examples.sampex --sf-no-sample
After the compaction, the sample scan works.
- root@instance sampex> scan --sample
+ root@instance examples.sampex> scan --sample
2317 doc:content [] milk, eggs, bread, parmigiano-reggiano
2317 doc:url [] file://groceries/9.txt
The commands below show that updates to data in the sample are seen when
scanning the sample.
- root@instance sampex> insert 2317 doc content 'milk, eggs, bread, parmigiano-reggiano, butter'
- root@instance sampex> scan --sample
+ root@instance examples.sampex> insert 2317 doc content 'milk, eggs, bread, parmigiano-reggiano, butter'
+ root@instance examples.sampex> scan --sample
2317 doc:content [] milk, eggs, bread, parmigiano-reggiano, butter
2317 doc:url [] file://groceries/9.txt
@@ -72,12 +73,12 @@
will detect this situation and fail sample scans. The commands below show this
failure and fixiing the problem with a compaction.
- root@instance sampex> config -t sampex -s table.sampler.opt.modulus=2
- root@instance sampex> scan --sample
+ root@instance examples.sampex> config -t examples.sampex -s table.sampler.opt.modulus=2
+ root@instance examples.sampex> scan --sample
2015-09-09 12:22:51,058 [shell.Shell] ERROR: org.apache.accumulo.core.client.SampleNotPresentException: Table sampex(ID:2) does not have sampling configured or built
- root@instance sampex> compact -t sampex --sf-no-sample
+ root@instance examples.sampex> compact -t examples.sampex --sf-no-sample
2015-09-09 12:23:07,242 [shell.Shell] INFO : Compaction of table sampex started for given range
- root@instance sampex> scan --sample
+ root@instance examples.sampex> scan --sample
2317 doc:content [] milk, eggs, bread, parmigiano-reggiano
2317 doc:url [] file://groceries/9.txt
3900 doc:content [] EC2 ate my homework
@@ -86,18 +87,18 @@
9255 doc:url [] file://foo.txt
The example above is replicated in a java program using the Accumulo API.
-Below is the program name and the command to run it.
+Below is the program name, and the command to run it.
./bin/runex sample.SampleExample
The commands below look under the hood to give some insight into how this
feature works. The commands determine what files the sampex table is using.
- root@instance sampex> tables -l
+ root@instance> tables -l
accumulo.metadata => !0
accumulo.replication => +rep
accumulo.root => +r
- sampex => 2
+ examples.sampex => 2
trace => 1
root@instance sampex> scan -t accumulo.metadata -c file -b 2 -e 2<
2< file:hdfs://localhost:10000/accumulo/tables/2/default_tablet/A000000s.rf [] 702,8
@@ -148,31 +149,31 @@
Shard Sampling Example
----------------------
-The [shard example][shard] shows how to index and search files using Accumulo. That
+The Shard example shows how to index and search files using Accumulo. That
example indexes documents into a table named `shard`. The indexing scheme used
in that example places the document name in the column qualifier. A useful
sample of this indexing scheme should contain all data for any document in the
sample. To accomplish this, the following commands build a sample for the
shard table based on the column qualifier.
- root@instance shard> config -t shard -s table.sampler.opt.hasher=murmur3_32
- root@instance shard> config -t shard -s table.sampler.opt.modulus=101
- root@instance shard> config -t shard -s table.sampler.opt.qualifier=true
- root@instance shard> config -t shard -s table.sampler=org.apache.accumulo.core.client.sample.RowColumnSampler
- root@instance shard> compact -t shard --sf-no-sample -w
+ root@instance examples.shard> config -t examples.shard -s table.sampler.opt.hasher=murmur3_32
+ root@instance examples.shard> config -t examples.shard -s table.sampler.opt.modulus=101
+ root@instance examples.shard> config -t examples.shard -s table.sampler.opt.qualifier=true
+ root@instance examples.shard> config -t examples.shard -s table.sampler=org.apache.accumulo.core.client.sample.RowColumnSampler
+ root@instance examples.shard> compact -t examples.shard --sf-no-sample -w
2015-07-23 15:00:09,280 [shell.Shell] INFO : Compacting table ...
2015-07-23 15:00:10,134 [shell.Shell] INFO : Compaction of table shard completed for given range
After enabling sampling, the command below counts the number of documents in
the sample containing the words `import` and `int`.
- $ ./bin/runex shard.Query --sample -t shard import int | fgrep '.java' | wc
+ $ ./bin/runex shard.Query --sample -t examples.shard import int | fgrep '.java' | wc
11 11 1246
The command below counts the total number of documents containing the words
`import` and `int`.
- $ ./bin/runex shard.Query -t shard import int | fgrep '.java' | wc
+ $ ./bin/runex shard.Query -t examples.shard import int | fgrep '.java' | wc
1085 1085 118175
The counts 11 out of 1085 total are around what would be expected for a modulus
@@ -188,4 +189,4 @@
`--sampleCutoff` option below will cause the query to return nothing if based
on the sample it appears a query would return more than 1000 documents.
- $ ./bin/runex shard.Query --sampleCutoff 1000 -t shard import int | fgrep '.java' | wc
+ $ ./bin/runex shard.Query --sampleCutoff 1000 -t examples.shard import int | fgrep '.java' | wc
diff --git a/docs/shard.md b/docs/shard.md
index dbae395..f6f6848 100644
--- a/docs/shard.md
+++ b/docs/shard.md
@@ -26,16 +26,17 @@
To run these example programs, create two tables like below.
- username@instance> createtable shard
- username@instance shard> createtable doc2term
+ username@instance> createnamespace examples
+ username@instance> createtable examples.shard
+ username@instance examples.shard> createtable examples.doc2term
After creating the tables, index some files. The following command indexes all of the java files in the Accumulo source code.
- $ find /path/to/accumulo/core -name "*.java" | xargs ./bin/runex shard.Index -t shard --partitions 30
+ $ find /path/to/accumulo/core -name "*.java" | xargs ./bin/runex shard.Index -t examples.shard --partitions 30
The following command queries the index to find all files containing 'foo' and 'bar'.
- $ ./bin/runex shard.Query -t shard foo bar
+ $ ./bin/runex shard.Query -t examples.shard foo bar
/local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/security/ColumnVisibilityTest.java
/local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/client/mock/MockConnectorTest.java
/local/username/workspace/accumulo/src/core/src/test/java/accumulo/core/security/VisibilityEvaluatorTest.java
@@ -44,12 +45,12 @@
In order to run ContinuousQuery, we need to run Reverse.java to populate doc2term.
- $ ./bin/runex shard.Reverse --shardTable shard --doc2Term doc2term
+ $ ./bin/runex shard.Reverse --shardTable examples.shard --doc2Term examples.doc2term
Below ContinuousQuery is run using 5 terms. So it selects 5 random terms from each document, then it continually
randomly selects one set of 5 terms and queries. It prints the number of matching documents and the time in seconds.
- $ ./bin/runex shard.ContinuousQuery --shardTable shard --doc2Term doc2term --terms 5
+ $ ./bin/runex shard.ContinuousQuery --shardTable examples.shard --doc2Term examples.doc2term --terms 5
[public, core, class, binarycomparable, b] 2 0.081
[wordtodelete, unindexdocument, doctablename, putdelete, insert] 1 0.041
[import, columnvisibilityinterpreterfactory, illegalstateexception, cv, columnvisibility] 1 0.049
diff --git a/docs/tabletofile.md b/docs/tabletofile.md
index 36078e6..cb9f248 100644
--- a/docs/tabletofile.md
+++ b/docs/tabletofile.md
@@ -22,26 +22,24 @@
put a trivial amount of data into accumulo using the accumulo shell:
$ accumulo shell
- username@instance> createtable input
- username@instance> insert dog cf cq dogvalue
- username@instance> insert cat cf cq catvalue
- username@instance> insert junk family qualifier junkvalue
- username@instance> quit
+ root@instance> createnamespace examples
+ root@instance> createtable examples.input
+ root@instance examples.input> insert dog cf cq dogvalue
+ root@instance examples.input> insert cat cf cq catvalue
+ root@instance examples.input> insert junk family qualifier junkvalue
+ root@instance examples.input> quit
The TableToFile class configures a map-only job to read the specified columns and
write the key/value pairs to a file in HDFS.
The following will extract the rows containing the column "cf:cq":
- $ ./bin/runmr mapreduce.TableToFile -t input --columns cf:cq --output /tmp/output
+ $ ./bin/runmr mapreduce.TableToFile -t exampmles.input --columns cf:cq --output /tmp/output
$ hadoop fs -ls /tmp/output
- -rw-r--r-- 1 username supergroup 0 2013-01-10 14:44 /tmp/output/_SUCCESS
- drwxr-xr-x - username supergroup 0 2013-01-10 14:44 /tmp/output/_logs
- drwxr-xr-x - username supergroup 0 2013-01-10 14:44 /tmp/output/_logs/history
- -rw-r--r-- 1 username supergroup 9049 2013-01-10 14:44 /tmp/output/_logs/history/job_201301081658_0011_1357847072863_username_TableToFile%5F1357847071434
- -rw-r--r-- 1 username supergroup 26172 2013-01-10 14:44 /tmp/output/_logs/history/job_201301081658_0011_conf.xml
- -rw-r--r-- 1 username supergroup 50 2013-01-10 14:44 /tmp/output/part-m-00000
+ Found 2 items
+ -rw-r--r-- 3 root supergroup 0 2021-05-04 10:32 /tmp/output/_SUCCESS
+ -rw-r--r-- 3 root supergroup 44 2021-05-04 10:32 /tmp/output/part-m-00000
We can see the output of our little map-reduce job:
diff --git a/docs/terasort.md b/docs/terasort.md
index 7e39a9e..16f2ea1 100644
--- a/docs/terasort.md
+++ b/docs/terasort.md
@@ -20,15 +20,20 @@
be sorted by storing it into accumulo. It uses data very similar to the
hadoop terasort benchmark.
+First, make sure the 'examples' namespace exists. If it already exists, the error message can be
+ignored.
+
+ $ accumulo shell -u root -p secret -e 'createnamespace examples'
+
To run this example you run it with arguments describing the amount of data:
$ ./bin/runmr mapreduce.TeraSortIngest --count 10 --minKeySize 10 --maxKeySize 10 \
- --minValueSize 78 --maxValueSize 78 --table sort --splits 10 \
+ --minValueSize 78 --maxValueSize 78 --table examples.sort --splits 10 \
After the map reduce job completes, scan the data:
$ accumulo shell
- username@instance> scan -t sort
+ username@instance> scan -t examples.sort
+l-$$OE/ZH c: 4 [] GGGGGGGGGGWWWWWWWWWWMMMMMMMMMMCCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYYYYOOOOOOOO
,C)wDw//u= c: 10 [] CCCCCCCCCCSSSSSSSSSSIIIIIIIIIIYYYYYYYYYYOOOOOOOOOOEEEEEEEEEEUUUUUUUUUUKKKKKKKK
75@~?'WdUF c: 1 [] IIIIIIIIIIYYYYYYYYYYOOOOOOOOOOEEEEEEEEEEUUUUUUUUUUKKKKKKKKKKAAAAAAAAAAQQQQQQQQ
diff --git a/docs/visibility.md b/docs/visibility.md
index daca822..b79f91c 100644
--- a/docs/visibility.md
+++ b/docs/visibility.md
@@ -18,12 +18,14 @@
## Creating a new user
+ root@instance> createnamespace examples
root@instance> createuser username
Enter new password for 'username': ********
Please confirm new password for 'username': ********
+
root@instance> user username
Enter password for user username: ********
- username@instance> createtable vistest
+ username@instance> createtable examples.vistest
06 10:48:47,931 [shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException: Error PERMISSION_DENIED - User does not have permission to perform this action
username@instance> userpermissions
System permissions:
@@ -44,8 +46,8 @@
root@instance> grant -s System.CREATE_TABLE -u username
root@instance> user username
Enter password for user username: ********
- username@instance> createtable vistest
- username@instance> userpermissions
+ username@instance> createtable examples.vistest
+ username@instance examples.vistest> userpermissions
System permissions: System.CREATE_TABLE
Namespace permissions (accumulo): Namespace.READ
@@ -53,8 +55,7 @@
Table permissions (accumulo.metadata): Table.READ
Table permissions (accumulo.replication): Table.READ
Table permissions (accumulo.root): Table.READ
- Table permissions (vistest): Table.READ, Table.WRITE, Table.BULK_IMPORT, Table.ALTER_TABLE, Table.GRANT, Table.DROP_TABLE, Table.GET_SUMMARIES
- username@instance vistest>
+ Table permissions (examples.vistest): Table.READ, Table.WRITE, Table.BULK_IMPORT, Table.ALTER_TABLE, Table.GRANT, Table.DROP_TABLE, Table.GET_SUMMARIES
## Inserting data with visibilities
@@ -63,14 +64,14 @@
ASCII character set. Parentheses are required to specify order of operations
in visibilities.
- username@instance vistest> insert row f1 q1 v1 -l A
- username@instance vistest> insert row f2 q2 v2 -l A&B
- username@instance vistest> insert row f3 q3 v3 -l apple&carrot|broccoli|spinach
+ username@instance examples.vistest> insert row f1 q1 v1 -l A
+ username@instance examples.vistest> insert row f2 q2 v2 -l A&B
+ username@instance examples.vistest> insert row f3 q3 v3 -l apple&carrot|broccoli|spinach
06 11:19:01,432 [shell.Shell] ERROR: org.apache.accumulo.core.util.BadArgumentException: cannot mix | and & near index 12
apple&carrot|broccoli|spinach
^
- username@instance vistest> insert row f3 q3 v3 -l (apple&carrot)|broccoli|spinach
- username@instance vistest>
+ username@instance examples.vistest> insert row f3 q3 v3 -l (apple&carrot)|broccoli|spinach
+ username@instance examples.vistest>
## Scanning with authorizations
@@ -79,54 +80,54 @@
are only allowed to be a subset of the user's authorizations. By default, a
user's authorizations set is empty.
- username@instance vistest> scan
- username@instance vistest> scan -s A
+ username@instance examples.vistest> scan
+ username@instance examples.vistest> scan -s A
06 11:43:14,951 [shell.Shell] ERROR: java.lang.RuntimeException: org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_AUTHORIZATIONS - The user does not have the specified authorizations assigned
- username@instance vistest>
+ username@instance examples.vistest>
## Setting authorizations for a user
- username@instance vistest> setauths -s A
+ username@instance examples.vistest> setauths -s A
06 11:53:42,056 [shell.Shell] ERROR: org.apache.accumulo.core.client.AccumuloSecurityException: Error PERMISSION_DENIED - User does not have permission to perform this action
- username@instance vistest>
+ username@instance examples.vistest>
A user cannot set authorizations unless the user has the System.ALTER_USER permission.
The root user has this permission.
- username@instance vistest> user root
+ username@instance examples.vistest> user root
Enter password for user root: ********
- root@instance vistest> setauths -s A -u username
- root@instance vistest> user username
+ root@instance examples.vistest> setauths -s A -u username
+ root@instance examples.vistest> user username
Enter password for user username: ********
- username@instance vistest> scan -s A
+ username@instance examples.vistest> scan -s A
row f1:q1 [A] v1
- username@instance vistest> scan
+ username@instance examples.vistest> scan
row f1:q1 [A] v1
- username@instance vistest>
+ username@instance examples.vistest>
The default authorizations for a scan are the user's entire set of authorizations.
- username@instance vistest> user root
+ username@instance examples.vistest> user root
Enter password for user root: ********
- root@instance vistest> setauths -s A,B,broccoli -u username
- root@instance vistest> user username
+ root@instance examples.vistest> setauths -s A,B,broccoli -u username
+ root@instance examples.vistest> user username
Enter password for user username: ********
- username@instance vistest> getauths
+ username@instance examples.vistest> getauths
A,B,broccoli
- username@instance vistest> getauths -u username
+ username@instance examples.vistest> getauths -u username
A,B,broccoli
- username@instance vistest> scan
+ username@instance examples.vistest> scan
row f1:q1 [A] v1
row f2:q2 [A&B] v2
row f3:q3 [(apple&carrot)|broccoli|spinach] v3
- username@instance vistest> scan -s B
- username@instance vistest>
+ username@instance examples.vistest> scan -s B
+ username@instance examples.vistest>
If you want, you can limit a user to only be able to insert data which they can read themselves.
First, check for any existing constraints.
- username@instance vistest> constraint -l -t vistest
+ username@instance examples.vistest> constraint -l -t examples.vistest
org.apache.accumulo.core.constraints.DefaultKeySizeConstraint=1
If existing constraints exists, take note of the values assigned to the constraints and use a
@@ -137,18 +138,18 @@
The constraint can be set with the following command:
- username@instance vistest> config -t vistest -s table.constraint.2=org.apache.accumulo.core.constraints.VisibilityConstraint
- username@instance vistest> constraint -l
- org.apache.accumulo.core.constraints.DefaultKeySizeConstraint=1
- org.apache.accumulo.core.constraints.VisibilityConstraint=2
- username@instance vistest> insert row f4 q4 v4 -l spinach
+ username@instance examples.vistest> config -t examples.vistest -s table.constraint.2=org.apache.accumulo.core.data.constraints.VisibilityConstraint
+ username@instance examples.vistest> constraint -l
+ org.apache.accumulo.core.data.constraints.DefaultKeySizeConstraint=1
+ org.apache.accumulo.core.data.constraints.VisibilityConstraint=2
+ username@instance examples.vistest> insert row f4 q4 v4 -l spinach
Constraint Failures:
ConstraintViolationSummary(constrainClass:org.apache.accumulo.core.constraints.VisibilityConstraint, violationCode:2, violationDescription:User does not have authorization on column visibility, numberOfViolatingMutations:1)
- username@instance vistest> insert row f4 q4 v4 -l spinach|broccoli
- username@instance vistest> scan
+ username@instance examples.vistest> insert row f4 q4 v4 -l spinach|broccoli
+ username@instance examples.vistest> scan
row f1:q1 [A] v1
row f2:q2 [A&B] v2
row f3:q3 [(apple&carrot)|broccoli|spinach] v3
row f4:q4 [spinach|broccoli] v4
- username@instance vistest>
+ username@instance examples.vistest>
diff --git a/docs/wordcount.md b/docs/wordcount.md
index 3e3cc52..8da6599 100644
--- a/docs/wordcount.md
+++ b/docs/wordcount.md
@@ -41,8 +41,8 @@
After the MapReduce job completes, query the Accumulo table to see word counts.
$ accumulo shell
- username@instance> table wordCount
- username@instance wordCount> scan -b the
+ username@instance> table examples.wordcount
+ username@instance examples.wordcount> scan -b the
the count:20080906 [] 75
their count:20080906 [] 2
them count:20080906 [] 1
@@ -56,14 +56,14 @@
to obtain the client properties from HDFS:
$ hdfs dfs -copyFromLocal ./conf/accumulo-client.properties /user/myuser/
- $ ./bin/runmr mapreduce.WordCount -i /wc -t wordCount2 -d /user/myuser/accumulo-client.properties
+ $ ./bin/runmr mapreduce.WordCount -i /wc -t examples.wordcount2 -d /user/myuser/accumulo-client.properties
-After the MapReduce job completes, query the `wordCount2` table. The results should
+After the MapReduce job completes, query the `wordcount2` table. The results should
be the same as before:
$ accumulo shell
- username@instance> table wordCount2
- username@instance wordCount2> scan -b the
+ username@instance> table examples.wordcount2
+ username@instance examples.wordcount2> scan -b the
the count:20080906 [] 75
their count:20080906 [] 2
...
diff --git a/spark/src/main/java/org/apache/accumulo/spark/CopyPlus5K.java b/spark/src/main/java/org/apache/accumulo/spark/CopyPlus5K.java
index 416d360..cd58889 100644
--- a/spark/src/main/java/org/apache/accumulo/spark/CopyPlus5K.java
+++ b/spark/src/main/java/org/apache/accumulo/spark/CopyPlus5K.java
@@ -41,6 +41,8 @@
public class CopyPlus5K {
+ private static final Logger log = LoggerFactory.getLogger(CopyPlus5K.class);
+
public static class AccumuloRangePartitioner extends Partitioner {
private static final long serialVersionUID = 1L;
@@ -76,8 +78,20 @@
client.tableOperations().delete(outputTable);
}
// Create tables
- client.tableOperations().create(inputTable);
- client.tableOperations().create(outputTable);
+ try {
+ client.tableOperations().create(inputTable);
+ } catch (TableExistsException e) {
+ log.error("Something went wrong. Table '{}' should have been deleted prior to creation "
+ + "attempt!", inputTable);
+ return;
+ }
+ try {
+ client.tableOperations().create(outputTable);
+ } catch (TableExistsException e) {
+ log.error("Something went wrong. Table '{}' should have been deleted prior to creation "
+ + "attempt!", inputTable);
+ return;
+ }
// Write data to input table
try (BatchWriter bw = client.createBatchWriter(inputTable)) {
diff --git a/src/main/java/org/apache/accumulo/examples/Common.java b/src/main/java/org/apache/accumulo/examples/Common.java
new file mode 100644
index 0000000..69d1043
--- /dev/null
+++ b/src/main/java/org/apache/accumulo/examples/Common.java
@@ -0,0 +1,51 @@
+package org.apache.accumulo.examples;
+
+import org.apache.accumulo.core.client.AccumuloClient;
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.NamespaceExistsException;
+import org.apache.accumulo.core.client.TableExistsException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class Common {
+
+ private static final Logger log = LoggerFactory.getLogger(Common.class);
+
+ public static final String NAMESPACE = "examples";
+
+ public static final String TABLE_EXISTS_MSG = "Table already exists. User may wish to delete "
+ + "existing table and re-run example. Table name: ";
+ public static final String NAMESPACE_EXISTS_MSG = "Namespace already exists. User can ignore "
+ + "this message and continue. Namespace: ";
+
+ /**
+ * Create a table within the supplied namespace.
+ *
+ * The incoming table name is expected to have the form "namespace.tablename". If the namespace
+ * portion of the name is blank then the table is created outside of a namespace.
+ *
+ * @param client
+ * AccumuloClient instance
+ * @param table
+ * The name of the table to be created
+ * @throws AccumuloException
+ * @throws AccumuloSecurityException
+ */
+ public static void createTableWithNamespace(final AccumuloClient client, final String table)
+ throws AccumuloException, AccumuloSecurityException {
+ String[] name = table.split("\\.");
+ if (name.length == 2 && !name[0].isEmpty()) {
+ try {
+ client.namespaceOperations().create(name[0]);
+ } catch (NamespaceExistsException e) {
+ log.info(NAMESPACE_EXISTS_MSG + name[0]);
+ }
+ }
+ try {
+ client.tableOperations().create(table);
+ } catch (TableExistsException e) {
+ log.warn(TABLE_EXISTS_MSG + table);
+ }
+ }
+}
diff --git a/src/main/java/org/apache/accumulo/examples/bloom/BloomBatchScanner.java b/src/main/java/org/apache/accumulo/examples/bloom/BloomBatchScanner.java
index 2391dda..80e9df2 100644
--- a/src/main/java/org/apache/accumulo/examples/bloom/BloomBatchScanner.java
+++ b/src/main/java/org/apache/accumulo/examples/bloom/BloomBatchScanner.java
@@ -32,19 +32,25 @@
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
import org.apache.accumulo.examples.cli.ClientOpts;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* Simple example for reading random batches of data from Accumulo.
*/
-public class BloomBatchScanner {
+public final class BloomBatchScanner {
+
+ private static final Logger log = LoggerFactory.getLogger(BloomBatchScanner.class);
+
+ private BloomBatchScanner() {}
public static void main(String[] args) throws TableNotFoundException {
ClientOpts opts = new ClientOpts();
opts.parseArgs(BloomBatchScanner.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- scan(client, "bloom_test1", 7);
- scan(client, "bloom_test2", 7);
+ scan(client, BloomCommon.BLOOM_TEST1_TABLE, 7);
+ scan(client, BloomCommon.BLOOM_TEST2_TABLE, 7);
}
}
@@ -64,22 +70,21 @@
long results = 0;
long lookups = ranges.size();
- System.out.println("Scanning " + tableName + " with seed " + seed);
+ log.info("Scanning {} with seed {}", tableName, seed);
try (BatchScanner scan = client.createBatchScanner(tableName, Authorizations.EMPTY, 20)) {
scan.setRanges(ranges);
for (Entry<Key,Value> entry : scan) {
Key key = entry.getKey();
- if (!expectedRows.containsKey(key.getRow().toString())) {
- System.out.println("Encountered unexpected key: " + key);
- } else {
+ if (expectedRows.containsKey(key.getRow().toString())) {
expectedRows.put(key.getRow().toString(), true);
+ } else {
+ log.info("Encountered unexpected key: {}", key);
}
results++;
}
}
-
long t2 = System.currentTimeMillis();
- System.out.println(String.format("Scan finished! %6.2f lookups/sec, %.2f secs, %d results",
+ log.info(String.format("Scan finished! %6.2f lookups/sec, %.2f secs, %d results",
lookups / ((t2 - t1) / 1000.0), ((t2 - t1) / 1000.0), results));
int count = 0;
@@ -89,8 +94,8 @@
}
}
if (count > 0)
- System.out.println("Did not find " + count);
+ log.info("Did not find " + count);
else
- System.out.println("All expected rows were scanned");
+ log.info("All expected rows were scanned");
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/bloom/BloomCommon.java b/src/main/java/org/apache/accumulo/examples/bloom/BloomCommon.java
new file mode 100644
index 0000000..ed72f3a
--- /dev/null
+++ b/src/main/java/org/apache/accumulo/examples/bloom/BloomCommon.java
@@ -0,0 +1,13 @@
+package org.apache.accumulo.examples.bloom;
+
+import org.apache.accumulo.examples.Common;
+
+enum BloomCommon {
+ ;
+ public static final String BLOOM_TEST1_TABLE = Common.NAMESPACE + ".bloom_test1";
+ public static final String BLOOM_TEST2_TABLE = Common.NAMESPACE + ".bloom_test2";
+ public static final String BLOOM_TEST3_TABLE = Common.NAMESPACE + ".bloom_test3";
+ public static final String BLOOM_TEST4_TABLE = Common.NAMESPACE + ".bloom_test4";
+
+ public static final String BLOOM_ENABLED_PROPERTY = "table.bloom.enabled";
+}
diff --git a/src/main/java/org/apache/accumulo/examples/bloom/BloomFilters.java b/src/main/java/org/apache/accumulo/examples/bloom/BloomFilters.java
index 6bba9ea..d2988f5 100644
--- a/src/main/java/org/apache/accumulo/examples/bloom/BloomFilters.java
+++ b/src/main/java/org/apache/accumulo/examples/bloom/BloomFilters.java
@@ -24,14 +24,20 @@
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.MutationsRejectedException;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.security.ColumnVisibility;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.apache.accumulo.examples.client.RandomBatchWriter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
-public class BloomFilters {
+public final class BloomFilters {
+
+ private static final Logger log = LoggerFactory.getLogger(BloomFilters.class);
+
+ private BloomFilters() {}
public static void main(String[] args)
throws AccumuloException, AccumuloSecurityException, TableNotFoundException {
@@ -40,36 +46,34 @@
opts.parseArgs(BloomFilters.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- System.out.println("Creating bloom_test1 and bloom_test2");
- client.tableOperations().create("bloom_test1");
- client.tableOperations().setProperty("bloom_test1", "table.compaction.major.ratio", "7");
- client.tableOperations().create("bloom_test2");
- client.tableOperations().setProperty("bloom_test2", "table.bloom.enabled", "true");
- client.tableOperations().setProperty("bloom_test2", "table.compaction.major.ratio", "7");
- } catch (TableExistsException e) {
- // ignore
- }
-
- // Write a million rows 3 times flushing files to disk separately
- System.out.println("Writing data to bloom_test1");
- writeData(client, "bloom_test1", 7);
- client.tableOperations().flush("bloom_test1", null, null, true);
- writeData(client, "bloom_test1", 8);
- client.tableOperations().flush("bloom_test1", null, null, true);
- writeData(client, "bloom_test1", 9);
- client.tableOperations().flush("bloom_test1", null, null, true);
-
- System.out.println("Writing data to bloom_test2");
- writeData(client, "bloom_test2", 7);
- client.tableOperations().flush("bloom_test2", null, null, true);
- writeData(client, "bloom_test2", 8);
- client.tableOperations().flush("bloom_test2", null, null, true);
- writeData(client, "bloom_test2", 9);
- client.tableOperations().flush("bloom_test2", null, null, true);
+ createTableAndSetCompactionRatio(client, BloomCommon.BLOOM_TEST1_TABLE);
+ createTableAndSetCompactionRatio(client, BloomCommon.BLOOM_TEST2_TABLE);
+ client.tableOperations().setProperty(BloomCommon.BLOOM_TEST2_TABLE,
+ BloomCommon.BLOOM_ENABLED_PROPERTY, "true");
+ writeAndFlushData(BloomCommon.BLOOM_TEST1_TABLE, client);
+ writeAndFlushData(BloomCommon.BLOOM_TEST2_TABLE, client);
}
}
+ private static void createTableAndSetCompactionRatio(AccumuloClient client,
+ final String tableName) throws AccumuloException, AccumuloSecurityException {
+ log.info("Creating {}", tableName);
+ Common.createTableWithNamespace(client, tableName);
+ client.tableOperations().setProperty(tableName, "table.compaction.major.ratio", "7");
+ }
+
+ // Write a million rows 3 times flushing files to disk separately
+ private static void writeAndFlushData(final String tableName, final AccumuloClient client)
+ throws TableNotFoundException, AccumuloSecurityException, AccumuloException {
+ log.info("Writing data to {}", tableName);
+ writeData(client, tableName, 7);
+ client.tableOperations().flush(tableName, null, null, true);
+ writeData(client, tableName, 8);
+ client.tableOperations().flush(tableName, null, null, true);
+ writeData(client, tableName, 9);
+ client.tableOperations().flush(tableName, null, null, true);
+ }
+
// write a million random rows
static void writeData(AccumuloClient client, String tableName, int seed)
throws TableNotFoundException, MutationsRejectedException {
diff --git a/src/main/java/org/apache/accumulo/examples/bloom/BloomFiltersNotFound.java b/src/main/java/org/apache/accumulo/examples/bloom/BloomFiltersNotFound.java
index 668ff3e..e050102 100644
--- a/src/main/java/org/apache/accumulo/examples/bloom/BloomFiltersNotFound.java
+++ b/src/main/java/org/apache/accumulo/examples/bloom/BloomFiltersNotFound.java
@@ -22,33 +22,39 @@
import org.apache.accumulo.core.client.AccumuloClient;
import org.apache.accumulo.core.client.AccumuloException;
import org.apache.accumulo.core.client.AccumuloSecurityException;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public class BloomFiltersNotFound {
+ private static final Logger log = LoggerFactory.getLogger(BloomFiltersNotFound.class);
+
public static void main(String[] args)
throws AccumuloException, AccumuloSecurityException, TableNotFoundException {
ClientOpts opts = new ClientOpts();
opts.parseArgs(BloomFiltersNotFound.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create("bloom_test3");
- client.tableOperations().create("bloom_test4");
- client.tableOperations().setProperty("bloom_test4", "table.bloom.enabled", "true");
- } catch (TableExistsException e) {
- // ignore
- }
- System.out.println("Writing data to bloom_test3 and bloom_test4 (bloom filters enabled)");
- writeData(client, "bloom_test3", 7);
- client.tableOperations().flush("bloom_test3", null, null, true);
- writeData(client, "bloom_test4", 7);
- client.tableOperations().flush("bloom_test4", null, null, true);
+ Common.createTableWithNamespace(client, BloomCommon.BLOOM_TEST3_TABLE);
+ Common.createTableWithNamespace(client, BloomCommon.BLOOM_TEST4_TABLE);
+ client.tableOperations().setProperty(BloomCommon.BLOOM_TEST4_TABLE,
+ BloomCommon.BLOOM_ENABLED_PROPERTY, "true");
- BloomBatchScanner.scan(client, "bloom_test3", 8);
- BloomBatchScanner.scan(client, "bloom_test4", 8);
+ writeAndFlush(BloomCommon.BLOOM_TEST3_TABLE, client);
+ writeAndFlush(BloomCommon.BLOOM_TEST4_TABLE, client);
+
+ BloomBatchScanner.scan(client, BloomCommon.BLOOM_TEST3_TABLE, 8);
+ BloomBatchScanner.scan(client, BloomCommon.BLOOM_TEST4_TABLE, 8);
}
}
+
+ private static void writeAndFlush(String tableName, AccumuloClient client)
+ throws TableNotFoundException, AccumuloException, AccumuloSecurityException {
+ log.info("Writing data to {} (bloom filters enabled)", tableName);
+ writeData(client, tableName, 7);
+ client.tableOperations().flush(tableName, null, null, true);
+ }
}
diff --git a/src/main/java/org/apache/accumulo/examples/client/RandomBatchScanner.java b/src/main/java/org/apache/accumulo/examples/client/RandomBatchScanner.java
index df955ae..0bd46f0 100644
--- a/src/main/java/org/apache/accumulo/examples/client/RandomBatchScanner.java
+++ b/src/main/java/org/apache/accumulo/examples/client/RandomBatchScanner.java
@@ -27,10 +27,7 @@
import org.apache.accumulo.core.client.Accumulo;
import org.apache.accumulo.core.client.AccumuloClient;
-import org.apache.accumulo.core.client.AccumuloException;
-import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchScanner;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Range;
@@ -43,22 +40,24 @@
/**
* Simple example for reading random batches of data from Accumulo.
*/
-public class RandomBatchScanner {
+public final class RandomBatchScanner {
private static final Logger log = LoggerFactory.getLogger(RandomBatchScanner.class);
- public static void main(String[] args)
- throws AccumuloException, AccumuloSecurityException, TableNotFoundException {
+ private RandomBatchScanner() {}
+
+ public static void main(String[] args) throws TableNotFoundException {
ClientOpts opts = new ClientOpts();
opts.parseArgs(RandomBatchScanner.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create("batch");
- } catch (TableExistsException e) {
- // ignore
+ if (!client.tableOperations().exists(SequentialBatchWriter.BATCH_TABLE)) {
+ log.error(
+ "Table " + SequentialBatchWriter.BATCH_TABLE + " does not exist. Nothing to scan!");
+ log.error("Try running './bin/runex client.SequentialBatchWriter' first");
+ return;
}
int totalLookups = 1000;
@@ -78,7 +77,8 @@
long lookups = 0;
log.info("Reading ranges using BatchScanner");
- try (BatchScanner scan = client.createBatchScanner("batch", Authorizations.EMPTY, 20)) {
+ try (BatchScanner scan = client.createBatchScanner(SequentialBatchWriter.BATCH_TABLE,
+ Authorizations.EMPTY, 20)) {
scan.setRanges(ranges);
for (Entry<Key,Value> entry : scan) {
Key key = entry.getKey();
@@ -93,10 +93,10 @@
new String(expectedValue.get(), UTF_8), new String(value.get(), UTF_8));
}
- if (!expectedRows.containsKey(key.getRow().toString())) {
- log.error("Encountered unexpected key: {} ", key);
- } else {
+ if (expectedRows.containsKey(key.getRow().toString())) {
expectedRows.put(key.getRow().toString(), true);
+ } else {
+ log.error("Encountered unexpected key: {} ", key);
}
lookups++;
@@ -105,7 +105,6 @@
}
}
}
-
long t2 = System.currentTimeMillis();
log.info(String.format("Scan finished! %6.2f lookups/sec, %.2f secs, %d results",
lookups / ((t2 - t1) / 1000.0), ((t2 - t1) / 1000.0), lookups));
diff --git a/src/main/java/org/apache/accumulo/examples/client/ReadWriteExample.java b/src/main/java/org/apache/accumulo/examples/client/ReadWriteExample.java
index 79c5e0e..8b3696d 100644
--- a/src/main/java/org/apache/accumulo/examples/client/ReadWriteExample.java
+++ b/src/main/java/org/apache/accumulo/examples/client/ReadWriteExample.java
@@ -20,14 +20,16 @@
import org.apache.accumulo.core.client.Accumulo;
import org.apache.accumulo.core.client.AccumuloClient;
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
-import org.apache.accumulo.core.client.NamespaceExistsException;
import org.apache.accumulo.core.client.Scanner;
-import org.apache.accumulo.core.client.TableExistsException;
+import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -35,44 +37,45 @@
public class ReadWriteExample {
private static final Logger log = LoggerFactory.getLogger(ReadWriteExample.class);
- private static final String namespace = "examples";
- private static final String table = namespace + ".readwrite";
- public static void main(String[] args) throws Exception {
+ private static final String READWRITE_TABLE = Common.NAMESPACE + ".readwrite";
+
+ private ReadWriteExample() {}
+
+ public static void main(String[] args) throws AccumuloSecurityException, AccumuloException {
ClientOpts opts = new ClientOpts();
opts.parseArgs(ReadWriteExample.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
-
- try {
- client.namespaceOperations().create(namespace);
- } catch (NamespaceExistsException e) {
- // ignore
- }
- try {
- client.tableOperations().create(table);
- } catch (TableExistsException e) {
- // ignore
- }
-
+ Common.createTableWithNamespace(client, READWRITE_TABLE);
// write data
- try (BatchWriter writer = client.createBatchWriter(table)) {
+ try (BatchWriter writer = client.createBatchWriter(READWRITE_TABLE)) {
for (int i = 0; i < 10; i++) {
Mutation m = new Mutation("hello" + i);
m.put("cf", "cq", new Value("world" + i));
writer.addMutation(m);
}
+ } catch (TableNotFoundException e) {
+ log.error("Could not find table {}: {}", e.getTableName(), e.getMessage());
+ System.exit(1);
}
// read data
- try (Scanner scanner = client.createScanner(table, Authorizations.EMPTY)) {
+ try (Scanner scanner = client.createScanner(READWRITE_TABLE, Authorizations.EMPTY)) {
for (Entry<Key,Value> entry : scanner) {
- log.info(entry.getKey().toString() + " -> " + entry.getValue().toString());
+ log.info("{} -> {}", entry.getKey().toString(), entry.getValue().toString());
}
+ } catch (TableNotFoundException e) {
+ log.error("Could not find table {}: {}", e.getTableName(), e.getMessage());
+ System.exit(1);
}
// delete table
- client.tableOperations().delete(table);
+ try {
+ client.tableOperations().delete(READWRITE_TABLE);
+ } catch (TableNotFoundException e) {
+ log.error("Unable to delete table '{}': {}", e.getTableName(), e.getMessage());
+ }
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/client/RowOperations.java b/src/main/java/org/apache/accumulo/examples/client/RowOperations.java
index 000bee1..d97e10e 100644
--- a/src/main/java/org/apache/accumulo/examples/client/RowOperations.java
+++ b/src/main/java/org/apache/accumulo/examples/client/RowOperations.java
@@ -24,15 +24,14 @@
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.MutationsRejectedException;
-import org.apache.accumulo.core.client.NamespaceExistsException;
import org.apache.accumulo.core.client.Scanner;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Range;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -40,25 +39,27 @@
/**
* A demonstration of reading entire rows and deleting entire rows.
*/
-public class RowOperations {
+public final class RowOperations {
private static final Logger log = LoggerFactory.getLogger(RowOperations.class);
- private static final String namespace = "examples";
- private static final String table = namespace + ".rowops";
+
+ static final String ROWOPS_TABLE = Common.NAMESPACE + ".rowops";
+
+ private RowOperations() {}
private static void printAll(AccumuloClient client) throws TableNotFoundException {
- try (Scanner scanner = client.createScanner(table, Authorizations.EMPTY)) {
+ try (Scanner scanner = client.createScanner(ROWOPS_TABLE, Authorizations.EMPTY)) {
for (Entry<Key,Value> entry : scanner) {
- log.info("Key: " + entry.getKey().toString() + " Value: " + entry.getValue().toString());
+ log.info("Key: {} Value: {}", entry.getKey().toString(), entry.getValue().toString());
}
}
}
private static void printRow(String row, AccumuloClient client) throws TableNotFoundException {
- try (Scanner scanner = client.createScanner(table, Authorizations.EMPTY)) {
+ try (Scanner scanner = client.createScanner(ROWOPS_TABLE, Authorizations.EMPTY)) {
scanner.setRange(Range.exact(row));
for (Entry<Key,Value> entry : scanner) {
- log.info("Key: " + entry.getKey().toString() + " Value: " + entry.getValue().toString());
+ log.info("Key: {} Value: {}", entry.getKey().toString(), entry.getValue().toString());
}
}
}
@@ -66,7 +67,7 @@
private static void deleteRow(String row, AccumuloClient client, BatchWriter bw)
throws MutationsRejectedException, TableNotFoundException {
Mutation mut = new Mutation(row);
- try (Scanner scanner = client.createScanner(table, Authorizations.EMPTY)) {
+ try (Scanner scanner = client.createScanner(ROWOPS_TABLE, Authorizations.EMPTY)) {
scanner.setRange(Range.exact(row));
for (Entry<Key,Value> entry : scanner) {
mut.putDelete(entry.getKey().getColumnFamily(), entry.getKey().getColumnQualifier());
@@ -82,16 +83,7 @@
opts.parseArgs(RowOperations.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.namespaceOperations().create(namespace);
- } catch (NamespaceExistsException e) {
- // ignore
- }
- try {
- client.tableOperations().create(table);
- } catch (TableExistsException e) {
- // ignore
- }
+ Common.createTableWithNamespace(client, ROWOPS_TABLE);
// lets create 3 rows of information
Mutation mut1 = new Mutation("row1");
@@ -111,7 +103,7 @@
mut3.put("col", "3", "v3");
// Now we'll make a Batch Writer
- try (BatchWriter bw = client.createBatchWriter(table)) {
+ try (BatchWriter bw = client.createBatchWriter(ROWOPS_TABLE)) {
// And add the mutations
bw.addMutation(mut1);
@@ -138,7 +130,7 @@
log.info("This is just row3");
printAll(client);
- client.tableOperations().delete(table);
+ client.tableOperations().delete(ROWOPS_TABLE);
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/client/SequentialBatchWriter.java b/src/main/java/org/apache/accumulo/examples/client/SequentialBatchWriter.java
index ea8ca2d..d8a575b 100644
--- a/src/main/java/org/apache/accumulo/examples/client/SequentialBatchWriter.java
+++ b/src/main/java/org/apache/accumulo/examples/client/SequentialBatchWriter.java
@@ -23,10 +23,10 @@
import org.apache.accumulo.core.client.AccumuloException;
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -36,13 +36,17 @@
/**
* Simple example for writing random data in sequential order to Accumulo.
*/
-public class SequentialBatchWriter {
+public final class SequentialBatchWriter {
private static final Logger log = LoggerFactory.getLogger(SequentialBatchWriter.class);
+ static final String BATCH_TABLE = Common.NAMESPACE + ".batch";
+
+ private SequentialBatchWriter() {}
+
public static Value createValue(long rowId, int size) {
Random r = new Random(rowId);
- byte value[] = new byte[size];
+ byte[] value = new byte[size];
r.nextBytes(value);
@@ -56,7 +60,7 @@
static class Opts extends ClientOpts {
@Parameter(names = {"-t"}, description = "table to use")
- public String tableName = "batch";
+ public String tableName = BATCH_TABLE;
@Parameter(names = {"--start"}, description = "starting row")
public Integer start = 0;
@@ -79,12 +83,7 @@
opts.parseArgs(SequentialBatchWriter.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create(opts.tableName);
- } catch (TableExistsException e) {
- // ignore
- }
-
+ Common.createTableWithNamespace(client, opts.tableName);
try (BatchWriter bw = client.createBatchWriter(opts.tableName)) {
for (int i = 0; i < opts.num; i++) {
int row = i + opts.start;
diff --git a/src/main/java/org/apache/accumulo/examples/client/TracingExample.java b/src/main/java/org/apache/accumulo/examples/client/TracingExample.java
index 899008c..551b43c 100644
--- a/src/main/java/org/apache/accumulo/examples/client/TracingExample.java
+++ b/src/main/java/org/apache/accumulo/examples/client/TracingExample.java
@@ -33,6 +33,7 @@
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.Authorizations;
// import org.apache.accumulo.core.trace.DistributedTrace;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOnDefaultTable;
import org.apache.accumulo.examples.cli.ScannerOpts;
import org.apache.htrace.Sampler;
@@ -81,7 +82,7 @@
AccumuloSecurityException, TableExistsException {
if (opts.createtable) {
- client.tableOperations().create(opts.getTableName());
+ Common.createTableWithNamespace(client, opts.getTableName());
}
if (opts.createEntries) {
diff --git a/src/main/java/org/apache/accumulo/examples/combiner/StatsCombiner.java b/src/main/java/org/apache/accumulo/examples/combiner/StatsCombiner.java
index 2794c22..14b24ca 100644
--- a/src/main/java/org/apache/accumulo/examples/combiner/StatsCombiner.java
+++ b/src/main/java/org/apache/accumulo/examples/combiner/StatsCombiner.java
@@ -49,7 +49,7 @@
long count = 0;
while (iter.hasNext()) {
- String stats[] = iter.next().toString().split(",");
+ String[] stats = iter.next().toString().split(",");
if (stats.length == 1) {
long val = Long.parseLong(stats[0], radix);
diff --git a/src/main/java/org/apache/accumulo/examples/constraints/AlphaNumKeyConstraint.java b/src/main/java/org/apache/accumulo/examples/constraints/AlphaNumKeyConstraint.java
index 0ad791e..321031c 100644
--- a/src/main/java/org/apache/accumulo/examples/constraints/AlphaNumKeyConstraint.java
+++ b/src/main/java/org/apache/accumulo/examples/constraints/AlphaNumKeyConstraint.java
@@ -28,19 +28,23 @@
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.MutationsRejectedException;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
-import org.apache.accumulo.core.constraints.Constraint;
import org.apache.accumulo.core.data.ColumnUpdate;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.data.constraints.Constraint;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* This class is an accumulo constraint that ensures all fields of a key are alpha numeric.
*/
public class AlphaNumKeyConstraint implements Constraint {
+ private static final Logger log = LoggerFactory.getLogger(AlphaNumKeyConstraint.class);
+
static final short NON_ALPHA_NUM_ROW = 1;
static final short NON_ALPHA_NUM_COLF = 2;
static final short NON_ALPHA_NUM_COLQ = 3;
@@ -49,23 +53,20 @@
static final String COLF_VIOLATION_MESSAGE = "Column family was not alpha numeric";
static final String COLQ_VIOLATION_MESSAGE = "Column qualifier was not alpha numeric";
- private boolean isAlphaNum(byte bytes[]) {
+ private boolean isNotAlphaNum(byte[] bytes) {
for (byte b : bytes) {
boolean ok = ((b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9'));
if (!ok)
- return false;
+ return true;
}
-
- return true;
+ return false;
}
private Set<Short> addViolation(Set<Short> violations, short violation) {
if (violations == null) {
violations = new LinkedHashSet<>();
- violations.add(violation);
- } else if (!violations.contains(violation)) {
- violations.add(violation);
}
+ violations.add(violation);
return violations;
}
@@ -73,15 +74,15 @@
public List<Short> check(Environment env, Mutation mutation) {
Set<Short> violations = null;
- if (!isAlphaNum(mutation.getRow()))
+ if (isNotAlphaNum(mutation.getRow()))
violations = addViolation(violations, NON_ALPHA_NUM_ROW);
Collection<ColumnUpdate> updates = mutation.getUpdates();
for (ColumnUpdate columnUpdate : updates) {
- if (!isAlphaNum(columnUpdate.getColumnFamily()))
+ if (isNotAlphaNum(columnUpdate.getColumnFamily()))
violations = addViolation(violations, NON_ALPHA_NUM_COLF);
- if (!isAlphaNum(columnUpdate.getColumnQualifier()))
+ if (isNotAlphaNum(columnUpdate.getColumnQualifier()))
violations = addViolation(violations, NON_ALPHA_NUM_COLQ);
}
@@ -109,30 +110,26 @@
opts.parseArgs(AlphaNumKeyConstraint.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create("testConstraints");
- } catch (TableExistsException e) {
- // ignore
- }
+ Common.createTableWithNamespace(client, ConstraintsCommon.CONSTRAINTS_TABLE);
- /**
+ /*
* Add the {@link AlphaNumKeyConstraint} to the table. Be sure to use the fully qualified
* class name.
*/
- int num = client.tableOperations().addConstraint("testConstraints",
+ int num = client.tableOperations().addConstraint(ConstraintsCommon.CONSTRAINTS_TABLE,
"org.apache.accumulo.examples.constraints.AlphaNumKeyConstraint");
- System.out.println("Attempting to write non alpha numeric data to testConstraints");
- try (BatchWriter bw = client.createBatchWriter("testConstraints")) {
+ log.info("Attempting to write non alpha numeric data to testConstraints");
+ try (BatchWriter bw = client.createBatchWriter(ConstraintsCommon.CONSTRAINTS_TABLE)) {
Mutation m = new Mutation("r1--$$@@%%");
m.put("cf1", "cq1", new Value(("value1").getBytes()));
bw.addMutation(m);
} catch (MutationsRejectedException e) {
- e.getConstraintViolationSummaries().forEach(violationSummary -> System.out
- .println("Constraint violated: " + violationSummary.constrainClass));
+ e.getConstraintViolationSummaries().forEach(violationSummary -> log
+ .error(ConstraintsCommon.CONSTRAINT_VIOLATED_MSG, violationSummary.constrainClass));
}
-
- client.tableOperations().removeConstraint("testConstraints", num);
+ client.tableOperations().removeConstraint(ConstraintsCommon.CONSTRAINTS_TABLE, num);
}
}
+
}
diff --git a/src/main/java/org/apache/accumulo/examples/constraints/ConstraintsCommon.java b/src/main/java/org/apache/accumulo/examples/constraints/ConstraintsCommon.java
new file mode 100644
index 0000000..1ff579d
--- /dev/null
+++ b/src/main/java/org/apache/accumulo/examples/constraints/ConstraintsCommon.java
@@ -0,0 +1,9 @@
+package org.apache.accumulo.examples.constraints;
+
+import org.apache.accumulo.examples.Common;
+
+public enum ConstraintsCommon {
+ ;
+ public static final String CONSTRAINTS_TABLE = Common.NAMESPACE + ".testConstraints";
+ public static final String CONSTRAINT_VIOLATED_MSG = "Constraint violated: {}";
+}
diff --git a/src/main/java/org/apache/accumulo/examples/constraints/MaxMutationSize.java b/src/main/java/org/apache/accumulo/examples/constraints/MaxMutationSize.java
index c98e0d7..81e4694 100644
--- a/src/main/java/org/apache/accumulo/examples/constraints/MaxMutationSize.java
+++ b/src/main/java/org/apache/accumulo/examples/constraints/MaxMutationSize.java
@@ -25,20 +25,25 @@
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.MutationsRejectedException;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
-import org.apache.accumulo.core.constraints.Constraint;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.data.constraints.Constraint;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* Ensure that mutations are a reasonable size: we must be able to fit several in memory at a time.
*/
public class MaxMutationSize implements Constraint {
+
+ private static final Logger log = LoggerFactory.getLogger(MaxMutationSize.class);
+
static final long MAX_SIZE = Runtime.getRuntime().maxMemory() >> 8;
static final List<Short> empty = Collections.emptyList();
- static final List<Short> violations = Collections.singletonList(Short.valueOf((short) 0));
+ static final List<Short> violations = Collections.singletonList((short) 0);
@Override
public String getViolationDescription(short violationCode) {
@@ -58,31 +63,26 @@
opts.parseArgs(MaxMutationSize.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create("testConstraints");
- } catch (TableExistsException e) {
- // ignore
- }
+ Common.createTableWithNamespace(client, ConstraintsCommon.CONSTRAINTS_TABLE);
- /**
+ /*
* Add the {@link MaxMutationSize} constraint to the table. Be sure to use the fully qualified
* class name
*/
- int num = client.tableOperations().addConstraint("testConstraints",
+ int num = client.tableOperations().addConstraint(ConstraintsCommon.CONSTRAINTS_TABLE,
"org.apache.accumulo.examples.constraints.MaxMutationSize");
- System.out.println("Attempting to write a lot of mutations to testConstraints");
- try (BatchWriter bw = client.createBatchWriter("testConstraints")) {
+ log.info("Attempting to write a lot of mutations to testConstraints");
+ try (BatchWriter bw = client.createBatchWriter(ConstraintsCommon.CONSTRAINTS_TABLE)) {
Mutation m = new Mutation("r1");
for (int i = 0; i < 1_000_000; i++)
m.put("cf" + i % 5000, "cq" + i, new Value(("value" + i).getBytes()));
bw.addMutation(m);
} catch (MutationsRejectedException e) {
e.getConstraintViolationSummaries()
- .forEach(m -> System.out.println("Constraint violated: " + m.constrainClass));
+ .forEach(m -> log.error(ConstraintsCommon.CONSTRAINT_VIOLATED_MSG, m.constrainClass));
}
-
- client.tableOperations().removeConstraint("testConstraints", num);
+ client.tableOperations().removeConstraint(ConstraintsCommon.CONSTRAINTS_TABLE, num);
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/constraints/NumericValueConstraint.java b/src/main/java/org/apache/accumulo/examples/constraints/NumericValueConstraint.java
index ca5a4d3..fe9d121 100644
--- a/src/main/java/org/apache/accumulo/examples/constraints/NumericValueConstraint.java
+++ b/src/main/java/org/apache/accumulo/examples/constraints/NumericValueConstraint.java
@@ -16,9 +16,7 @@
*/
package org.apache.accumulo.examples.constraints;
-import java.util.Arrays;
import java.util.Collection;
-import java.util.Collections;
import java.util.List;
import org.apache.accumulo.core.client.Accumulo;
@@ -27,32 +25,34 @@
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.MutationsRejectedException;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
-import org.apache.accumulo.core.constraints.Constraint;
import org.apache.accumulo.core.data.ColumnUpdate;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.data.constraints.Constraint;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* This class is an accumulo constraint that ensures values are numeric strings.
*/
public class NumericValueConstraint implements Constraint {
+ private static final Logger log = LoggerFactory.getLogger(NumericValueConstraint.class);
+
static final short NON_NUMERIC_VALUE = 1;
static final String VIOLATION_MESSAGE = "Value is not numeric";
- private static final List<Short> VIOLATION_LIST = Collections
- .unmodifiableList(Arrays.asList(NON_NUMERIC_VALUE));
+ private static final List<Short> VIOLATION_LIST = List.of(NON_NUMERIC_VALUE);
- private boolean isNumeric(byte bytes[]) {
+ private boolean isNumeric(byte[] bytes) {
for (byte b : bytes) {
boolean ok = (b >= '0' && b <= '9');
if (!ok)
return false;
}
-
return true;
}
@@ -64,18 +64,14 @@
if (!isNumeric(columnUpdate.getValue()))
return VIOLATION_LIST;
}
-
return null;
}
@Override
public String getViolationDescription(short violationCode) {
-
- switch (violationCode) {
- case NON_NUMERIC_VALUE:
- return "Value is not numeric";
+ if (violationCode == NON_NUMERIC_VALUE) {
+ return VIOLATION_MESSAGE;
}
-
return null;
}
@@ -85,30 +81,25 @@
opts.parseArgs(NumericValueConstraint.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create("testConstraints");
- } catch (TableExistsException e) {
- // ignore
- }
+ Common.createTableWithNamespace(client, ConstraintsCommon.CONSTRAINTS_TABLE);
- /**
+ /*
* Add the {@link NumericValueConstraint} constraint to the table. Be sure to use the fully
* qualified class name
*/
- int num = client.tableOperations().addConstraint("testConstraints",
+ int num = client.tableOperations().addConstraint(ConstraintsCommon.CONSTRAINTS_TABLE,
"org.apache.accumulo.examples.constraints.NumericValueConstraint");
- System.out.println("Attempting to write non numeric data to testConstraints");
- try (BatchWriter bw = client.createBatchWriter("testConstraints")) {
+ log.info("Attempting to write non-numeric data to testConstraints");
+ try (BatchWriter bw = client.createBatchWriter(ConstraintsCommon.CONSTRAINTS_TABLE)) {
Mutation m = new Mutation("r1");
m.put("cf1", "cq1", new Value(("value1--$$@@%%").getBytes()));
bw.addMutation(m);
} catch (MutationsRejectedException e) {
e.getConstraintViolationSummaries()
- .forEach(m -> System.out.println("Constraint violated: " + m.constrainClass));
+ .forEach(m -> log.error(ConstraintsCommon.CONSTRAINT_VIOLATED_MSG, m.constrainClass));
}
-
- client.tableOperations().removeConstraint("testConstraints", num);
+ client.tableOperations().removeConstraint(ConstraintsCommon.CONSTRAINTS_TABLE, num);
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/dirlist/Ingest.java b/src/main/java/org/apache/accumulo/examples/dirlist/Ingest.java
index 24eab3b..d5f3b35 100644
--- a/src/main/java/org/apache/accumulo/examples/dirlist/Ingest.java
+++ b/src/main/java/org/apache/accumulo/examples/dirlist/Ingest.java
@@ -29,11 +29,14 @@
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.iterators.LongCombiner;
import org.apache.accumulo.core.security.ColumnVisibility;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.BatchWriterOpts;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.apache.accumulo.examples.filedata.ChunkCombiner;
import org.apache.accumulo.examples.filedata.FileDataIngest;
import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import com.beust.jcommander.Parameter;
@@ -42,7 +45,14 @@
* into one Accumulo table, indexes the file names in a separate table, and the file data into a
* third table.
*/
-public class Ingest {
+public final class Ingest {
+
+ private static final Logger log = LoggerFactory.getLogger(Ingest.class);
+
+ static final String DIR_TABLE = Common.NAMESPACE + ".dirTable";
+ static final String INDEX_TABLE = Common.NAMESPACE + ".indexTable";
+ static final String DATA_TABLE = Common.NAMESPACE + ".dataTable";
+
static final Value nullValue = new Value(new byte[0]);
public static final String LENGTH_CQ = "length";
public static final String HIDDEN_CQ = "hidden";
@@ -51,6 +61,8 @@
public static final String HASH_CQ = "md5";
public static final Encoder<Long> encoder = LongCombiner.FIXED_LEN_ENCODER;
+ private Ingest() {}
+
public static Mutation buildMutation(ColumnVisibility cv, String path, boolean isDir,
boolean isHidden, boolean canExec, long length, long lastmod, String hash) {
if (path.equals("/"))
@@ -73,13 +85,13 @@
private static void ingest(File src, ColumnVisibility cv, BatchWriter dirBW, BatchWriter indexBW,
FileDataIngest fdi, BatchWriter data) throws Exception {
// build main table entry
- String path = null;
+ String path;
try {
path = src.getCanonicalPath();
} catch (IOException e) {
path = src.getAbsolutePath();
}
- System.out.println(path);
+ log.info(path);
String hash = null;
if (!src.isDirectory()) {
@@ -126,11 +138,11 @@
static class Opts extends ClientOpts {
@Parameter(names = "--dirTable", description = "a table to hold the directory information")
- String nameTable = "dirTable";
+ String dirTable = DIR_TABLE;
@Parameter(names = "--indexTable", description = "an index over the ingested data")
- String indexTable = "indexTable";
+ String indexTable = INDEX_TABLE;
@Parameter(names = "--dataTable", description = "the file data, chunked into parts")
- String dataTable = "dataTable";
+ String dataTable = DATA_TABLE;
@Parameter(names = "--vis", description = "the visibility to mark the data",
converter = VisibilityConverter.class)
ColumnVisibility visibility = new ColumnVisibility();
@@ -146,17 +158,13 @@
opts.parseArgs(Ingest.class.getName(), args, bwOpts);
try (AccumuloClient client = opts.createAccumuloClient()) {
- if (!client.tableOperations().exists(opts.nameTable))
- client.tableOperations().create(opts.nameTable);
- if (!client.tableOperations().exists(opts.indexTable))
- client.tableOperations().create(opts.indexTable);
- if (!client.tableOperations().exists(opts.dataTable)) {
- client.tableOperations().create(opts.dataTable);
- client.tableOperations().attachIterator(opts.dataTable,
- new IteratorSetting(1, ChunkCombiner.class));
- }
+ Common.createTableWithNamespace(client, opts.dirTable);
+ Common.createTableWithNamespace(client, opts.indexTable);
+ Common.createTableWithNamespace(client, opts.dataTable);
+ client.tableOperations().attachIterator(opts.dataTable,
+ new IteratorSetting(1, ChunkCombiner.class));
- BatchWriter dirBW = client.createBatchWriter(opts.nameTable, bwOpts.getBatchWriterConfig());
+ BatchWriter dirBW = client.createBatchWriter(opts.dirTable, bwOpts.getBatchWriterConfig());
BatchWriter indexBW = client.createBatchWriter(opts.indexTable,
bwOpts.getBatchWriterConfig());
BatchWriter dataBW = client.createBatchWriter(opts.dataTable, bwOpts.getBatchWriterConfig());
@@ -165,8 +173,8 @@
recurse(new File(dir), opts.visibility, dirBW, indexBW, fdi, dataBW);
// fill in parent directory info
- int slashIndex = -1;
- while ((slashIndex = dir.lastIndexOf("/")) > 0) {
+ int slashIndex;
+ while ((slashIndex = dir.lastIndexOf('/')) > 0) {
dir = dir.substring(0, slashIndex);
ingest(new File(dir), opts.visibility, dirBW, indexBW, fdi, dataBW);
}
diff --git a/src/main/java/org/apache/accumulo/examples/filedata/CharacterHistogram.java b/src/main/java/org/apache/accumulo/examples/filedata/CharacterHistogram.java
index 5928786..e83bed2 100644
--- a/src/main/java/org/apache/accumulo/examples/filedata/CharacterHistogram.java
+++ b/src/main/java/org/apache/accumulo/examples/filedata/CharacterHistogram.java
@@ -23,7 +23,6 @@
import java.util.Map.Entry;
import java.util.Properties;
-import org.apache.accumulo.core.client.Accumulo;
import org.apache.accumulo.core.client.security.tokens.PasswordToken;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
diff --git a/src/main/java/org/apache/accumulo/examples/filedata/FileDataIngest.java b/src/main/java/org/apache/accumulo/examples/filedata/FileDataIngest.java
index 5095ab6..d649ac8 100644
--- a/src/main/java/org/apache/accumulo/examples/filedata/FileDataIngest.java
+++ b/src/main/java/org/apache/accumulo/examples/filedata/FileDataIngest.java
@@ -33,9 +33,12 @@
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
import org.apache.accumulo.core.security.ColumnVisibility;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.BatchWriterOpts;
import org.apache.accumulo.examples.cli.ClientOnRequiredTable;
import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import com.beust.jcommander.Parameter;
@@ -43,6 +46,9 @@
* Takes a list of files and archives them into Accumulo keyed on hashes of the files.
*/
public class FileDataIngest {
+
+ private static final Logger log = LoggerFactory.getLogger(FileDataIngest.class);
+
public static final Text CHUNK_CF = new Text("~chunk");
public static final Text REFS_CF = new Text("refs");
public static final String REFS_ORIG_FILE = "name";
@@ -52,6 +58,9 @@
public static final ByteSequence REFS_CF_BS = new ArrayByteSequence(REFS_CF.getBytes(), 0,
REFS_CF.getLength());
+ public static final String TABLE_EXISTS_MSG = "Table already exists. User may wish to delete existing "
+ + "table and re-run example. Table name: ";
+
int chunkSize;
byte[] chunkSizeBytes;
byte[] buf;
@@ -194,11 +203,10 @@
opts.parseArgs(FileDataIngest.class.getName(), args, bwOpts);
try (AccumuloClient client = opts.createAccumuloClient()) {
- if (!client.tableOperations().exists(opts.getTableName())) {
- client.tableOperations().create(opts.getTableName());
- client.tableOperations().attachIterator(opts.getTableName(),
- new IteratorSetting(1, ChunkCombiner.class));
- }
+ Common.createTableWithNamespace(client, opts.getTableName());
+ client.tableOperations().attachIterator(opts.getTableName(),
+ new IteratorSetting(1, ChunkCombiner.class));
+
try (BatchWriter bw = client.createBatchWriter(opts.getTableName(),
bwOpts.getBatchWriterConfig())) {
FileDataIngest fdi = new FileDataIngest(opts.chunkSize, opts.visibility);
diff --git a/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java b/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java
index 3ee8864..a7884be 100644
--- a/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java
+++ b/src/main/java/org/apache/accumulo/examples/helloworld/Insert.java
@@ -21,10 +21,10 @@
import org.apache.accumulo.core.client.AccumuloException;
import org.apache.accumulo.core.client.AccumuloSecurityException;
import org.apache.accumulo.core.client.BatchWriter;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.TableNotFoundException;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -36,19 +36,17 @@
private static final Logger log = LoggerFactory.getLogger(Insert.class);
+ static final String HELLO_TABLE = Common.NAMESPACE + ".hellotable";
+
public static void main(String[] args)
throws AccumuloException, AccumuloSecurityException, TableNotFoundException {
ClientOpts opts = new ClientOpts();
opts.parseArgs(Insert.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create("hellotable");
- } catch (TableExistsException e) {
- // ignore
- }
+ Common.createTableWithNamespace(client, HELLO_TABLE);
- try (BatchWriter bw = client.createBatchWriter("hellotable")) {
+ try (BatchWriter bw = client.createBatchWriter(HELLO_TABLE)) {
log.trace("writing ...");
for (int i = 0; i < 10000; i++) {
Mutation m = new Mutation(String.format("row_%d", i));
diff --git a/src/main/java/org/apache/accumulo/examples/helloworld/Read.java b/src/main/java/org/apache/accumulo/examples/helloworld/Read.java
index 41400d0..e9e82b7 100644
--- a/src/main/java/org/apache/accumulo/examples/helloworld/Read.java
+++ b/src/main/java/org/apache/accumulo/examples/helloworld/Read.java
@@ -42,13 +42,14 @@
opts.parseArgs(Read.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build();
- Scanner scan = client.createScanner("hellotable", Authorizations.EMPTY)) {
+ Scanner scan = client.createScanner(Insert.HELLO_TABLE, Authorizations.EMPTY)) {
scan.setRange(new Range(new Key("row_0"), new Key("row_1002")));
for (Entry<Key,Value> e : scan) {
Key key = e.getKey();
log.trace(key.getRow() + " " + key.getColumnFamily() + " " + key.getColumnQualifier() + " "
+ e.getValue());
}
+ log.info("Scan complete");
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/isolation/InterferenceTest.java b/src/main/java/org/apache/accumulo/examples/isolation/InterferenceTest.java
index 6de2929..61cea3b 100644
--- a/src/main/java/org/apache/accumulo/examples/isolation/InterferenceTest.java
+++ b/src/main/java/org/apache/accumulo/examples/isolation/InterferenceTest.java
@@ -28,6 +28,7 @@
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.BatchWriterOpts;
import org.apache.accumulo.examples.cli.ClientOnRequiredTable;
import org.apache.hadoop.io.Text;
@@ -46,13 +47,19 @@
*
*/
-public class InterferenceTest {
+public final class InterferenceTest {
private static final int NUM_ROWS = 500;
private static final int NUM_COLUMNS = 113; // scanner batches 1000 by default, so make num
// columns not a multiple of 10
+ private static final String ERROR_MISSING_COLS = "ERROR Did not see {} columns in row {}";
+ private static final String ERROR_MULTIPLE_VALS = "ERROR Columns in row {} had multiple values "
+ + "{}";
+
private static final Logger log = LoggerFactory.getLogger(InterferenceTest.class);
+ private InterferenceTest() {}
+
static class Writer implements Runnable {
private final BatchWriter bw;
@@ -95,7 +102,7 @@
static class Reader implements Runnable {
- private Scanner scanner;
+ private final Scanner scanner;
volatile boolean stop = false;
Reader(Scanner scanner) {
@@ -118,10 +125,10 @@
if (!row.equals(entry.getKey().getRowData())) {
if (count != NUM_COLUMNS)
- System.err.println("ERROR Did not see " + NUM_COLUMNS + " columns in row " + row);
+ log.error(ERROR_MISSING_COLS, NUM_COLUMNS, row);
if (values.size() > 1)
- System.err.println("ERROR Columns in row " + row + " had multiple values " + values);
+ log.error(ERROR_MULTIPLE_VALS, row, values);
row = entry.getKey().getRowData();
count = 0;
@@ -134,10 +141,10 @@
}
if (count > 0 && count != NUM_COLUMNS)
- System.err.println("ERROR Did not see " + NUM_COLUMNS + " columns in row " + row);
+ log.error(ERROR_MISSING_COLS, NUM_COLUMNS, row);
if (values.size() > 1)
- System.err.println("ERROR Columns in row " + row + " had multiple values " + values);
+ log.error(ERROR_MULTIPLE_VALS, row, values);
}
}
@@ -162,9 +169,7 @@
opts.iterations = Long.MAX_VALUE;
try (AccumuloClient client = opts.createAccumuloClient()) {
- if (!client.tableOperations().exists(opts.getTableName()))
- client.tableOperations().create(opts.getTableName());
-
+ Common.createTableWithNamespace(client, opts.getTableName());
Thread writer = new Thread(
new Writer(client.createBatchWriter(opts.getTableName(), bwOpts.getBatchWriterConfig()),
opts.iterations));
@@ -180,7 +185,7 @@
writer.join();
r.stopNow();
reader.join();
- System.out.println("finished");
+ log.info("finished");
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/mapreduce/NGramIngest.java b/src/main/java/org/apache/accumulo/examples/mapreduce/NGramIngest.java
index 23937f6..2755858 100644
--- a/src/main/java/org/apache/accumulo/examples/mapreduce/NGramIngest.java
+++ b/src/main/java/org/apache/accumulo/examples/mapreduce/NGramIngest.java
@@ -23,6 +23,7 @@
import org.apache.accumulo.core.client.AccumuloClient;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.apache.accumulo.hadoop.mapreduce.AccumuloOutputFormat;
import org.apache.hadoop.fs.Path;
@@ -89,7 +90,7 @@
try (AccumuloClient client = opts.createAccumuloClient()) {
if (!client.tableOperations().exists(opts.tableName)) {
log.info("Creating table " + opts.tableName);
- client.tableOperations().create(opts.tableName);
+ Common.createTableWithNamespace(client, opts.tableName);
SortedSet<Text> splits = new TreeSet<>();
String numbers[] = "1 2 3 4 5 6 7 8 9".split("\\s");
String lower[] = "a b c d e f g h i j k l m n o p q r s t u v w x y z".split("\\s");
diff --git a/src/main/java/org/apache/accumulo/examples/mapreduce/WordCount.java b/src/main/java/org/apache/accumulo/examples/mapreduce/WordCount.java
index 1864fe3..d86be5a 100644
--- a/src/main/java/org/apache/accumulo/examples/mapreduce/WordCount.java
+++ b/src/main/java/org/apache/accumulo/examples/mapreduce/WordCount.java
@@ -23,9 +23,9 @@
import org.apache.accumulo.core.client.AccumuloClient;
import org.apache.accumulo.core.client.IteratorSetting;
-import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.data.Mutation;
import org.apache.accumulo.core.iterators.user.SummingCombiner;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.apache.accumulo.hadoop.mapreduce.AccumuloOutputFormat;
import org.apache.hadoop.fs.Path;
@@ -42,13 +42,15 @@
/**
* A simple MapReduce job that inserts word counts into Accumulo. See docs/mapred.md
*/
-public class WordCount {
+public final class WordCount {
private static final Logger log = LoggerFactory.getLogger(WordCount.class);
+ private WordCount() {}
+
static class Opts extends ClientOpts {
@Parameter(names = {"-t", "--table"}, description = "Name of output Accumulo table")
- String tableName = "wordCount";
+ String tableName = Common.NAMESPACE + ".wordcount";
@Parameter(names = {"-i", "--input"}, required = true, description = "HDFS input directory")
String inputDirectory;
@Parameter(names = {"-d", "--dfsPath"},
@@ -81,14 +83,12 @@
// Create Accumulo table and attach Summing iterator
try (AccumuloClient client = opts.createAccumuloClient()) {
- client.tableOperations().create(opts.tableName);
+ Common.createTableWithNamespace(client, opts.tableName);
IteratorSetting is = new IteratorSetting(10, SummingCombiner.class);
SummingCombiner.setColumns(is,
Collections.singletonList(new IteratorSetting.Column("count")));
SummingCombiner.setEncodingType(is, SummingCombiner.Type.STRING);
client.tableOperations().attachIterator(opts.tableName, is);
- } catch (TableExistsException e) {
- // ignore
}
// Create M/R job
diff --git a/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/BulkIngestExample.java b/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/BulkIngestExample.java
index 6b27309..0225731 100644
--- a/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/BulkIngestExample.java
+++ b/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/BulkIngestExample.java
@@ -37,12 +37,17 @@
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* Example map reduce job that bulk ingest data into an accumulo table. The expected input is text
* files containing tab separated key value pairs on each line.
*/
public final class BulkIngestExample {
+
+ private static final Logger log = LoggerFactory.getLogger(BulkIngestExample.class);
+
static final String workDir = "tmp/bulkWork";
static final String inputDir = "bulk";
static final String outputFile = "bulk/test_1.txt";
@@ -141,7 +146,8 @@
try (PrintStream out = new PrintStream(
new BufferedOutputStream(fs.create(new Path(workDir + SPLITS_TXT))))) {
- Collection<Text> splits = client.tableOperations().listSplits(SetupTable.tableName, 100);
+ Collection<Text> splits = client.tableOperations().listSplits(SetupTable.BULK_INGEST_TABLE,
+ 100);
for (Text split : splits)
out.println(Base64.getEncoder().encodeToString(split.copyBytes()));
job.setNumReduceTasks(splits.size() + 1);
@@ -158,9 +164,9 @@
FsShell fsShell = new FsShell(opts.getHadoopConfig());
fsShell.run(new String[] {"-chmod", "-R", "777", workDir});
System.err.println("Importing Directory '" + workDir + SLASH_FILES + "' to table '"
- + SetupTable.tableName + "'");
- client.tableOperations().importDirectory(workDir + SLASH_FILES).to(SetupTable.tableName)
- .load();
+ + SetupTable.BULK_INGEST_TABLE + "'");
+ client.tableOperations().importDirectory(workDir + SLASH_FILES)
+ .to(SetupTable.BULK_INGEST_TABLE).load();
}
return job.isSuccessful() ? 0 : 1;
}
diff --git a/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/SetupTable.java b/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/SetupTable.java
index ef4edb3..259f77b 100644
--- a/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/SetupTable.java
+++ b/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/SetupTable.java
@@ -20,34 +20,35 @@
import org.apache.accumulo.core.client.Accumulo;
import org.apache.accumulo.core.client.AccumuloClient;
-import org.apache.accumulo.core.client.TableExistsException;
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.examples.Common;
import org.apache.accumulo.examples.cli.ClientOpts;
import org.apache.hadoop.io.Text;
public final class SetupTable {
- static final String[] splits = {"row_00000333", "row_00000666"};
- static final String tableName = "test_bulk";
+ static final String BULK_INGEST_TABLE = Common.NAMESPACE + ".test_bulk";
private SetupTable() {}
- public static void main(String[] args) throws Exception {
+ public static void main(String[] args)
+ throws AccumuloSecurityException, TableNotFoundException, AccumuloException {
+
+ final String[] splits = {"row_00000333", "row_00000666"};
ClientOpts opts = new ClientOpts();
opts.parseArgs(SetupTable.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build()) {
- try {
- client.tableOperations().create(tableName);
- } catch (TableExistsException e) {
- // ignore
- }
+ Common.createTableWithNamespace(client, BULK_INGEST_TABLE);
// create a table with initial partitions
TreeSet<Text> initialPartitions = new TreeSet<>();
for (String split : splits) {
initialPartitions.add(new Text(split));
}
- client.tableOperations().addSplits(tableName, initialPartitions);
+ client.tableOperations().addSplits(BULK_INGEST_TABLE, initialPartitions);
}
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/VerifyIngest.java b/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/VerifyIngest.java
index dc354b2..3e6bf0c 100644
--- a/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/VerifyIngest.java
+++ b/src/main/java/org/apache/accumulo/examples/mapreduce/bulk/VerifyIngest.java
@@ -45,7 +45,8 @@
opts.parseArgs(VerifyIngest.class.getName(), args);
try (AccumuloClient client = Accumulo.newClient().from(opts.getClientPropsPath()).build();
- Scanner scanner = client.createScanner(SetupTable.tableName, Authorizations.EMPTY)) {
+ Scanner scanner = client.createScanner(SetupTable.BULK_INGEST_TABLE,
+ Authorizations.EMPTY)) {
scanner.setRange(new Range(String.format(ROW_FORMAT, 0), null));
@@ -59,30 +60,30 @@
Entry<Key,Value> entry = si.next();
if (!entry.getKey().getRow().toString().equals(String.format(ROW_FORMAT, i))) {
- log.error("unexpected row key {}; expected {}", entry.getKey().getRow(),
- String.format(ROW_FORMAT, i));
+ String formattedRow = String.format(ROW_FORMAT, i);
+ log.error("unexpected row key {}; expected {}", entry.getKey().getRow(), formattedRow);
ok = false;
}
if (!entry.getValue().toString().equals(String.format(VALUE_FORMAT, i))) {
- log.error("unexpected value {}; expected {}", entry.getValue(),
- String.format(VALUE_FORMAT, i));
+ var formattedValue = String.format(VALUE_FORMAT, i);
+ log.error("unexpected value {}; expected {}", entry.getValue(), formattedValue);
ok = false;
}
} else {
- log.error("no more rows, expected {}", String.format(ROW_FORMAT, i));
+ var formattedRow = String.format(ROW_FORMAT, i);
+ log.error("no more rows, expected {}", formattedRow);
ok = false;
break;
}
-
}
if (ok) {
- System.out.println("Data verification succeeded!");
+ log.info("Data verification succeeded!");
System.exit(0);
} else {
- System.out.println("Data verification failed!");
+ log.info("Data verification failed!");
System.exit(1);
}
}
diff --git a/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java b/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java
index 5447e28..304f33a 100644
--- a/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java
+++ b/src/main/java/org/apache/accumulo/examples/sample/SampleExample.java
@@ -24,6 +24,7 @@
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.SampleNotPresentException;
import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableExistsException;
import org.apache.accumulo.core.client.admin.CompactionConfig;
import org.apache.accumulo.core.client.admin.CompactionStrategyConfig;
import org.apache.accumulo.core.client.sample.RowSampler;
@@ -36,6 +37,8 @@
import org.apache.accumulo.examples.cli.ClientOnDefaultTable;
import org.apache.accumulo.examples.client.RandomBatchWriter;
import org.apache.accumulo.examples.shard.CutoffIntersectingIterator;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import com.google.common.collect.ImmutableMap;
@@ -46,6 +49,8 @@
*/
public class SampleExample {
+ private static final Logger log = LoggerFactory.getLogger(SampleExample.class);
+
// a compaction strategy that only selects files for compaction that have no sample data or sample
// data created in a different way than the tables
static final CompactionStrategyConfig NO_SAMPLE_STRATEGY = new CompactionStrategyConfig(
@@ -54,7 +59,7 @@
static class Opts extends ClientOnDefaultTable {
public Opts() {
- super("sampex");
+ super("examples.sampex");
}
}
@@ -64,11 +69,11 @@
opts.parseArgs(RandomBatchWriter.class.getName(), args, bwOpts);
try (AccumuloClient client = opts.createAccumuloClient()) {
-
- if (!client.tableOperations().exists(opts.getTableName())) {
+ try {
client.tableOperations().create(opts.getTableName());
- } else {
- System.out.println("Table exists, not doing anything.");
+ } catch (TableExistsException e) {
+ System.out.println("Table exists, not doing anything. Delete table " + opts.getTableName()
+ + " and re-run");
return;
}
diff --git a/src/main/java/org/apache/accumulo/examples/shard/Reverse.java b/src/main/java/org/apache/accumulo/examples/shard/Reverse.java
index 2d93f4c..139d9cd 100644
--- a/src/main/java/org/apache/accumulo/examples/shard/Reverse.java
+++ b/src/main/java/org/apache/accumulo/examples/shard/Reverse.java
@@ -41,10 +41,10 @@
static class Opts extends ClientOpts {
@Parameter(names = "--shardTable", description = "name of the shard table")
- String shardTable = "shard";
+ String shardTable;
@Parameter(names = "--doc2Term", description = "name of the doc2Term table")
- String doc2TermTable = "doc2Term";
+ String doc2TermTable;
}
public static void main(String[] args) throws Exception {
