| commit | 6d0ab9f820748d2d0612d6359fce408c6edb4d87 | [log] [tgz] |
|---|---|---|
| author | Christopher Tubbs <ctubbsii@apache.org> | Tue Mar 29 11:52:21 2022 -0400 |
| committer | GitHub <noreply@github.com> | Tue Mar 29 11:52:21 2022 -0400 |
| tree | 681ca71e267d8ab56539a131b3f67f71acc24bca | |
| parent | 6830a84c578c5de019b994d84cbec1b4177fa047 [diff] |
Fix logging bugs and update plugins (#87) * Update README to refer to lib/ directory (lib/ext/ is no longer expected to exist by default) * Add some color to the log output * Rename log4j configuration file so it uses a name that is recognized as a properties file so log4j doesn't try to parse it as XML, this rename also allows it to be loaded from the CLASSPATH * Use CLASSPATH environment instead of `-cp` option * Set up class path to include the Accumulo lib/ directory so it doesn't complain about missing log4j runtime libraries (This fixes #86) * Update pom.xml by updating ZK version, removing accumulo-tracer, and bumping plugin versions * Fix shading warning due to migration of javax.activation-api to jakarta.activation-api
Follow the steps below to run the Accumulo examples:
Clone this repository
git clone https://github.com/apache/accumulo-examples.git
Follow Accumulo's quickstart to install and run an Accumulo instance. Accumulo has an accumulo-client.properties in conf/ that must be configured as the examples will use this file to connect to your instance.
Review env.sh.example in to see if you need to customize it. If ACCUMULO_HOME & HADOOP_HOME are set in your shell, you may be able skip this step. Make sure ACCUMULO_CLIENT_PROPS is set to the location of your accumulo-client.properties.
cp conf/env.sh.example conf/env.sh vim conf/env.sh
Build the examples repo and copy the examples jar to Accumulo's lib/ directory to get on its class path:
./bin/build cp target/accumulo-examples.jar /path/to/accumulo/lib/
Each Accumulo example has its own documentation and instructions for running the example which are linked to below.
When running the examples, remember the tips below:
runex or runmr commands which are located in the bin/ directory of this repo. The runex command is a simple script that use the examples shaded jar to run a a class. The runmr starts a MapReduce job in YARN.accumulo and accumulo-util commands which are expected to be on your PATH. These commands are found in the bin/ directory of your Accumulo installation.Each example below highlights a feature of Apache Accumulo.
| Example | Description |
|---|---|
| batch | Using the batch writer and batch scanner |
| bloom | Creating a bloom filter enabled table to increase query performance |
| bulkIngest | Ingesting bulk data using map/reduce jobs on Hadoop |
| classpath | Using per-table classpaths |
| client | Using table operations, reading and writing data in Java. |
| combiner | Using example StatsCombiner to find min, max, sum, and count. |
| compactionStrategy | Configuring a compaction strategy |
| constraints | Using constraints with tables. Limit the mutation size to avoid running out of memory |
| deleteKeyValuePair | Deleting a key/value pair and verifying the deletion in RFile. |
| dirlist | Storing filesystem information. |
| export | Exporting and importing tables. |
| filedata | Storing file data. |
| filter | Using the AgeOffFilter to remove records more than 30 seconds old. |
| helloworld | Inserting records both inside map/reduce jobs and outside. And reading records between two rows. |
| isolation | Using the isolated scanner to ensure partial changes are not seen. |
| regex | Using MapReduce and Accumulo to find data using regular expressions. |
| reservations | Using conditional mutations to implement simple reservation system. |
| rgbalancer | Using a balancer to spread groups of tablets within a table evenly |
| rowhash | Using MapReduce to read a table and write to a new column in the same table. |
| sample | Building and using sample data in Accumulo. |
| shard | Using the intersecting iterator with a term index partitioned by document. |
| spark | Using Accumulo as input and output for Apache Spark jobs |
| tabletofile | Using MapReduce to read a table and write one of its columns to a file in HDFS. |
| terasort | Generating random data and sorting it using Accumulo. |
| uniquecols | Use MapReduce to count unique columns in Accumulo |
| visibility | Using visibilities (or combinations of authorizations). Also shows user permissions. |
| wordcount | Use MapReduce and Accumulo to do a word count on text files |
This repository can be used to test Accumulo release candidates. See docs/release-testing.md.