layout: recipes-doc title: Accumulo Export Queue Specialization version: 1.0.0-beta-2

Background

The Export Queue Recipe provides a generic foundation for building export mechanism to any external data store. A specific Export Queue implementation for Accumulo is provided in the Fluo Recipes Accumulo module.

This implementation provides the following functionality :

  • Safely batches writes to Accumulo made by multiple transactions exporting data.
  • Stores Accumulo connection information in Fluo configuration, making it accessible by Export Observers running on other nodes.
  • Provides utility code that make it easier and shorter to code common Accumulo export patterns.

Example Use

Exporting to Accumulo is easy. Only two things need to be done.

  • Configure the export queue.
  • Implement [AccumuloExport][2] with custom code that generates Accumulo mutations.

The following code shows how to configure an Export Queue that will write to an external Accumulo table.


FluoConfiguration fluoConfig = ...;

//Configure an export queue to use the classes Fluo Recipes provides for
//exporting to Accumulo
ExportQueue.configure(fluoConfig, new ExportQueue.Options(EXPORT_QUEUE_ID,
    AccumuloExporter.class.getName(), String.class.getName(), AccumuloExport.class.getName(),
    numMapBuckets));

String instance = //Name of accumulo instance exporting to
String zookeepers = //zookeepers used by Accumulo instance exporting to
String user = //Accumulo username, user that can write to exportTable
String password = //Accumulo user password
String exportTable = //Name of table to export to

//Configure the Accumulo table to export to.
AccumuloExporter.setExportTableInfo(fluoConfig, EXPORT_QUEUE_ID,
    new TableInfo(instance, zookeepers, user, password, exportTable));

//initialize Fluo using fluoConfig

After the export queue is initialized as specified above, any Object that implements [AccumuloExport][2] can be added to the queue. For the common pattern of deleting old data and inserting new data, consider extending [DifferenceExport][3].

[2]: {{ site.old_api_static }}/fluo-recipes-accumulo/1.0.0-beta-2/io/fluo/recipes/accumulo/export/AccumuloExport.html [3]: {{ site.old_api_static }}/fluo-recipes-accumulo/1.0.0-beta-2/io/fluo/recipes/accumulo/export/DifferenceExport.html