Development in IDE

1. Start eagle-server

In IDE, configure the following main class and program arguments

org.apache.eagle.server.ServerMain server src/main/resources/configuration.yml

2. Start alert engine

2.1 Create new site

http://localhost:9090/rest/sites POST

{
"siteId" : "testsite",
"siteName" :"testsite",
"description" : "test description",
"context" : {}
}

###n2.2 Create logic alert engine topology

http://localhost:9090/rest/metadata/topologies POST

{
   "name": "alertUnitTopology_1",
   "numOfSpout": 1,
   "numOfAlertBolt": 10,
   "numOfGroupBolt": 4,
   "spoutId": "alertEngineSpout",
   "groupNodeIds": [
      "streamRouterBolt0",
      "streamRouterBolt1",
      "streamRouterBolt2",
      "streamRouterBolt3"
   ],
   "alertBoltIds": [
      "alertBolt0",
      "alertBolt1",
      "alertBolt2",
      "alertBolt3",
      "alertBolt4",
      "alertBolt5",
      "alertBolt6",
      "alertBolt7",
      "alertBolt8",
      "alertBolt9"
   ],
   "pubBoltId": "alertPublishBolt",
   "spoutParallelism": 1,
   "groupParallelism": 1,
   "alertParallelism": 1
}

2.3 Install alert engine application

Please reference eagle-core/eagle-alert-parent/eagle-alert-app/src/main/resources/META-INF/providers/org.apache.eagle.alert.app.AlertUnitTopologyAppProvider.xml for complete configuration.

http://localhost:9090/rest/apps/install POST

{
"siteId" : "testsite",
"appType" : "AlertUnitTopologyApp",
"mode" : "LOCAL",
"configuration" : {
  }
}

2.4 Run alert engine

Please use correct uuid

http://localhost:9090/rest/apps/start POST

{
"uuid": "dc61c4b8-f60d-4d95-bfd7-f6b07382a3f3",
"appId": "AlertUnitTopologyApp－testsite"
}

3 Start Hdfs audit log monitoring application

3.1 Install HdfsAuditLog app

http://localhost:9090/rest/apps/install POST

{
"siteId" : "testsite",
"appType" : "HdfsAuditLogApplication",
"mode" : "LOCAL",
"configuration" : {
  "dataSourceConfig.topic" :"hdfs_audit_log"}
}

3.2 Start HdfsAuditLog app

Please use correct uuid

http://localhost:9090/rest/apps/start POST

{
"uuid": "dc61c4b8-f60d-4d95-bfd7-f6b07382a3f3",
"appId": "HdfsAuditLogApplication－testsite"
}

4 Check

4.1 Check if alert data source is created

http://localhost:9090/rest/metadata/datasources GET

4.2 Check if alert stream is creatd

http://localhost:9090/rest/metadata/streams GET

5 Create alert policy and verify alert

5.1 create one policy

http://localhost:9090/rest/metadata/policies POST

{
   "name": "hdfsPolicy",
   "description": "hdfsPolicy",
   "inputStreams": [
      "hdfs_audit_log_enriched_stream"
   ],
   "outputStreams": [
      "hdfs_audit_log_enriched_stream_out"
   ],
   "definition": {
      "type": "siddhi",
      "value": "from hdfs_audit_log_enriched_stream[user=='hadoop'] select * insert into hdfs_audit_log_enriched_stream_out"
   },
   "partitionSpec": [
      {
         "streamId": "hdfs_audit_log_enriched_stream",
         "type": "GROUPBY",
         "columns" : [
            "user"
         ]
      }
   ],
   "parallelismHint": 2
}

5.2 Create alert publishment

{
	"name":"hdfs_audit_log_enriched_stream_out",
	"type":"org.apache.eagle.alert.engine.publisher.impl.AlertEmailPublisher",
	"policyIds": [
		"hdfsPolicy"
	],
	"properties": {
	  "subject":"alert when user is hadoop",
	  "template":"",
	  "sender": "eagle@apache.org",
	  "recipients": "eagle@apache.org",
	  "mail.smtp.host":"",
	  "connection": "plaintext",
	  "mail.smtp.port": "25"
	},
	"dedupIntervalMin" : "PT1M",
	"serializer" : "org.apache.eagle.alert.engine.publisher.impl.StringEventSerializer"
}

5.3 Send message and verify alert

./kafka-console-producer.sh --topic hdfs_audit_log --broker-list sandbox.hortonworks.com:6667

2015-04-24 12:51:31,798 INFO FSNamesystem.audit: allowed=true ugi=hdfs (auth:SIMPLE) ip=/10.0.2.15 cmd=getfileinfo src=/apps/hbase/data dst=null perm=null proto=rpc