addons/adf/README

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

ADF Provider
=======================


Overview
---------

This integration allows Microsoft Azure Data Factory pipelines to invoke Falcon activities
(i.e. replication, hive and pig proessing work), so the user can build a hybrid Hadoop data pipelines
leveraging on-premises Hadoop clusters and cloud based Cortana Analytics services
like HDInsight Hadoop clusters and Azure Machine Learning.


Usage
---------

1. ADFProviderService is not enabled by default. Please make sure it is enabled in conf/startup.properties:

*.application.services= ...
                        org.apache.falcon.service.ProxyUserService,\
                        org.apache.falcon.adfservice.ADFProviderService

2. Falcon reads Azure Service Bus credentials from conf/startup.properties when it starts.
So, the credential needs to be added before starting Falcon,
and Falcon needs to be restarted if there is any change in the credential.

Example:

######### ADF Configurations start #########

# A String object that represents the namespace
*.microsoft.windowsazure.services.servicebus.namespace=hwtransport

# Request and status queues on the namespace
*.microsoft.windowsazure.services.servicebus.requestqueuename=adfrequest
*.microsoft.windowsazure.services.servicebus.statusqueuename=adfstatus

# A String object that contains the SAS key name
*.microsoft.windowsazure.services.servicebus.sasKeyName=RootManageSharedAccessKey

# A String object that contains the SAS key
*.microsoft.windowsazure.services.servicebus.sasKey=4kt2x6yEoWZZSFZofyXEoxly0knHL7FPMqLD14ov1jo=

# A String object containing the base URI that is added to your Service Bus namespace to form the URI to connect
# to the Service Bus service. To access the default public Azure service, pass ".servicebus.windows.net"
*.microsoft.windowsazure.services.servicebus.serviceBusRootUri=.servicebus.windows.net

# Service bus polling frequency (in seconds)
*.microsoft.windowsazure.services.servicebus.polling.frequency=60