| commit | fa913b4174e1c874ddb12e4901c48bd86a7f70e4 | [log] [tgz] |
|---|---|---|
| author | Aldrin Piri <aldrin@apache.org> | Sun May 07 00:29:54 2017 -0400 |
| committer | Tony Kurc <tkurc@apache.org> | Sun May 07 17:12:30 2017 +0000 |
| tree | 37fe81e990b1f88add274c2ecde22f62a8c3b19f | |
| parent | 65ca9ecd43af56ed86816a2f834633a862d2d564 [diff] |
MINIFI-302 Exclude generated docs from the source tarball via CPack ignore. This closes #94 Signed-off-by: Tony Kurc <tkurc@apache.org>
MiNiFi is a child project effort of Apache NiFi. This repository is for a native implementation in C++.
Apache NiFi - MiNiFi C++ is a complementary data collection approach that supplements the core tenets of NiFi in dataflow management, focusing on the collection of data at the source of its creation. The C++ implementation is an additional implementation to the one in Java with the aim of an even smaller resource footprint.
Specific goals for MiNiFi are comprised of:
Perspectives of the role of MiNiFi should be from the perspective of the agent acting immediately at, or directly adjacent to, source sensors, systems, or servers.
The needed dependencies can be installed with the following commands for:
Yum based Linux Distributions
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ yum install cmake \ gcc gcc-c++ \ libcurl-devel \ leveldb-devel leveldb \ libuuid libuuid-devel \ boost-devel \ libssl-dev \ doxygen
Aptitude based Linux Distributions
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ apt-get install cmake \ gcc g++ \ libcurl-dev \ libleveldb-dev libleveldb1v5 \ uuid-dev uuid \ libboost-all-dev libssl-dev \ doxygen
OS X Using Homebrew (with XCode Command Line Tools installed)
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ brew install cmake \ curl \ leveldb \ ossp-uuid \ boost \ openssl \ doxygen
From your source checkout, create a directory to perform the build (e.g. build) and cd into that directory.
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ mkdir build # ~/Development/code/apache/nifi-minifi-cpp on git:master $ cd build
Perform a cmake .. to generate the project files
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ cmake .. ... -- Configuring done -- Generating done -- Build files have been written to: /Users/apiri/Development/code/apache/nifi-minifi-cpp/build
Perform a build
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ make Scanning dependencies of target gmock_main Scanning dependencies of target gmock Scanning dependencies of target minifi Scanning dependencies of target gtest Scanning dependencies of target yaml-cpp [ 1%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/CMakeFiles/gtest.dir/src/gtest-all.cc.o [ 3%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock.dir/gtest/src/gtest-all.cc.o [ 3%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock.dir/src/gmock-all.cc.o [ 6%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock_main.dir/gtest/src/gtest-all.cc.o [ 6%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock_main.dir/src/gmock-all.cc.o [ 7%] Building CXX object libminifi/CMakeFiles/minifi.dir/src/Configure.cpp.o ... [ 97%] Linking CXX executable minifi [ 97%] Built target minifiexe [ 98%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/CMakeFiles/run-tests.dir/node/node_test.cpp.o [100%] Linking CXX executable run-tests [100%] Built target run-tests
Create a binary assembly located in your build directory with suffix -bin.tar.gz
~/Development/code/apache/nifi-minifi-cpp/build $ make package Run CPack packaging tool for source... CPack: Create package using TGZ CPack: Install projects CPack: - Install directory: ~/Development/code/apache/nifi-minifi-cpp CPack: Create package CPack: - package: ~/Development/code/apache/nifi-minifi-cpp/build/nifi-minifi-cpp-0.1.0-bin.tar.gz generated.
Create a source assembly located in your build directory with suffix -source.tar.gz
~/Development/code/apache/nifi-minifi-cpp/build $ make package_source Run CPack packaging tool for source... CPack: Create package using TGZ CPack: Install projects CPack: - Install directory: ~/Development/code/apache/nifi-minifi-cpp CPack: Create package CPack: - package: ~/Development/code/apache/nifi-minifi-cpp/build/nifi-minifi-cpp-0.1.0-source.tar.gz generated.
(Optional) Create a Docker image from the resulting binary assembly output from “make package”.
~/Development/code/apache/nifi-minifi-cpp/build $ make docker NiFi-MiNiFi-CPP Version: 0.2.0 Current Working Directory: /Users/jdyer/Development/github/nifi-minifi-cpp/docker CMake Source Directory: /Users/jdyer/Development/github/nifi-minifi-cpp MiNiFi Package: nifi-minifi-cpp-0.2.0-bin.tar.gz Docker Command: 'docker build --build-arg UID=1000 --build-arg GID=1000 --build-arg MINIFI_VERSION=0.2.0 --build-arg MINIFI_PACKAGE=nifi-minifi-cpp-0.2.0-bin.tar.gz -t apacheminificpp:0.2.0 .' Sending build context to Docker daemon 777.2 kB Step 1 : FROM alpine:3.5 ---> 88e169ea8f46 Step 2 : MAINTAINER Apache NiFi <dev@nifi.apache.org> ... Step 15 : CMD $MINIFI_HOME/bin/minifi.sh run ---> Using cache ---> c390063d9bd1 Successfully built c390063d9bd1 Built target docker
Remove the build directory created above.
# ~/Development/code/apache/nifi-minifi-cpp on git:master $ rm -rf ./build
The ‘conf’ directory in the root contains a template config.yml document.
This is compatible with the format used with the Java MiNiFi application. Currently, a subset of the configuration is supported and MiNiFi C++ is currently compatible with version 1 of the MiNiFi YAML schema. Additional information on the YAML format for the config.yml and schema versioning can be found in the MiNiFi System Administrator Guide.
Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - schema version 1) to aid in creating a flow configuration from a generated template exported from a NiFi instance. The MiNiFi Toolkit Converter tool can be downloaded from http://nifi.apache.org/minifi/download.html under the MiNiFi Toolkit Binaries section. Information on its usage is available at https://nifi.apache.org/minifi/minifi-toolkit.html.
Flow Controller:
id: 471deef6-2a6e-4a7d-912a-81cc17e3a205
name: MiNiFi Flow
Processors:
- name: GetFile
id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
class: org.apache.nifi.processors.standard.GetFile
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 1 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list:
Properties:
Input Directory: /tmp/getfile
Keep Source File: true
Connections:
- name: TransferFilesToRPG
id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
source name: GetFile
source id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
source relationship name: success
destination id: 471deef6-2a6e-4a7d-912a-81cc17e3a204
max work queue size: 0
max work queue data size: 1 MB
flowfile expiration: 60 sec
Remote Processing Groups:
- name: NiFi Flow
id: 471deef6-2a6e-4a7d-912a-81cc17e3a208
url: http://localhost:8080/nifi
timeout: 30 secs
yield period: 10 sec
Input Ports:
- id: 471deef6-2a6e-4a7d-912a-81cc17e3a204
name: From Node A
max concurrent tasks: 1
Properties:
Port: 10001
Host Name: localhost
in minifi.properties enable tls ssl nifi.remote.input.secure=true if you want to enable client certificate base authorization nifi.security.need.ClientAuth=true setup the client certificate and private key PEM files nifi.security.client.certificate=./conf/client.pem nifi.security.client.private.key=./conf/client.pem setup the client private key passphrase file nifi.security.client.pass.phrase=./conf/password setup the client CA certificate file nifi.security.client.ca.certificate=./conf/nifi-cert.pem if you do not want to enable client certificate base authorization nifi.security.need.ClientAuth=false
Add Provenance Reporting to config.yml Provenance Reporting: scheduling strategy: TIMER_DRIVEN scheduling period: 1 sec port: 10001 host: localhost port uuid: 471deef6-2a6e-4a7d-912a-81cc17e3a204 batch size: 100
After completing a build, the application can be run by issuing the following from :
$ ./bin/minifi.sh start
By default, this will make use of a config.yml located in the conf directory. This configuration file location can be altered by adjusting the property nifi.flow.configuration.file in minifi.properties located in the conf directory.
MiNiFi can then be stopped by issuing:
$ ./bin/minifi.sh stop
MiNiFi can also be installed as a system service using minifi.sh with an optional “service name” (default: minifi)
$ ./bin/minifi.sh install [service name]
See https://nifi.apache.org/minifi for the latest documentation.
We welcome all contributions to Apache MiNiFi. To make development easier, we've included the linter for the Google Style guide. Google provides an Eclipse formatter for their style guide. It is located here. New contributions are expected to follow the Google style guide when it is reasonable. Additionally, all new files must include a copy of the Apache License Header.
Once you have completed your changes, including source code and tests, you can verify that you follow the Google style guide by running the following command: $ make linter. This will provide output for all source files.
Except as otherwise noted this software is licensed under the Apache License, Version 2.0
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.