| commit | b9bcfa0100b13dc02dbb2c3d2e62fb536be6959c | [log] [tgz] |
|---|---|---|
| author | Jay Sen <jsenjaliya@paypal.com> | Fri Sep 27 13:37:41 2019 -0700 |
| committer | suvasude <suvasude@linkedin.biz> | Fri Sep 27 13:37:41 2019 -0700 |
| tree | 368debe2d23840c136bf16bc4c7dc7f7e55984a5 | |
| parent | 5d4795e28fe7502b0e2233fc830c2b3f7adeb637 [diff] |
[GOBBLIN-707][GOBBLIN-694][GOBBLIN-477][GOBBLIN-719] rewrite gobblin script to combine all modes and command [GOBBLIN-719] fix invalid git links for classes in docs fixing all file paths and linksA [GOBBLIN-707]-rewrite gobblin script combining all gobblin modes and commands along with configs updating as per review 1 updates docs as per gobblin.sh changes + some more improvements in gobblin.sh updates docs as per gobblin.sh changes + some more improvements in gobblin.sh - 2 Merge branch 'master' of github.com:jhsenjaliya/incubator-gobblin Merge branch 'master' of github.com:apache/incubator-gobblin Merge branch 'master' of github.com:apache/incubator-gobblin [GOBBLIN-707]-rewrite gobblin script combining all gobblin modes and commands along with configs updating as per review 1 updates docs as per gobblin.sh changes + some more improvements in gobblin.sh updates docs as per gobblin.sh changes + some more improvements in gobblin.sh - 2 Merge branch 'GOBBLIN-707' of github.com:jhsenjaliya/incubator-gobblin into GOBBLIN-707 adding project version Merge branch 'master' of github.com:apache/incubator-gobblin into GOBBLIN-707 arrange HADOOP classpath restructure command vs services help all command and cli classes under GobblinCli with Alias, remove Cli class uncomment required code keep individual script that calls new gobblin script internally resolves findBug error, replaced system exit with return update in config, keeping default as before update to yarn, MR config updating gobblin.sh and adding gobblin-service config Merge branch 'master' into GOBBLIN-707 updates to gobblin.sh and help on cli classes rename service manager to gobblin-as-service merge upstream changes revert to upstream config to put change later Closes #2578 from jhsenjaliya/GOBBLIN-707
Apache Gobblin is a universal data ingestion framework for extracting, transforming, and loading large volume of data from a variety of data sources, e.g., databases, rest APIs, FTP/SFTP servers, filers, etc., onto Hadoop. Apache Gobblin handles the common routine tasks required for all data ingestion ETLs, including job/task scheduling, task partitioning, error handling, state management, data quality checking, data publishing, etc. Gobblin ingests data from different data sources in the same execution framework, and manages metadata of different sources all in one place. This, combined with other features such as auto scalability, fault tolerance, data quality assurance, extensibility, and the ability of handling data model evolution, makes Gobblin an easy-to-use, self-serving, and efficient data ingestion framework.
If building the distribution with tests turned on:
Run the following command for downloading the gradle-wrapper.jar from Gobblin git repository to gradle/wrapper directory.
wget --no-check-certificate -P gradle/wrapper https://github.com/apache/incubator-gobblin/raw/0.12.0/gradle/wrapper/gradle-wrapper.jar (or) curl --insecure -L https://github.com/apache/incubator-gobblin/raw/0.12.0/gradle/wrapper/gradle-wrapper.jar > gradle/wrapper/gradle-wrapper.jar
Alternatively, you can download it manually from: https://github.com/apache/incubator-gobblin/blob/0.12.0/gradle/wrapper/gradle-wrapper.jar
Make sure that you download it to gradle/wrapper directory.
./gradlew rat. Report will be generated under build/rat/rat-report.html./gradlew build -x findbugsMain -x test -x rat -x checkstyleMain The distribution will be created in build/gobblin-distribution/distributions directory. (or)./gradlew build The distribution will be created in build/gobblin-distribution/distributions directory.