blob: 39a328608edc4d49fa359544b731e4f692f7bbe4 [file] [log] [blame]
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
[[quickstart]]
= Apache Kudu Quickstart
:author: Kudu Team
:imagesdir: ./images
:icons: font
:toc: left
:toclevels: 2
:doctype: book
:backend: html5
:sectlinks:
:experimental:
Follow these instructions to set up and run a local Kudu Cluster using Docker,
and get started using Apache Kudu in minutes.
NOTE: This is intended for demonstration purposes only and shouldn't
be used for production or performance/scale testing.
[[quickstart_vm]]
== Install Docker
Follow the Docker link:https://docs.docker.com/install/[install documentation]
to install docker in your Linux, Mac, or Windows environment.
Configure the Docker install to have enough resources to run the quickstart guides.
- link:https://docs.docker.com/docker-for-mac/#resources[Docker for Mac Resource Configuration Guide]
A minimum configuration that can run all the quickstart examples comfortably is:
- 4 CPUs
- 6 GB Memory
- 50 GB Disk
NOTE: You can likely get by with a lower resource configuration, but you may lose some performance and stability.
You may also want to read through the Docker getting started guide, but that isn't a requirement.
== Clone the Repository
Clone the Apache Kudu repository using Git and change to the `kudu` directory:
[source,bash]
----
git clone https://github.com/apache/kudu
cd kudu
----
== Start the Quickstart Cluster
=== Set KUDU_QUICKSTART_IP
Set the `KUDU_QUICKSTART_IP` environment variable to your ip address:
[source,bash]
----
export KUDU_QUICKSTART_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}' | tail -1)
----
=== Bring up the Cluster
Then use `docker-compose` to start a cluster with 3 master servers and 5 tablet servers.
When inside the docker network/containers the master addresses will be
`kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251` and when on the host machine
you can specify the master addresses with `localhost:7051,localhost:7151,localhost:7251`.
[source,bash]
----
docker-compose -f docker/quickstart.yml up -d
----
NOTE: You can remove the `-d` flag to run the cluster in the foreground.
=== View the Web-UI
Once the cluster is started you can view the master web-ui by visiting link:localhost:8050[localhost:8050].
=== Check the cluster health
Use the command below to get a bash shell in the `kudu-master-1` container:
[source,bash]
----
docker exec -it $(docker ps -aqf "name=kudu-master-1") /bin/bash
----
You can now run the Kudu `ksck` tool to verify the cluster is healthy:
[source,bash]
----
kudu cluster ksck kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
----
Alternatively, if you have a kudu binary available on your host machine,
you can run `ksck` there via:
[source,bash]
----
export KUDU_USER_NAME=kudu
kudu cluster ksck localhost:7051,localhost:7151,localhost:7251
----
NOTE: Setting `KUDU_USER_NAME=kudu` simplifies using Kudu from various user
accounts in a non-secure environment.
== Running a Brief Example
Now that a Kudu cluster is up and running, examples and integrations can be
run against the cluster. The commands below run the `java-example` against
the quickstart cluster:
[source,bash]
----
export KUDU_USER_NAME=kudu
cd examples/java/java-example
mvn package
java -DkuduMasters=localhost:7051,localhost:7151,localhost:7251 -jar target/kudu-java-example-1.0-SNAPSHOT.jar
----
== More Examples
More complete walkthroughs using the quickstart Kudu cluster can be found in the
`examples/quickstart` directory. For convenience you can browse them on
link:https://github.com/apache/kudu/tree/master/examples/quickstart[Github].
- link:https://github.com/apache/kudu/tree/master/examples/quickstart/nifi[NiFi Quickstart Guide]
- link:https://github.com/apache/kudu/tree/master/examples/quickstart/spark[Spark Quickstart Guide]
- link:https://github.com/apache/kudu/tree/master/examples/quickstart/impala[Impala Quickstart Guide]
== Destroying the Cluster
Once you are done with the quickstart cluster you can shutdown in a couple of ways.
If you ran `docker-compose` without the `-d` flag, you can use `ctrl + c` to
stop the cluster.
If you ran `docker-compose` with the `-d` flag, you can use the following to
gracefully shutdown the cluster:
[source,bash]
----
docker-compose -f docker/quickstart.yml down
----
Another alternative is to stop all of the Kudu containers via:
[source,bash]
----
docker stop $(docker ps -aqf "name=kudu")
----
If you want to remove the cluster state you can also remove the docker
containers and volumes via:
[source,bash]
----
docker rm $(docker ps -aqf "name=kudu")
docker volume rm $(docker volume ls --filter name=kudu -q)
----
== Troubleshooting
=== Viewing the logs
To view the logs you can use the `docker logs` command. Below is an example
that will show the logs one of the tablet servers:
[source,bash]
----
docker logs $(docker ps -aqf "name=kudu-tserver-1")
----
=== Changing the Kudu version
To change the version of Kudu Docker images used you can override the default value
of `latest` by setting the `KUDU_QUICKSTART_VERSION` environment variable.
[source,bash]
----
export KUDU_QUICKSTART_VERSION="1.9.0"
----
=== Changing the Kudu configuration
To change the configuration flags passed to the master and tablet servers you
can edit the `docker/quickstart.yml` file before starting the cluster.
=== Killing a Kudu master
Due to link:https://issues.apache.org/jira/browse/KUDU-1620[KUDU-1620] master hosts
are always expected to be reachable.
== Next Steps
- link:installation.html[Installing Kudu]
- link:configuration.html[Configuring Kudu]