In order to allow users to easily try Kylin, and to facilitate developers to verify and debug after modifying the source code. We provide Kylin's docker image. In this image, each service that Kylin relies on is properly installed and deployed, including:
We have pushed the Kylin image for the user to the docker hub. Users do not need to build the image locally, just execute the following command to pull the image from the docker hub:
docker pull apachekylin/apache-kylin-standalone:3.0.0-alpha2
After the pull is successful, execute the following command to start the container:
docker run -d \ -m 8G \ -p 7070:7070 \ -p 8088:8088 \ -p 50070:50070 \ -p 8032:8032 \ -p 8042:8042 \ -p 60010:60010 \ apachekylin/apache-kylin-standalone:3.0.0-alpha2
The following services are automatically started when the container starts:
and run automatically $KYLIN_HOME/bin/sample.sh , create a kylin_streaming_topic topic in Kafka and continue to send data to this topic. This is to let the users start the container and then experience the batch and streaming way to build the cube and query.
After the container is started, we can enter the container through the docker exec command. Of course, since we have mapped the specified port in the container to the local port, we can open the pages of each service directly in the native browser, such as:
In the container, the relevant environment variables are as follows:
JAVA_HOME=/home/admin/jdk1.8.0_141 HADOOP_HOME=/home/admin/hadoop-2.7.0 KAFKA_HOME=/home/admin/kafka_2.11-1.1.1 SPARK_HOME=/home/admin/spark-2.3.1-bin-hadoop2.6 HBASE_HOME=/home/admin/hbase-1.1.2 HIVE_HOME=/home/admin/apache-hive-1.2.1-bin KYLIN_HOME=/home/admin/apache-kylin-3.0.0-alpha2-bin-hbase1x
The docker image can also be used when developers have modified the source code and want to package, deploy, and verify the source code. First, we go to the docker directory under the root directory of the source and execute the script below to build the image and copy the source into the image.:
#!/usr/bin/env bash
echo "start build kylin image base on current source code"
rm -rf ./kylin
mkdir -p ./kylin
echo "start copy kylin source code"
for file in `ls ../../kylin/`
do
if [ docker != $file ]
then
cp -r ../../kylin/$file ./kylin/
fi
done
echo "finish copy kylin source code"
docker build -t apache-kylin-standalone .⏎
Due to need to download and deploy various binary packages over the network, the entire build process can last for tens of minutes, depending on the network.
When the image build is complete, execute the following command to start the container:
docker run -d \ -m 8G \ -p 7070:7070 \ -p 8088:8088 \ -p 50070:50070 \ -p 8032:8032 \ -p 8042:8042 \ -p 60010:60010 \ apache-kylin-standalone
When the container starts, execute the docker exec command to enter the container. The source code is stored in the container dir /home/admin/kylin_sourcecode, execute the following command to package the source code:
cd /home/admin/kylin_sourcecod build/script/package.sh
After the package is complete, an binary package ending in .tar.gz will be generated in the /home/admin/kylin_sourcecode/dist directory, such as apache-kylin-3.0.0-alpha2-bin-hbase1x.tar.gz. We can use this binary package to deploy and launch Kylin services such as:
cp /home/admin/kylin_sourcecode/dist/apache-kylin-3.0.0-alpha2-bin-hbase1x.tar.gz /home/admin tar -zxvf /home/admin/apache-kylin-3.0.0-alpha2-bin-hbase1x.tar.gz /home/admin/apache-kylin-3.0.0-alpha2-bin-hbase1x/kylin.sh start
We can also open pages for services such as Hdfs, Yarn, HBase, and Kylin in the browser of this machine.
In order to allow Kylin to build the cube smoothly, the memory resource we configured for Yarn NodeManager is 6G, plus the memory occupied by each service, please ensure that the memory of the container is not less than 8G, so as to avoid errors due to insufficient memory.
For the resource setting method for the container, please refer to:
For old docker image, please check the github page kylin-docker.