Kylin need many dependent jars (hadoop/hive/hcat/hbase/kafka) on classpath to work, but Kylin doesn‘t ship them. It will seek these jars from your local machine by running commands like hbase classpath, hive -e set etc. The founded jars’ path will be appended to the environment variable HBASE_CLASSPATH (Kylin uses hbase shell command to start up, which will read this). But in some Hadoop distribution (like AWS EMR 5.0), the hbase shell doesn't keep the origin HBASE_CLASSPATH value, that causes the “NoClassDefFoundError”.
To fix this, find the hbase shell script (in hbase/bin folder), and search HBASE_CLASSPATH, check whether it overwrite the value like :
{% highlight Groff markup %} export HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/:$HADOOP_HOME/lib/:$ZOOKEEPER_HOME/:$ZOOKEEPER_HOME/lib/ {% endhighlight %}
{% highlight Groff markup %} export HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/:$HADOOP_HOME/lib/:$ZOOKEEPER_HOME/:$ZOOKEEPER_HOME/lib/:$HBASE_CLASSPATH {% endhighlight %}
fixed_length, integer) 2) or set a bigger value for kylin.dictionary.max.cardinality in conf/kylin.properties.{% highlight Groff markup %} I was able to deploy Kylin with following option in POM. <hadoop2.version>2.5.0</hadoop2.version> <yarn.version>2.5.0</yarn.version> <hbase-hadoop2.version>0.98.6-hadoop2</hbase-hadoop2.version> <zookeeper.version>3.4.5</zookeeper.version> <hive.version>0.13.1</hive.version> My Cluster is running on Cloudera Distribution CDH 5.2.0. {% endhighlight %}
{% highlight Groff markup %} ${KYLIN_HOME}/tomcat/webapps/kylin/WEB-INF/classes/kylinSecurity.xml {% endhighlight %}
{% highlight Groff markup %} Original SQL: select fact.slr_sgmt, sum(case when cal.RTL_WEEK_BEG_DT = ‘2015-09-06’ then gmv else 0 end) as W36, sum(case when cal.RTL_WEEK_BEG_DT = ‘2015-08-30’ then gmv else 0 end) as W35 from ih_daily_fact fact inner join dw_cal_dt cal on fact.cal_dt = cal.cal_dt group by fact.slr_sgmt {% endhighlight %}
{% highlight Groff markup %} Using sub-query select a.slr_sgmt, sum(case when a.RTL_WEEK_BEG_DT = ‘2015-09-06’ then gmv else 0 end) as W36, sum(case when a.RTL_WEEK_BEG_DT = ‘2015-08-30’ then gmv else 0 end) as W35 from ( select fact.slr_sgmt as slr_sgmt, cal.RTL_WEEK_BEG_DT as RTL_WEEK_BEG_DT, sum(gmv) as gmv36, sum(gmv) as gmv35 from ih_daily_fact fact inner join dw_cal_dt cal on fact.cal_dt = cal.cal_dt group by fact.slr_sgmt, cal.RTL_WEEK_BEG_DT ) a group by a.slr_sgmt {% endhighlight %}
Please add proxy for your NPM:npm config set proxy http://YOUR_PROXY_IP
Please update your local NPM repository to using any mirror of npmjs.org, like Taobao NPM (请更新您本地的NPM仓库以使用国内的NPM镜像，例如淘宝NPM镜像) :
http://npm.taobao.org
kylinSecurity.xml. To modify the password, you need firstly get the new password's encrypted value (with BCrypt), and then set it in kylinSecurity.xml. Here is a sample with password ‘ABCDE’{% highlight Groff markup %}
cd $KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/lib
java -classpath kylin-server-base-2.3.0.jar:spring-beans-4.3.10.RELEASE.jar:spring-core-4.3.10.RELEASE.jar:spring-security-core-4.2.3.RELEASE.jar:commons-codec-1.7.jar:commons-logging-1.1.3.jar org.apache.kylin.rest.security.PasswordPlaceholderConfigurer BCrypt ABCDE
BCrypt encrypted password is: $2a$10$A7.J.GIEOQknHmJhEeXUdOnj2wrdG4jhopBgqShTgDkJDMoKxYHVu
{% endhighlight %}
Then you can set it into `kylinSecurity.xml'
{% highlight Groff markup %}
vi ./tomcat/webapps/kylin/WEB-INF/classes/kylinSecurity.xml
{% endhighlight %}
Replace the origin encrypted password with the new one: {% highlight Groff markup %}
<bean class="org.springframework.security.core.userdetails.User" id="adminUser">
<constructor-arg value="ADMIN"/>
<constructor-arg
value="$2a$10$A7.J.GIEOQknHmJhEeXUdOnj2wrdG4jhopBgqShTgDkJDMoKxYHVu"/>
<constructor-arg ref="adminAuthorities"/>
</bean>
{% endhighlight %}
Restart Kylin to take effective. If you have multiple Kylin server as a cluster, do the same on each instance.
The data in ‘hdfs-working-dir’ (‘hdfs:///kylin/kylin_metadata/’ by default) includes intermediate files (will be GC) and Cuboid data (won‘t be GC). The Cuboid data is kept for the further segments’ merge, as Kylin couldn‘t merge from HBase. If you’re sure those segments won't be merged, you can move them to other paths or even delete.
Please pay attention to the “resources” sub-folder under ‘hdfs-working-dir’, which persists some big metadata files like dictionaries and lookup tables' snapshots. They shouldn't be moved.