commit | 270dc2da9224341363ff7ea5074064a8ef2e9c17 | [log] [tgz] |
---|---|---|
author | fgreg <fgreguska+github@gmail.com> | Fri Jun 29 07:48:30 2018 -0700 |
committer | GitHub <noreply@github.com> | Fri Jun 29 07:48:30 2018 -0700 |
tree | 8f8cab5e80e1232ef0b10980763b638e807f2e6b | |
parent | d6a9b9210f02bcb4ede37a90283cb169448b2095 [diff] |
SDAP-103 Update developer documentation (#17) * Added Introduction, Quickstart, and Ningester sections to docs
The next generation cloud-based science data service platform
solr
, xd
, and kafka
Install and configure Apache Solr Cloud 5.3.x
Download JTS Topology Suite v1.13 and extract the zip.
From the exploded JTS zip, copy $JTS_ZIP/lib/jts-1.13.jar
and $JTS_ZIP/lib/jtsio-1.13.jar
into $SOLR_INSTALL_DIR/server/lib/ext
on all Solr nodes.
Configure Solr Cloud to use the /solr
chroot of zookeeper
On one of the Solr cloud nodes, upload the nexustiles
configuration (located in data-access/config/schemas/solr) as a configset
./zkcli.sh -cmd upconfig -z $ZK_SERVERS/solr -confname nexustiles -confdir /path/to/nexustiles/conf
Create a new collection with the name nexustiles. Use the nexustiles configset uploaded previously.
curl "http://<SOLR_HOST>/solr/admin/collections?action=CREATE&name=nexustiles&collection.configName=nexustiles"
Repeat steps 5 & 6 for the datasets collection.
/kafka
chroot of zookeeperxd
chrootnone
in $SPRING_XD_HOME/xd/lib
In order for streams to use the custom python and groovy scripts found in nexus-ingest, the machines that will be running Spring XD containers need to have some software installed on them.
Install Anaconda 4.0.0 with Python 2.7
Create an Anaconda environment
conda create --name nexus-xd-python-modules python
Install conda dependencies
conda install libnetcdf conda install netcdf4 conda install numpy
Install nexusproto
Install spring-xd-python
Install nexus-xd-python-modules
The nexus-messages JAR needs to be available via Maven. There are a number of ways to do this. One way is to explicitly install the dependency into the local maven cache on each XD Container node.
mvn install:install-file -DpomFile=nexus-messages-VERSION.xml -Dfile=nexus-messages-VERSION.jar