| Building SensSoft Docker Containers |
| ======================================= |
| *Last Tested (on macOS Mojave) 23 JAN 2019* |
| |
| Prerequisites |
| ------------- |
| |
| 1. Install [``Docker``](http://docker.com) on your machine. Requires Docker 1.7 and above. |
| |
| 1. Install docker-compose. Full instructions can be found [``here``](https://docs.docker.com/compose/install/). |
| If you install Docker through the [``Desktop bundle``](https://www.docker.com/products/docker-desktop), docker-compose is included. |
| |
| Single Node Deployment |
| ---------------------- |
| |
| The single node deployment steps below will build a single-node logging server on a single |
| machine. This is suitable for demonstrations and very limited data collections. Please |
| refer to our [``Kubernetes``](https://github.com/apache/incubator-senssoft/tree/master/kubernetes) guide for deployments that scale to your needs. In some cases, |
| you may be able to use single node containers within scaling services (e.g., AWS EBS), but this |
| requires special configuration. Please reach out to us at [our dev list](mailto:dev@senssoft.incubator.apache.org) for recommendations. |
| |
| 1. Create docker-machine instance. |
| **Note**: If using Docker Desktop bundle, there is a known bug in |
| the bundled version of virtualbox that will prevent a successful docker-machine creation. |
| Before installation, check that virtualbox version is at least 5.2. [``Reinstall virtualbox``](https://www.virtualbox.org/wiki/Downloads), if needed. |
| |
| ```bash |
| $ docker-machine create --virtualbox-memory 3072 --virtualbox-cpu-count 2 senssoft |
| ``` |
| |
| 1. Before launching the Docker containers, ensure your ``vm_max_map_count`` |
| kernel setting is set to at least 262144. |
| Visit [``Running Elasticsearch in Production mode``](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/docker.html#docker-cli-run-prod-mode) for OS specific instructions. |
| |
| ```bash |
| # Example for Linux systems |
| $ docker-machine ssh senssoft sudo sysctl -w vm.max_map_count=262144 |
| ``` |
| |
| 1. Create externel docker network to enable system monitoring. Only enable if running |
| the Elasticsearch 6.5.4 configuration (single and cluster mode) |
| |
| ```bash |
| $ docker network create esnet |
| ``` |
| |
| 1. Start Elasticsearch 6.2.2 (Deprecated) or 6.5.4 (Recommended) Give Elasticsearch about 1-2 minutes to start before confirming its state. |
| |
| ```bash |
| #start Elasticsearch v6.2.2 (Deprecated) |
| $ docker-compose -f docker-compose.single-6.2.2.yml up -d elasticsearch |
| |
| or |
| |
| #start Elasticsearch v6.5.4 (Recommended) |
| $ docker-compose up -d elasticsearch |
| ``` |
| |
| 1. Confirm state: |
| ```bash |
| # if senssoft vm is remote |
| $ docker-machine ssh senssoft curl -XGET http://localhost:9200/_cluster/health?pretty |
| # if senssoft virtual machine is running on your local machine, no need for ssh, instead: |
| $ curl -XGET http://localhost:9200/_cluster/health?pretty |
| #output should look like this: |
| "cluster_name" : "SensSoft", |
| "status" : "yellow", |
| "timed_out" : false, |
| "number_of_nodes" : 1, |
| "number_of_data_nodes" : 1, |
| "active_primary_shards" : 0, |
| "active_shards" : 0, |
| "relocating_shards" : 0, |
| "initializing_shards" : 0, |
| "unassigned_shards" : 0, |
| "delayed_unassigned_shards" : 0, |
| "number_of_pending_tasks" : 0, |
| "number_of_in_flight_fetch" : 0, |
| "task_max_waiting_in_queue_millis" : 0, |
| "active_shards_percent_as_number" : 100.0 |
| ``` |
| |
| 1. Launch logging server. Give Logstash about 2 minutes to start before confirming |
| its state. |
| |
| ```bash |
| $ docker-compose up -d logstash |
| # if senssoft vm is remote |
| $ docker-machine ssh senssoft curl -XGET http://localhost:8100 |
| # if senssoft virtual machine is running on your local machine, no need for ssh, instead: |
| $ curl -XGET http://localhost:8100 |
| #ouput should look like this: |
| ok |
| ``` |
| |
| 1. Before Kibana can be used, we will need to generate some data. We provide an example instrumented website to assist. |
| |
| ```bash |
| $ docker-compose up -d site |
| # for remote users, forwards port to localhost |
| $ ssh docker@$(docker-machine ip senssoft) -L 8080:localhost:8080 |
| ``` |
| |
| Visit `http://localhost:8080` and you will see Apache SensSoft's home page. |
| |
| Note that the `userale` index uses dynamic mapping configurations for many fields. This means that if no valid data exists for |
| certain fields in the logs you collect at this step, Kibana won't know to map these fields to data types (e.g., string, text, |
| boolean, etc.). This can prevent certain dashboards and visualizations from appropriately displaying log aggregations. It is worth |
| 1-2 mins collecting some UserALE.js data in whichever way best emulates your use-case: from the same website, the [``UserALE.js example utilty``](https://github.com/apache/incubator-senssoft-useralejs/tree/SENSSOFT-192/example), or the [``UserALE.js Web Extension``](https://github.com/apache/incubator-senssoft-useralejs/tree/SENSSOFT-192/src/UserALEWebExtension). If you run into issues with data fields or visualizations, see the `Having Issues?` section below. |
| |
| 1. Launch Kibana. Give Kibana about 2-5 minutes to start before accessing |
| `http://localhost:5601`. |
| |
| ```bash |
| $ docker-compose up -d kibana |
| # for remote users, forwards port to localhost |
| $ ssh docker@$(docker-machine ip senssoft) -L 5601:localhost:5601 |
| ``` |
| |
| 1. Register an index in Kibana to see the logs: |
| |
| Goto: Management -> Index Patterns and enter `userale` in the Index pattern box. |
| Choose `clientTime` in the drop down `Time Filter field name` field. |
| |
| ![alt text][configure_index] |
| |
| 1. Load example Dashboard and Visualizations under docker/kibana/. |
| |
| Goto: Management -> Saved Objects and select the `Import` button. Import the |
| `Apache SensSoft Visualizations.json`, `Drill-Down Search.json`, `Apache SensSoft Page Usage Dashboard.json`, and `Apache SensSoft User Access Dashboard.json` files from the "Saved Objects" folder in the kibana directory. |
| |
| ![alt text][management] |
| |
| Set `userale` index if Kibana detects conflicts when you load visualizations and searches. |
| |
| ![alt text][viz_import] |
| |
| Once that is complete, navigate to the `Dashboard` view in Kibana and click the |
| `Apache SensSoft Page Usage Dashboard` object. |
| |
| ![alt text][dashboard] |
| |
| 1. To see container health metrics, launch Metricbeat: |
| |
| ```bash |
| $ docker-compose up -d metricbeat |
| ``` |
| |
| Once the container is running, metricbeat dashboards will automatically load in Kibana. Navigate to the `Container Dashboard`. |
| |
| ![alt text][metrics] |
| |
| 1. To stop all containers. |
| ```bash |
| $ docker-compose stop |
| ``` |
| |
| 1. To kill the senssoft machine. |
| ```bash |
| $ docker-machine rm senssoft |
| ``` |
| |
| If running on a single machine, on reboot or restart your senssoft machine is available, but |
| in a "stopped" state. You'll need to restart the machine, then you'll need to use docker- |
| compose up commands above to restart containers. |
| |
| 1. Restart the senssoft machine. |
| ```bash |
| $ docker-machine start senssoft |
| $ docker-machine ls #confirm state |
| #output should look like this: |
| NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS |
| senssoft - virtualbox Running tcp://192.168.99.100:2376 v18.09.0 |
| ``` |
| |
| Multi-Node Deployment on a Single Machine |
| ----------------------------------------- |
| |
| <aside class="warning"> |
| Starting an elasticsearch cluster is not recommended on a single server. This |
| is just for demonstration purposes only. Please refer to our [Kubernetes] guide to |
| deploy an Elasticsearch cluster for an enterprise scale logging capability. |
| </aside> |
| |
| 1. Create docker-machine instance |
| ```bash |
| docker-machine create --virtualbox-memory 2048 --virtualbox-cpu-count 2 senssoft |
| ``` |
| |
| 1. Before launching the Docker containers, ensure your ``vm_max_map_count`` |
| kernel setting is set to at least 262144. |
| Visit [``Running Elasticsearch in Production mode``](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/docker.html#docker-cli-run-prod-mode) for OS specific instructions. |
| |
| ```bash |
| # Example for Linux systems |
| $ docker-machine ssh senssoft sudo sysctl -w vm.max_map_count=262144 |
| ``` |
| |
| 1. Create externel docker network to enable system monitoring. Only enable if running |
| the Elasticsearch 6.2.2 configuration (single and cluster mode) |
| |
| ```bash |
| $ docker network create esnet |
| ``` |
| |
| 1. Start Elasticsearch cluster: |
| |
| ```bash |
| $ docker-compose -f docker-compose.cluster.yml up -d --scale elasticsearch=3 elasticsearch |
| $ docker-compose -f docker-compose.cluster.yml up -d loadbalancer |
| ``` |
| |
| The loadbalancer node exposes port 9200 on localhost and is the only node |
| that has HTTP enabled. Services such as Kibana and Logstash connect to the |
| loadbalancer node directly. Loadbalancer accepts requests from Kibana and Logstash |
| and balances them across the elasticsearch worker nodes. The elasticsearch |
| worker nodes communicate to each other and the loadbalancer via TCP on port 9300. |
| |
| 1. Confirm cluster state: |
| ```bash |
| $ docker-machine ssh senssoft curl -XGET http://localhost:9200/_cluster/health\?pretty |
| # if senssoft virtual machine is running on your local machine, no need for ssh, instead: |
| $ curl -XGET http://localhost:9200/_cluster/health?pretty |
| #output should look like this: |
| { |
| "cluster_name" : "SensSoft", |
| "status" : "green", |
| "timed_out" : false, |
| "number_of_nodes" : 4, |
| "number_of_data_nodes" : 3, |
| "active_primary_shards" : 0, |
| "active_shards" : 0, |
| "relocating_shards" : 0, |
| "initializing_shards" : 0, |
| "unassigned_shards" : 0, |
| "delayed_unassigned_shards" : 0, |
| "number_of_pending_tasks" : 0, |
| "number_of_in_flight_fetch" : 0, |
| "task_max_waiting_in_queue_millis" : 0, |
| "active_shards_percent_as_number" : 100.0 |
| } |
| ``` |
| Confirm that the `number_of_nodes` is 4 and `number_of_data_nodes` is 3. |
| |
| 1. Follow remaining instructions in [Single Node Deployment] starting at #6. |
| |
| 1. To stop all containers. |
| ```bash |
| $ docker-compose stop |
| ``` |
| |
| 1. To kill the "senssoft" machine. |
| ```bash |
| $ docker-machine rm senssoft |
| ``` |
| |
| If running on a single machine, on reboot or restart your senssoft machine is available, but |
| in a "stopped" state. You'll need to restart the machine, then you'll need to use docker- |
| compose up commands above to restart containers. |
| |
| 1. Restart the senssoft machine. |
| ```bash |
| $ docker-machine start senssoft |
| $ docker-machine ls #confirm state |
| #output should look like this: |
| NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS |
| senssoft - virtualbox Running tcp://192.168.99.100:2376 v18.09.0 |
| ``` |
| |
| Having Issues? |
| -------------- |
| 1. Check out the docker-compose logs for the service(s) that are having issues. |
| |
| ```bash |
| $ docker-compose ps |
| ``` |
| |
| 1. If you find containers failing, you may have duplicate or dangling images! |
| |
| This can happen if you've played around with multiple machines and builds of the containers |
| on the same machine. Visit [``this excellent how to guide for removing images, containers, |
| and volumes``](https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes). Remove any duplicate images and rebuild the containers. |
| |
| 1. If you find that the `logstash` or `site` containers don't respond immediately, give them a few minutes. |
| In the case of the `site` container, you might try giving it a kick if its taking more than three minutes to load in browser" |
| |
| ```bash |
| #after loading container, confirming status is "up", and localhost:8080 still isn't loading, bring the container down |
| $ docker-compose kill site |
| |
| # then bring it back up, and see if it loads |
| $ docker-compose up -d site |
| ``` |
| |
| 1. If you find that Apache SensSoft Kibana Dashboards aren't loading, or Apache UserALE.js log fields in Kibana's `Discover` view |
| appear with a warning icon, it could be that you didn't collect logs with valid data for those fields prior to loading the userale |
| index in Kibana. Don't worry, your data is fine--just navigate to the Management -> Index Patterns page, and click the "refresh" button in the upper right hand of the page (Disregard the "popularity metrics" warning). This will refresh the index, making those |
| fields aggregatable, and Dashboards should render properly. |
| |
| 1. Make sure to send us the docker-compose logs to help diagnose your issues please! |
| |
| ```bash |
| $ docker-compose logs > err.dump |
| ``` |
| |
| You can attach logs directly to tickets on our [``Apache Jira board``](https://issues.apache.org/jira/issues/?jql=project+%3D+SENSSOFT+AND+component+%3D+builds) |
| |
| 1. Still having issues? Reach out to us at at [our dev list](mailto:dev@senssoft.incubator.apache.org). |
| |
| [configure_index]: ./docs/images/configure_index.png "Configure Kibana index" |
| [confirmation]: ./docs/images/confirmation.png "Confirm index pattern conflicts" |
| [dashboard]: ./docs/images/dashboard.png "Apache Senssoft Page Usage Dashboard" |
| [management]: ./docs/images/management.png "Kibana management console" |
| [metrics]: ./docs/images/DockerBeats_Dashboard.png "Metricbeat Dashboard" |
| [viz_import]: ./docs/images/viz_import.png "Visualization Import Configuration" |
| |
| Licensing |
| -------------- |
| |
| Apache SensSoft is provided under Apache License version 2.0. See LICENSE (at Master) file for more |
| details. "Software as a Sensor" is Copyright © 2016 The Charles Stark Draper Laboratory, Inc. All rights reserved. |