Dev Lake

Requirements

Node.js
Docker

Setup

Local

Clone this repository
Install dependencies with npm i
Install postgres and redis and startup
Create config file cp .env.sample .env. change DB_URL and REDIS_URL to your local db
Start services npm run all
You can now see the REST application running on localhost:3000

NOTE: To run the tests:

Create config file cp .env.sample .env.test
npm run test:e2e

Docker

Clone this repository
From the newly cloned repo directory, run npm run compose
Run docker-compose ps to see containers runnning.
Install dependencies with npm i
Create config file cp .env.sample .env. change DB_URL and REDIS_URL to your docker container db
Start services npm run all
You can now see the REST application running on localhost:3000

Configuration

Make a copy of .env.sample to .env
SET REDIS_URL AND DB_URL. Queue service require redis for the task queue. Plugins write datas into configed DB.

Grafana Connection For Data Visualization (https://localhost:3002)

Connect to the Grafana database: Inside ./devops/docker-compose.yml edit the environment variables as needed to connect to your local postgres instance, specifically:

GF_DATABASE_NAME
GF_DATABASE_USER
GF_DATABASE_PASSWORD

Connect the Grafana data source: Additionally to use the postgres database as data source inside grafana, ensure postgres config options are correct in ./grafana/datasources/datasource.yml, specifically:

database
user
secureJsonData/password

Usage

Add Data Source

POST localhost:3000/source
{
    "type": "Jira",
    "name": "Jira Collector",
    "options": {
        "host": "https://www.atlassian.com/",
        "email": "xx@example.com",
        "auth": "base64EncodedAuthToken"
    }
}

response:
{ source: 'source id' }

Add Data Source Task

POST  localhost:3000/sources/${source id}
{
    collector: ['Issue'],
    enricher: ['LeadTime'],
    options: {
        projects: ['ProjectName'],
        boards: ['Scrum Board Id']
    }
}

response:
{ task: 'task id' }

Waiting Task Finished

GET  localhost:3000/tasks/${task id}

response:
{
    task: {
        id: 'task id',
        status: 'finished',
        progress: {
            collector: [
                {name: 'Issue', status: 'finished'}
            ],
            enricher: [
                {name: 'LeadTime', status: 'finished'}
            ]
        }
    } 
}

Using Grafana

Login Credentials

Visit: http://localhost:3002
Username: admin
Password: admin

Provisioning a Grafana Dashboard

To save a dashboard in the lake repo and load it:

Create a dashboard in browser (visit /dashboard/new, or use sidebar)
Save dashboard (in top right of screen)
Go to dashboard settings (in top right of screen)
Click on JSON Model in sidebar
Copy code into a new .json file in /grafana/dashboards

Contributing

CONTRIBUTING.md