Google Git
Sign in
apache / incubator-devlake / refs/tags/v0.2.0
commitcfde5733f07ceec8b9060663e3915968fee7a8bb[log] [tgz]
authorNaRro <cong.wang@merico.dev>Thu Sep 30 18:27:23 2021 +0800
committerGitHub <noreply@github.com>Thu Sep 30 18:27:23 2021 +0800
treee3585de50950b6f5e4c62dcd5612a475e8e4288d
parent418543f27d03a095ac08d75c2b64ed63745bd2be [diff]
parentcd213e949ba1fd7bb48a69ef7887dbf331b9b9e7 [diff]
Merge pull request #475 from merico-dev/fix-config-ui-again

Fix config UI again
tree: e3585de50950b6f5e4c62dcd5612a475e8e4288d
  1. .github/
  2. api/
  3. cmd/
  4. config/
  5. config-ui/
  6. devops/
  7. docs/
  8. errors/
  9. grafana/
  10. logger/
  11. models/
  12. plugins/
  13. scripts/
  14. services/
  15. test/
  16. utils/
  17. .editorconfig
  18. .env.example
  19. .gitignore
  20. ARCHITECTURE.md
  21. CONTRIBUTING-zh-CN.md
  22. CONTRIBUTING.md
  23. docker-compose.yml
  24. Dockerfile
  25. go.mod
  26. go.sum
  27. LICENSE
  28. main.go
  29. Makefile
  30. README-zh-CN.md
  31. README.md
README.md

Dev Lake

PRs Welcome Discord badge Go Report Card

English中文

What is Dev Lake?

Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data throughout the software development life cycle (SDLC) for engineering teams.

Why choose Dev Lake?

  1. Supports various data sources (Gitlab, Jira) and more are being added all the time
  2. Relevant, customizable data metrics ready to view as visual charts
  3. Easily build and view new charts and dashboards with Grafana
  4. Easy-to-setup via Docker
  5. Extensible plugin system to add your own data collectors
  6. Designed to process enterprise-scale data

Contents

SectionDescriptionDocumentation Link
Data SourcesLinks to specific plugin usage & detailsView Section
User SetupSteps to run the project as a userView Section
Developer SetupHow to setup dev environmentView Section
TestsCommands for running testsView Section
GrafanaHow to visualize the dataView Section
Build a PluginDetails on how to make your ownLink
Add Plugin MetricsGuide to adding plugin metricsLink
ContributingHow to contribute to this repoLink

Data Sources We Currently Support

Below is a list of data source plugins used to collect & enrich data from specific sources. Each have a README.md file with basic setup, troubleshooting and metrics info.

For more information on building a new data source plugin see Build a Plugin.

SectionSection InfoDocs
JiraMetrics, Generating API Token, Find Board IDLink
GitlabMetrics, Generating API Token, Find Project IDLink
JenkinsMetrics, Generating API TokenLink

User setup

NOTE: If you only plan to run the product, this is the only section you should need NOTE: Commands written like this are to be run in your terminal

Required Packages to Install

NOTE: After installing docker, you may need to run the docker application and restart your terminal

Commands to run in your terminal

  1. Clone repository

    git clone https://github.com/merico-dev/lake.git devlake
cd devlake
cp .env.example .env

  2. Start Docker on your machine and then you can run docker-compose up -d config-ui to start up the configuration interface

    For more info on how to configure plugins, please refer to the data source plugins section

  3. Visit localhost:4000 to setup configuration files

    • Finish the configuration on the main configuration page (localhost:4000)
    • Navigate to desired plugins pages on the sidebar under “Plugins”, e.g. Jira, Gitlab, Jenkins etc. Enter in required information for those plugins
    • Submit the form to update the values by clicking on the Save Config button on each form page

  4. Run docker-compose up -d to start up the other services

  5. Visit localhost:4000/triggers to trigger data collection

    Please replace your gitlab projectId and jira boardId in the request body. Click the Trigger Collection button. This can take up to 20 minutes for large projects. (gitlab 10k+ commits or jira 5k+ issues)

  6. Click Go to grafana button when done (username: admin, password: admin). The button will be shown on the Trigger Collection page when data collection has finished.

Setup cron job

Commonly, we have requirement to synchorize data periodly. We providered a tool called lake-cli to meet that requirement. Check lake-cli usage at here.

Otherwise, if you just want to use the cron job, please check docker-compose version at here

Developer Setup

Requirements

  • Docker
  • Golang
  • Make
    • Mac (Already installed)
    • Windows: Download
    • Ubuntu: sudo apt-get install build-essential

How to setup dev environment

  1. Navigate to where you would like to install this project and clone the repository

    git clone https://github.com/merico-dev/lake.git
cd lake

  2. Install go packages

    make install

  3. Copy sample config files to new local file

    cp .env.example .env

  4. Start the docker containers

    Make sure the docker application is running before this step

    make compose

  5. Run the project

    make dev

  6. You can now post to /task to create a jira task. This will collect data from Jira

    curl -XPOST 'localhost:8080/task' \
-H 'Content-Type: application/json' \
-d '[{
    "plugin": "jira",
    "options": {
        "boardId": 8
    }
}]'

  7. Visualize the data in the Grafana Dashboard

    From here you can see existing data visualized from collected & enriched data

    • Navigate to http://localhost:3002 (username: admin, password: admin)
    • You can also create/modify existing/save dashboards to lake
    • For more info on working with Grafana in Dev Lake see Grafana Doc

Tests

To run the tests: make test

Grafana

We use Grafana as a visualization tool to build charts for the data stored in our database. Using SQL queries we can add panels to build, save, and edit customized dashboards.

All the details on provisioning, and customizing a dashboard can be found in the Grafana Doc

Contributing

CONTRIBUTING.md

Need help?

Message us on Discord