feat: gitlab extractors and convertors support incremental mode (#7997) * feat: gitlab mr_extractor support incremental sync * feat: incr mode transformation support for deployment/issue/job/mr and others * feat: removed mr_enricher and all gitlab transformers support incr-mode * fix: some gitlab subtasks are missing connection in left join clause * fix: test cases failed due to gitlab transformers support incr-mode * fix: linting * fix: do not collect all accounts from jihulab.com * fix: typo * fix: gitlab mr comments won't be converted to domain layer till next run * feat: improve gitlab MR comments/commits collection performance * fix: gitlab issues/mrs child records are not deleted * docs: update stateful extractor doc * fix: gitlab mr detail test * refactor: unify stateful extractor and convertor helper * docs: update stateful extractor/convertor doc * refactor: remove useless type hint * fix: shoud not be deleting records not extracted by current extractor * fix: jira issue extractor should not deleting sprint_issue * refactor: remove commit related subtasks * fix: remove commit conversion e2e test * feat(gitlab): update ExtractApiChildPipelines (#8016) * fix: retransform should run in fullsync mode * fix: gitlab issue assignees are not being converted --------- Co-authored-by: Lynwee <1507509064@qq.com>
Apache DevLake is an open-source dev data platform that ingests, analyzes, and visualizes the fragmented data from DevOps tools to extract insights for engineering excellence, developer experience, and community growth.
Apache DevLake is used by Engineering Leads, Open Source Software Maintainers and development teams looking to make better sense of their development process and to bring a more data-driven approach to their own practices. You can ask Apache DevLake many questions regarding your development process. Just connect and query.
The main way you interact with DevLake is through the integrated dashboards powered by Grafana.
Dashboards for Engineering Leads
Dashboards for OSS Maintainers
DevLake supports connections to many popular development tools, including GitHub, GitLab, Jenkins, Jira, Sonarqube and more. Here you can find all data sources supported by DevLake, their scopes, supported versions and more!
You can set up Apache DevLake by following our step-by-step instructions for either Docker Compose or Helm. Feel free to ask the community if you get stuck at any point.
Please see detailed usage instructions. Here's an overview on how to get started using DevLake.
Install using either Docker Compose or Helm.
The DevLake Configuration UI will guide you through the process (a Blueprint) to define the data connections, data scope, transformation and sync frequency of the data you wish to collect.
You can track the progress of the Blueprint you have just set up.
Once the first run of the Blueprint is completed, you can view the corresponding dashboards.
If the pre-built dashboards are limited for your use cases, you can always customize or create your own metrics or dashboards with SQL.
Please read the contribution guidelines before you make contribution. The following docs list the resources you might need to know after you decided to make contribution.
If you plan to contribute code to Apache DevLake, we have instructions on how to get started with setting up your Development environemtn.
One of the best ways to get started contributing is by improving DevLake's documentation.
This project is licensed under Apache License 2.0 - see the LICENSE file for details.
