Apache DevLake(incubating) supports the following data sources. The data from each data source is collected with one or more plugins. Detailed plugin docs can be found here.
Data Source | Domain(s) | Supported Versions | Config UI Availability | Triggered Plugins | Collection Mode |
---|---|---|---|---|---|
GitHub | Source Code Management, Code Review, Issue Tracking, CI/CD (GitHub Actions) | Cloud | Available | github , gitextractor | Incremental Sync |
GitLab | Source Code Management, Code Review, Issue Tracking, CI/CD (GitLab CI) | Cloud, Community Edition 11+ | Available | gitlab , gitextractor | Full Refresh, Incremental Sync(for issues ,MRs ) |
Jira | Issue Tracking | Cloud, Server/Data Center 7.x, 8.x | Available | jira | Full Refresh, Incremental Sync(for issues and related) |
Jenkins | CI/CD | 2.263.x+ | Available | jenkins | Incremental Sync |
BitBucket (Beta) | Source Code Management, Code Review | Cloud | Advanced Mode Available | bitbucket , gitextractor | Full Refresh |
TAPD (Beta) | Issue Tracking | Cloud | Advanced Mode Available | tapd | Full Refresh, Incremental Sync(for stories , bugs , tasks ) |
Teambition (Beta) | Issue Tracking | Cloud | Advanced Mode Available | teambition | Full Refresh |
Zentao (Beta) | Issue Tracking | v17.x, v18.x | Advanced Mode Available | zentao | Full Refresh |
Gitee (WIP) | Source Code Management, Code Review, Issue Tracking | Cloud | Not Available | gitee , gitextractor | Full Refresh, Incremental Sync(for issues ,MRs ) |
PagerDuty (WIP) | Issue Tracking | Cloud | Not Available | pagerduty | Full Refresh |
Feishu (WIP) | Calendar | Cloud | Not Available | feishu | Full Refresh |
AE | Source Code Management | On-prem | Not Available | ae | Full Refresh |
Sonarqube | CODE QUALITY | SonarQube v8.x, v9.x | Available | sonarqube | Full Refresh |
Bamboo CI(WIP) | CI/CD | v6.8.1 | Not Available | bamboo | Full Refresh |
Azure Devops (Beta) | CI/CD, Source Code Management, Code Review | Cloud | Available | azuredevops , gitextractor | Full Refresh |
This table shows the entities collected by each plugin. Domain layer entities in this table are consistent with the entities here. ✅ : Collect by default. 💪 : Collect not by default. You need to add the corresponding subtasks to collect these entities in the advanced mode.
Domain Layer Entities | ae | dora | gitextractor | incoming webhook | github | gitlab | jenkins | jira | refdiff | tapd | sonarqube | bamboo | azuredevops |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
accounts | ✅ | ✅ | ✅ | ✅ | ✅ | ||||||||
board_issues | ✅ | ✅ | ✅ | ✅ | |||||||||
board_repos | ✅ | ✅ | ✅ | ||||||||||
board_sprints | ✅ | ✅ | ✅ | ||||||||||
boards | ✅ | ✅ | ✅ | ✅ | |||||||||
cicd_pipeline_commits | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
cicd_pipelines | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
cicd_scopes | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
cicd_tasks | ✅ | 💪 | ✅ | ✅ | ✅ | ✅ | ✅ | ||||||
commit_file_components | ✅ | ||||||||||||
commit_files | ✅ | ||||||||||||
commit_line_change | ✅ | ||||||||||||
commit_parents | ✅ | ||||||||||||
commits | ✅ | ✅ | 💪 | 💪 | |||||||||
commits_diffs | ✅ | ||||||||||||
components | |||||||||||||
finished_commits_diffs | |||||||||||||
issue_changelogs | ✅ | ✅ | |||||||||||
issue_comments | ✅ | ✅ | |||||||||||
issue_commits | ✅ | ✅ | |||||||||||
issue_labels | ✅ | ✅ | ✅ | ||||||||||
issue_repo_commits | ✅ | ||||||||||||
issue_worklogs | ✅ | ✅ | |||||||||||
issues | ✅ | ✅ | ✅ | ||||||||||
project_issue_metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ||||||||
project_mapping | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
project_metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
project_pr_metrics | ✅ | ✅ | ✅ | ✅ | |||||||||
projects | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||
pull_request_comments | ✅ | ✅ | |||||||||||
pull_request_commits | ✅ | ✅ | ✅ | ||||||||||
pull_request_issues | ✅ | ||||||||||||
pull_request_labels | ✅ | ✅ | |||||||||||
pull_requests | ✅ | ✅ | ✅ | ||||||||||
ref_commits | ✅ | ||||||||||||
refs | ✅ | ✅ | |||||||||||
refs_issues_diffs | ✅ | ||||||||||||
ref_pr_cherry_picks | ✅ | ||||||||||||
repo_commits | ✅ | 💪 | 💪 | ||||||||||
repo_snapshot | ✅ | ||||||||||||
repos | ✅ | ✅ | |||||||||||
sprint_issues | ✅ | ✅ | ✅ | ||||||||||
sprints | ✅ | ✅ | ✅ | ||||||||||
team_users | |||||||||||||
teams | |||||||||||||
user_account | |||||||||||||
users | ✅ | ✅ | |||||||||||
cq_projects | ✅ | ✅ | ✅ | ||||||||||
cq_issues | ✅ | ✅ | ✅ | ||||||||||
cq_issue_code_blocks | ✅ | ✅ | ✅ | ||||||||||
cq_file_metrics | ✅ | ✅ | ✅ |
bold: means it may collect slowly.
*bold*: means it may collect very slowly.
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectStatusMeta | 1 | - | - |
CollectProjectsMeta | <10 | ❌ | - |
CollectIssueTypesMeta | <10 | ❌ | - |
CollectIssuesMeta | <10^4 | ✅ | ✅ |
CollectIssueChangelogsMeta | 1000~10^5 | ✅ | ✅ |
CollectAccountsMeta | <10^3 | ❌ | ❌ |
CollectWorklogsMeta | 1000~10^5 | ✅ | ✅ |
CollectRemotelinksMeta | 1000~10^5 | ✅ | ✅ |
CollectSprintsMeta | <100 | ❌ | ❌ |
CollectEpicsMeta | <100 | ❌ | ✅ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectApiBuildsMeta | ≈100 | ❌ | ❌ |
CollectApiStagesMeta | ≈10^4 | ❌ | ✅ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectApiIssuesMeta | <10^4 | ✅ | ✅ |
CollectApiMergeRequestsMeta | <10^3 | ✅ | ✅ |
CollectApiMrNotesMeta | <10^5 | ❌ | ✅ |
CollectApiMrCommitsMeta | <10^5 | ❌ | ✅ |
CollectApiPipelinesMeta | <10^4 | ✅ | ❌ |
CollectApiJobsMeta | <10^5 | ❌ | ✅ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
--------------------------------- | Common | ----------------------- | |
CollectMilestonesMeta | ≈10 | ✅ | ❌ |
CollectRunsMeta | <10^4 | ✅ | ✅ |
CollectApiCommentsMeta | 400 (max page that GitHub supports) | ✅ | ✅ |
CollectApiEventsMeta | 400 (max page that GitHub supports) | ❌ | ❌ |
CollectApiPullRequestReviewsMeta | <10^5 | ✅ | ✅ |
--------------------------------- | Graphql Only (Default) | ----------------------- | |
CollectIssueMeta | ≈10^4 | ❌ | ✅ |
CollectPrMeta | ≈10^3 | ❌ | ✅ |
CollectCheckRunMeta | <10^4 | ❌ | ✅ |
CollectAccountMeta | ≈10^2 | ❌ | - |
--------------------------------- | Restful Only (Not by Default) | ----------------------- | |
CollectApiIssuesMeta | ≈10^4 | ✅ | ❌ |
CollectApiPullRequestsMeta | ≈10^2 | ❌ | ❌ |
CollectApiPullRequestCommitsMeta | ≈10^4 | ✅ | ✅ |
CollectApiPrReviewCommentsMeta | ≈10^4 | ✅ | ✅ |
CollectAccountsMeta | ≈10^4 | ❌ | ❌ |
CollectAccountOrgMeta | ≈10^4 | ❌ | ❌ |
CollectJobsMeta | <10^6 | ❌ | ✅ |
CollectApiCommitsMeta | Not enabled | - | - |
CollectApiCommitStatsMeta | Not enabled | - | - |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectMeetingTopUserItemMeta | ≈10^3 | ❌ | ✅ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
1 | ❌ | ❌ | |
CollectApiPullRequestsMeta | ≈10^3 | ❌ | ❌ |
CollectApiIssuesMeta | ≈10^4 | ❌ | ❌ |
CollectApiPrCommentsMeta | ≈10^5 | ❌ | ❌ |
*CollectApiIssueCommentsMeta* | ≈10^6 | ❌ | ❌ |
CollectApiPipelinesMeta | <10^4 | ❌ | ❌ |
CollectApiDeploymentsMeta | <10^2 | ❌ | ❌ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
1 | ❌ | ❌ | |
CollectApiPullRequestsMeta | ≈10^3 | ✅ | ❌ |
CollectApiIssuesMeta | ≈10^4 | ✅ | ❌ |
CollectCommitsMeta? | ≈10^4 | ✅ | ❌ |
CollectApiPrCommentsMeta | ≈10^5 | ❌ | ❌ |
*CollectApiIssueCommentsMeta* | ≈10^6 | ✅ | ❌ |
CollectApiPullRequestCommitsMeta | ≈10^5 | ❌ | ❌ |
CollectApiPullRequestReviewsMeta | ≈10^5 | ❌ | ❌ |
*CollectApiCommitStatsMeta* | ≈10^6 (Not enable) | ❌ | ❌ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectAccounts | <10^4 | ❌ | ❌ |
CollectIssues | <10^4 | ❌ | ❌ |
CollectHotspots | <10^4 | ❌ | ❌ |
CollectFilemetrics | <10^4 | ❌ | ❌ |
CollectAdditionalFilemetrics | <10^4 | ❌ | ❌ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectPlanMeta | <10^4 | ❌ | ❌ |
CollectJobMeta | <10^5 | ❌ | ❌ |
CollectPlanBuildMeta | <10^6 | ❌ | ❌ |
CollectJobBuildMeta | <10^6 | ❌ | ❌ |
CollectDeployMeta | 1 | ❌ | ❌ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectExecutionMeta | <10^3 | ❌ | ❌ |
CollectStoryMeta | <10^4 | ❌ | ❌ |
CollectBugMeta | <10^4 | ❌ | ❌ |
CollectTaskMeta | <10^4 | ❌ | ❌ |
CollectAccountMeta | ≈10^2 | ❌ | ❌ |
CollectDepartmentMeta | ≈10^2 | ❌ | ❌ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectWorkitemTypesMeta | ≈10 | - | - |
CollectStoryCustomFieldsMeta | ≈10 | - | - |
CollectTaskCustomFieldsMeta | ≈10 | - | - |
CollectBugCustomFieldsMeta | ≈10 | - | - |
CollectStoryCategoriesMeta | ≈10 | - | - |
CollectStoryStatusMeta | ≈10 | - | - |
CollectStoryStatusLastStepMeta | ≈10 | - | - |
CollectBugStatusMeta | ≈10 | - | - |
CollectBugStatusLastStepMeta | ≈10 | - | - |
CollectAccountsMeta | ≈10^3 | ❌ | ❌ |
CollectIterationMeta | ≈10^4 | ✅ | ✅ |
CollectStoryMeta | ≈10^4 | ✅ | ✅ |
CollectBugMeta | ≈10^4 | ✅ | ✅ |
CollectTaskMeta | ≈10^4 | ✅ | ✅ |
CollectBugChangelogMeta | ≈10^6 | ✅ | ✅ |
CollectStoryChangelogMeta | ≈10^6 | ✅ | ✅ |
CollectTaskChangelogMeta | ≈10^6 | ✅ | ✅ |
CollectWorklogMeta | ≈10^6 | ✅ | ✅ |
CollectBugCommitMeta | ≈10^6 | ✅ | ✅ |
CollectStoryCommitMeta | ≈10^6 | ✅ | ✅ |
CollectTaskCommitMeta | ≈10^6 | ✅ | ✅ |
CollectStoryBugMeta | ≈10^6 | ✅ | ✅ |
Subtask Name | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
---|---|---|---|
CollectBuilds | <10^3 | ❌ | ❌ |
CollectJobs | <10^4 | ❌ | ❌ |
CollectPullRequests | <10^3 | ❌ | ❌ |
CollectPullRequestCommits | <10^4 | ❌ | ❌ |