title: “Supported Data Sources” description: > Data sources that DevLake supports sidebar_position: 5

Data Sources and Data Plugins

Apache DevLake(incubating) supports the following data sources. The data from each data source is collected with one or more plugins. Detailed plugin docs can be found here.

Data SourceDomain(s)Supported VersionsConfig UI AvailabilityTriggered PluginsCollection Mode
GitHubSource Code Management, Code Review, Issue Tracking, CI/CD (GitHub Actions)CloudAvailablegithub, gitextractorIncremental Sync
GitLabSource Code Management, Code Review, Issue Tracking, CI/CD (GitLab CI)Cloud, Community Edition 11+Availablegitlab, gitextractorFull Refresh, Incremental Sync(for issues,MRs)
JiraIssue TrackingCloud, Server/Data Center 7.x, 8.xAvailablejiraFull Refresh, Incremental Sync(for issues and related)
JenkinsCI/CD2.263.x+AvailablejenkinsIncremental Sync
BitBucket (Beta)Source Code Management, Code ReviewCloudAdvanced Mode Availablebitbucket, gitextractorFull Refresh
TAPD (Beta)Issue TrackingCloudAdvanced Mode AvailabletapdFull Refresh, Incremental Sync(for stories, bugs, tasks)
Teambition (Beta)Issue TrackingCloudAdvanced Mode AvailableteambitionFull Refresh
Zentao (Beta)Issue Trackingv17.x, v18.xAdvanced Mode AvailablezentaoFull Refresh
Gitee (WIP)Source Code Management, Code Review, Issue TrackingCloudNot Availablegitee, gitextractorFull Refresh, Incremental Sync(for issues,MRs)
PagerDutyIssue TrackingCloudAvailablepagerdutyIncremental Sync
OpsgenieIssue TrackingCloudAvailableopsgenieFull Refresh (for users,teams), Incremental Sync (for issues)
Feishu (WIP)CalendarCloudNot AvailablefeishuFull Refresh
AESource Code ManagementOn-premNot AvailableaeFull Refresh
SonarqubeCODE QUALITYSonarQube v8.x, v9.xAvailablesonarqubeFull Refresh
Bamboo CI(WIP)CI/CDv6.8.1Not AvailablebambooFull Refresh
Azure Devops (Beta)CI/CD, Source Code Management, Code ReviewCloudAvailableazuredevops, gitextractorFull Refresh
CircleCICI/CDCloudAvailablecircleciFull Refresh

Data Collection Scope By Each Plugin

This table shows the entities collected by each plugin. Domain layer entities in this table are consistent with the entities here. ✅ : Collect by default. 💪 : Collect not by default. You need to add the corresponding subtasks to collect these entities in the advanced mode.

Domain Layer Entitiesaedoragitextractorincoming webhookgithubgitlabjenkinsjirarefdifftapdsonarqubebambooazuredevopsopsgeniecircleci
accounts✅✅✅✅✅✅
board_issues✅✅✅✅✅
board_repos✅✅
board_sprints✅✅✅
boards✅✅✅✅✅
cicd_pipeline_commits✅✅✅✅✅✅✅
cicd_pipelines✅✅✅✅✅✅✅
cicd_scopes✅✅✅✅✅✅✅
cicd_tasks✅💪✅✅✅✅✅✅
commit_file_components✅
commit_files✅
commit_line_change✅
commit_parents✅
commits✅✅💪💪
commits_diffs✅
components
finished_commits_diffs
issue_changelogs✅✅
issue_comments✅✅
issue_commits✅✅
issue_labels✅✅✅
issue_repo_commits✅
issue_worklogs✅✅
issues✅✅✅✅
project_issue_metrics✅✅✅✅✅
project_mapping✅✅✅✅✅✅
project_metrics✅✅✅✅✅✅
project_pr_metrics✅✅✅✅
project✅✅✅✅✅✅✅
pull_request_comments✅✅
pull_request_commits✅✅✅
pull_request_issues✅
pull_request_labels✅✅
pull_requests✅✅✅
ref_commits✅
refs✅✅
refs_issues_diffs✅
ref_pr_cherry_picks✅
repo_commits✅💪💪
repo_snapshot✅
repos✅✅
sprint_issues✅✅✅
sprints✅✅✅
team_users
teams✅
user_account
users✅✅✅
cq_projects✅
cq_issues✅
cq_issue_code_blocks✅
cq_file_metrics✅

Data Sync Policy

bold: means it may collect slowly.

*bold*: means it may collect very slowly.

Jira

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectStatusMeta1--
CollectProjectsMeta<10❌-
CollectIssueTypesMeta<10❌-
CollectIssuesMeta<10^4✅✅
CollectIssueChangelogsMeta1000~10^5✅✅
CollectAccountsMeta<10^3❌❌
CollectWorklogsMeta1000~10^5✅✅
CollectRemotelinksMeta1000~10^5✅✅
CollectSprintsMeta<100❌❌
CollectEpicsMeta<100❌✅

Jenkins

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiBuildsMeta≈100❌❌
CollectApiStagesMeta≈10^4❌✅

GitLab

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiIssuesMeta<10^4✅✅
CollectApiMergeRequestsMeta<10^3✅✅
CollectApiMrNotesMeta<10^5❌✅
CollectApiMrCommitsMeta<10^5❌✅
CollectApiPipelinesMeta<10^4✅❌
CollectApiJobsMeta<10^5❌✅

Github

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
---------------------------------Common-----------------------
CollectMilestonesMeta≈10✅❌
CollectRunsMeta<10^4✅✅
CollectApiCommentsMeta400 (max page that GitHub supports)✅✅
CollectApiEventsMeta400 (max page that GitHub supports)❌❌
CollectApiPullRequestReviewsMeta<10^5✅✅
---------------------------------Graphql Only (Default)-----------------------
CollectIssueMeta≈10^4❌✅
CollectPrMeta≈10^3❌✅
CollectCheckRunMeta<10^4❌✅
CollectAccountMeta≈10^2❌-
---------------------------------Restful Only (Not by Default)-----------------------
CollectApiIssuesMeta≈10^4✅❌
CollectApiPullRequestsMeta≈10^2❌❌
CollectApiPullRequestCommitsMeta≈10^4✅✅
CollectApiPrReviewCommentsMeta≈10^4✅✅
CollectAccountsMeta≈10^4❌❌
CollectAccountOrgMeta≈10^4❌❌
CollectJobsMeta<10^6❌✅
CollectApiCommitsMetaNot enabled--
CollectApiCommitStatsMetaNot enabled--

Feishu

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectMeetingTopUserItemMeta≈10^3❌✅

Bitbucket

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiRepoMeta1❌❌
CollectApiPullRequestsMeta≈10^3❌❌
CollectApiIssuesMeta≈10^4❌❌
CollectApiPrCommentsMeta≈10^5❌❌
*CollectApiIssueCommentsMeta*≈10^6❌❌
CollectApiPipelinesMeta<10^4❌❌
CollectApiDeploymentsMeta<10^2❌❌

Gitee

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectApiRepoMeta1❌❌
CollectApiPullRequestsMeta≈10^3✅❌
CollectApiIssuesMeta≈10^4✅❌
CollectCommitsMeta?≈10^4✅❌
CollectApiPrCommentsMeta≈10^5❌❌
*CollectApiIssueCommentsMeta*≈10^6✅❌
CollectApiPullRequestCommitsMeta≈10^5❌❌
CollectApiPullRequestReviewsMeta≈10^5❌❌
*CollectApiCommitStatsMeta*≈10^6 (Not enable)❌❌

SonarQube

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectAccounts<10^4❌❌
CollectIssues<10^4❌❌
CollectHotspots<10^4❌❌
CollectFilemetrics<10^4❌❌
CollectAdditionalFilemetrics<10^4❌❌

Bamboo

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectPlanMeta<10^4❌❌
CollectJobMeta<10^5❌❌
CollectPlanBuildMeta<10^6❌❌
CollectJobBuildMeta<10^6❌❌
CollectDeployMeta1❌❌

Zentao

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectExecutionMeta<10^3❌❌
CollectStoryMeta<10^4❌❌
CollectBugMeta<10^4❌❌
CollectTaskMeta<10^4❌❌
CollectAccountMeta≈10^2❌❌
CollectDepartmentMeta≈10^2❌❌

Tapd

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectWorkitemTypesMeta≈10--
CollectStoryCustomFieldsMeta≈10--
CollectTaskCustomFieldsMeta≈10--
CollectBugCustomFieldsMeta≈10--
CollectStoryCategoriesMeta≈10--
CollectStoryStatusMeta≈10--
CollectStoryStatusLastStepMeta≈10--
CollectBugStatusMeta≈10--
CollectBugStatusLastStepMeta≈10--
CollectAccountsMeta≈10^3❌❌
CollectIterationMeta≈10^4✅✅
CollectStoryMeta≈10^4✅✅
CollectBugMeta≈10^4✅✅
CollectTaskMeta≈10^4✅✅
CollectBugChangelogMeta≈10^6✅✅
CollectStoryChangelogMeta≈10^6✅✅
CollectTaskChangelogMeta≈10^6✅✅
CollectWorklogMeta≈10^6✅✅
CollectBugCommitMeta≈10^6✅✅
CollectStoryCommitMeta≈10^6✅✅
CollectTaskCommitMeta≈10^6✅✅
CollectStoryBugMeta≈10^6✅✅

Azure Devops

Subtask NameEstimated Max Number of RequestDoes It support Incremental Collection?Does It Support Time Filter?
CollectBuilds<10^3❌❌
CollectJobs<10^4❌❌
CollectPullRequests<10^3❌❌
CollectPullRequestCommits<10^4❌❌