---
title: "Supported Data Sources"
description: >
  Data sources that DevLake supports
sidebar_position: 5
---

## Data Sources and Data Plugins

Apache DevLake(incubating) supports the following data sources. The data from each data source is collected with one or more plugins. Detailed plugin docs can be found [here](/docs/Plugins).

| Data Source         | Domain(s)                                                                    | Supported Versions                 | Config UI Availability  | Triggered Plugins             | Collection Mode                                                |
|---------------------| ---------------------------------------------------------------------------- | ---------------------------------- | ----------------------- |-------------------------------|----------------------------------------------------------------|
| GitHub              | Source Code Management, Code Review, Issue Tracking, CI/CD (GitHub Actions)  | Cloud                              | Available               | `github`, `gitextractor`      | Incremental Sync                                               |
| GitLab              | Source Code Management, Code Review, Issue Tracking, CI/CD (GitLab CI)       | Cloud, Community Edition 11+       | Available               | `gitlab`, `gitextractor`      | Full Refresh, Incremental Sync(for `issues`,`MRs`)             |
| Jira                | Issue Tracking                                                               | Cloud, Server/Data Center 7.x, 8.x | Available               | `jira`                        | Full Refresh, Incremental Sync(for `issues` and related)       |
| Jenkins             | CI/CD                                                                        | 2.263.x+                           | Available               | `jenkins`                     | Incremental Sync                                               |
| BitBucket (Beta)    | Source Code Management, Code Review                                          | Cloud                              | Advanced Mode Available | `bitbucket`, `gitextractor`   | Full Refresh                                                   |
| TAPD (Beta)         | Issue Tracking                                                               | Cloud                              | Advanced Mode Available | `tapd`                        | Full Refresh, Incremental Sync(for `stories`, `bugs`, `tasks`) |
| Teambition (Beta)   | Issue Tracking                                                               | Cloud                              | Advanced Mode Available | `teambition`                  | Full Refresh                                                   |
| Zentao (Beta)       | Issue Tracking                                                               | v17.x, v18.x                       | Advanced Mode Available | `zentao`                      | Full Refresh                                                   |
| Gitee (WIP)         | Source Code Management, Code Review, Issue Tracking                          | Cloud                              | Not Available           | `gitee`, `gitextractor`       | Full Refresh, Incremental Sync(for `issues`,`MRs`)             |
| PagerDuty           | Issue Tracking                                                               | Cloud                              | Available               | `pagerduty`                   | Incremental Sync                                               |
| Feishu (WIP)        | Calendar                                                                     | Cloud                              | Not Available           | `feishu`                      | Full Refresh                                                   |
| AE                  | Source Code Management                                                       | On-prem                            | Not Available           | `ae`                          | Full Refresh                                                   |
| Sonarqube           | CODE QUALITY                                                                 | SonarQube v8.x, v9.x               | Available               | `sonarqube`                   | Full Refresh                                                   |
| Bamboo CI(WIP)      | CI/CD                                                                        | v6.8.1                             | Not Available           | `bamboo`                      | Full Refresh                                                   |
| Azure Devops (Beta) | CI/CD, Source Code Management, Code Review                                   | Cloud                              | Available               | `azuredevops`, `gitextractor` | Full Refresh                                                   |

## Data Collection Scope By Each Plugin

This table shows the entities collected by each plugin. Domain layer entities in this table are consistent with the entities [here](/DataModels/DevLakeDomainLayerSchema.md).
✅ : Collect by default.
💪 : Collect not by default. You need to add the corresponding subtasks to collect these entities in the [advanced mode](../Configuration/AdvancedMode.md).

| Domain Layer Entities                                                                       | ae  | dora | gitextractor | incoming webhook | github | gitlab | jenkins | jira | refdiff | tapd | sonarqube | bamboo | azuredevops |
| ------------------------------------------------------------------------------------------- | --- | ---- | ------------ | ---------------- | ------ | ------ | ------- | ---- | ------- | ---- | --------- | ------ | ----------- |
| [accounts](../DataModels/DevLakeDomainLayerSchema.md/#accounts)                             |     |      |              |                  | ✅     | ✅     |         | ✅   |         | ✅   | ✅        |        |             |
| [board_issues](../DataModels/DevLakeDomainLayerSchema.md/#board_issues)                     |     |      |              |                  | ✅     | ✅     |         | ✅   |         | ✅   |           |        |             |
| [board_repos](../DataModels/DevLakeDomainLayerSchema.md/#board_repos)                       |     |      |              |                  | ✅     | ✅     |         |      |         |      |           |        |             |
| [board_sprints](../DataModels/DevLakeDomainLayerSchema.md/#board_sprints)                   |     |      |              |                  | ✅     |        |         | ✅   |         | ✅   |           |        |             |
| [boards](../DataModels/DevLakeDomainLayerSchema.md/#boards)                                 |     |      |              |                  | ✅     | ✅     |         | ✅   |         | ✅   |           |        |             |
| [cicd_pipeline_commits](../DataModels/DevLakeDomainLayerSchema.md/#cicd_pipeline_commits)   |     | ✅   |              |                  | ✅     | ✅     | ✅      |      |         |      |           |   ✅   |      ✅     |
| [cicd_pipelines](../DataModels/DevLakeDomainLayerSchema.md/#cicd_pipelines)                 |     | ✅   |              |                  | ✅     | ✅     | ✅      |      |         |      |           |   ✅   |      ✅     |
| [cicd_scopes](../DataModels/DevLakeDomainLayerSchema.md/#cicd_scopes)                       |     | ✅   |              |                  | ✅     | ✅     | ✅      |      |         |      |           |   ✅   |      ✅     |
| [cicd_tasks](../DataModels/DevLakeDomainLayerSchema.md/#cicd_tasks)                         |     | ✅   |              | 💪               | ✅     | ✅     | ✅      |      |         |      |           |   ✅   |      ✅     |
| [commit_file_components](../DataModels/DevLakeDomainLayerSchema.md/#commit_file_components) |     |      | ✅           |                  |        |        |         |      |         |      |           |        |             |
| [commit_files](../DataModels/DevLakeDomainLayerSchema.md/#commit_files)                     |     |      | ✅           |                  |        |        |         |      |         |      |           |        |             |
| [commit_line_change](../DataModels/DevLakeDomainLayerSchema.md/#commit_line_change)         |     |      | ✅           |                  |        |        |         |      |         |      |           |        |             |
| [commit_parents](../DataModels/DevLakeDomainLayerSchema.md/#commit_parents)                 |     |      | ✅           |                  |        |        |         |      |         |      |           |        |             |
| [commits](../DataModels/DevLakeDomainLayerSchema.md/#commits)                               | ✅  |      | ✅           |                  | 💪     | 💪     |         |      |         |      |           |        |             |
| [commits_diffs](../DataModels/DevLakeDomainLayerSchema.md/#commits_diffs)                   |     |      |              |                  |        |        |         |      | ✅      |      |           |        |             |
| [components](../DataModels/DevLakeDomainLayerSchema.md/#components)                         |     |      |              |                  |        |        |         |      |         |      |           |        |             |
| [finished_commits_diffs](../DataModels/DevLakeDomainLayerSchema.md/#finished_commits_diffs) |     |      |              |                  |        |        |         |      |         |      |           |        |             |
| [issue_changelogs](../DataModels/DevLakeDomainLayerSchema.md/#issue_changelogs)             |     |      |              |                  |        |        |         | ✅   |         | ✅   |           |        |             |
| [issue_comments](../DataModels/DevLakeDomainLayerSchema.md/#issue_comments)                 |     |      |              |                  | ✅     |        |         |      |         | ✅   |           |        |             |
| [issue_commits](../DataModels/DevLakeDomainLayerSchema.md/#issue_commits)                   |     |      |              |                  |        |        |         | ✅   |         | ✅   |           |        |             |
| [issue_labels](../DataModels/DevLakeDomainLayerSchema.md/#issue_labels)                     |     |      |              |                  | ✅     | ✅     |         |      |         | ✅   |           |        |             |
| [issue_repo_commits](../DataModels/DevLakeDomainLayerSchema.md/#issue_repo_commits)         |     |      |              |                  |        |        |         | ✅   |         |      |           |        |             |
| [issue_worklogs](../DataModels/DevLakeDomainLayerSchema.md/#issue_worklogs)                 |     |      |              |                  |        |        |         | ✅   |         | ✅   |           |        |             |
| [issues](../DataModels/DevLakeDomainLayerSchema.md/#issues)                                 |     |      |              |                  | ✅     |        |         | ✅   |         | ✅   |           |        |             |
| [project_issue_metrics](../DataModels/DevLakeDomainLayerSchema.md/#project_issue_metrics)   |     | ✅   |              |                  | ✅     | ✅     |         | ✅   |         | ✅   |           |        |             |
| [project_mapping](../DataModels/DevLakeDomainLayerSchema.md/#project_mapping)               |     | ✅   |              |                  | ✅     | ✅     | ✅      | ✅   |         | ✅   |           |        |             |
| [project_metrics](../DataModels/DevLakeDomainLayerSchema.md/#project_metrics)               |     | ✅   |              |                  | ✅     | ✅     | ✅      | ✅   |         | ✅   |           |        |             |
| [project_pr_metrics](../DataModels/DevLakeDomainLayerSchema.md/#project_pr_metrics)         |     | ✅   |              |                  | ✅     | ✅     |         |      |         | ✅   |           |        |             |
| [project](../DataModels/DevLakeDomainLayerSchema.md/#project)                               |     | ✅   |              |                  | ✅     | ✅     | ✅      | ✅   |         | ✅   |           |        |             |
| [pull_request_comments](../DataModels/DevLakeDomainLayerSchema.md/#pull_request_comments)   |     |      |              |                  | ✅     | ✅     |         |      |         |      |           |        |             |
| [pull_request_commits](../DataModels/DevLakeDomainLayerSchema.md/#pull_request_commits)     |     |      |              |                  | ✅     | ✅     |         |      |         |      |           |        |      ✅     |
| [pull_request_issues](../DataModels/DevLakeDomainLayerSchema.md/#pull_request_issues)       |     |      |              |                  | ✅     |        |         |      |         |      |           |        |             |
| [pull_request_labels](../DataModels/DevLakeDomainLayerSchema.md/#pull_request_labels)       |     |      |              |                  | ✅     | ✅     |         |      |         |      |           |        |             |
| [pull_requests](../DataModels/DevLakeDomainLayerSchema.md/#pull_requests)                   |     |      |              |                  | ✅     | ✅     |         |      |         |      |           |        |      ✅     |
| [ref_commits](../DataModels/DevLakeDomainLayerSchema.md/#ref_commits)                       |     |      |              |                  |        |        |         |      | ✅      |      |           |        |             |
| [refs](../DataModels/DevLakeDomainLayerSchema.md/#refs)                                     |     |      | ✅           |                  |        |        |         |      | ✅      |      |           |        |             |
| [refs_issues_diffs](../DataModels/DevLakeDomainLayerSchema.md/#refs_issues_diffs)           |     |      |              |                  |        |        |         |      | ✅      |      |           |        |             |
| [ref_pr_cherry_picks](../DataModels/DevLakeDomainLayerSchema.md/#ref_pr_cherry_picks)       |     |      |              |                  |        |        |         |      | ✅      |      |           |        |             |
| [repo_commits](../DataModels/DevLakeDomainLayerSchema.md/#repo_commits)                     |     |      | ✅           |                  | 💪     | 💪     |         |      |         |      |           |        |             |
| [repo_snapshot](../DataModels/DevLakeDomainLayerSchema.md/#repo_snapshot)                   |     |      | ✅           |                  |        |        |         |      |         |      |           |        |             |
| [repos](../DataModels/DevLakeDomainLayerSchema.md/#repos)                                   |     |      |              |                  | ✅     | ✅     |         |      |         |      |           |        |             |
| [sprint_issues](../DataModels/DevLakeDomainLayerSchema.md/#sprint_issues)                   |     |      |              |                  | ✅     |        |         | ✅   |         | ✅   |           |        |             |
| [sprints](../DataModels/DevLakeDomainLayerSchema.md/#sprints)                               |     |      |              |                  | ✅     |        |         | ✅   |         | ✅   |           |        |             |
| [team_users](../DataModels/DevLakeDomainLayerSchema.md/#team_users)                         |     |      |              |                  |        |        |         |      |         |      |           |        |             |
| [teams](../DataModels/DevLakeDomainLayerSchema.md/#teams)                                   |     |      |              |                  |        |        |         |      |         |      |           |        |             |
| [user_account](../DataModels/DevLakeDomainLayerSchema.md/#user_accounts)                    |     |      |              |                  |        |        |         |      |         |      |           |        |             |
| [users](../DataModels/DevLakeDomainLayerSchema.md/#users)                                   |     |      |              |                  |        |        |         | ✅   |         | ✅   |           |        |             |
| [cq_projects](../DataModels/DevLakeDomainLayerSchema.md/#cq_projects)                       |     |      |              |                  |        |        |         |      |         |      | ✅        |        |             |
| [cq_issues](../DataModels/DevLakeDomainLayerSchema.md/#cq_issues)                           |     |      |              |                  |        |        |         |      |         |      | ✅        |        |             |
| [cq_issue_code_blocks](../DataModels/DevLakeDomainLayerSchema.md/#cq_issue_code_blocks)     |     |      |              |                  |        |        |         |      |         |      | ✅        |        |             |
| [cq_file_metrics](../DataModels/DevLakeDomainLayerSchema.md/#cq_file_metrics)               |     |      |              |                  |        |        |         |      |         |      | ✅        |        |             |

## Data Sync Policy

**bold:** means it may collect slowly.

**\*bold\*:** means it may collect very slowly.

### Jira

| Subtask Name               | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| -------------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectStatusMeta          | 1                               | -                                       | -                            |
| CollectProjectsMeta        | <10                             | ❌                                      | -                            |
| CollectIssueTypesMeta      | <10                             | ❌                                      | -                            |
| CollectIssuesMeta          | <10^4                           | ✅                                      | ✅                           |
| CollectIssueChangelogsMeta | 1000~10^5                       | ✅                                      | ✅                           |
| CollectAccountsMeta        | <10^3                           | ❌                                      | ❌                           |
| CollectWorklogsMeta        | 1000~10^5                       | ✅                                      | ✅                           |
| CollectRemotelinksMeta     | 1000~10^5                       | ✅                                      | ✅                           |
| CollectSprintsMeta         | <100                            | ❌                                      | ❌                           |
| CollectEpicsMeta           | <100                            | ❌                                      | ✅                           |

### Jenkins

| Subtask Name         | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| -------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectApiBuildsMeta | ≈100                            | ❌                                      | ❌                           |
| CollectApiStagesMeta | ≈10^4                           | ❌                                      | ✅                           |

### GitLab

| Subtask Name                | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| --------------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectApiIssuesMeta        | <10^4                           | ✅                                      | ✅                           |
| CollectApiMergeRequestsMeta | <10^3                           | ✅                                      | ✅                           |
| CollectApiMrNotesMeta       | <10^5                           | ❌                                      | ✅                           |
| CollectApiMrCommitsMeta     | <10^5                           | ❌                                      | ✅                           |
| **CollectApiPipelinesMeta** | <10^4                           | ✅                                      | ❌                           |
| CollectApiJobsMeta          | <10^5                           | ❌                                      | ✅                           |

### Github

| Subtask Name                       | Estimated Max Number of Request     | Does It support Incremental Collection? | Does It Support Time Filter? |
| ---------------------------------- | ----------------------------------- | --------------------------------------- | ---------------------------- |
| ---------------------------------  | Common                              | -----------------------                 |                              |
| CollectMilestonesMeta              | ≈10                                 | ✅                                      | ❌                           |
| CollectRunsMeta                    | <10^4                               | ✅                                      | ✅                           |
| CollectApiCommentsMeta             | 400 (max page that GitHub supports) | ✅                                      | ✅                           |
| **CollectApiEventsMeta**           | 400 (max page that GitHub supports) | ❌                                      | ❌                           |
| CollectApiPullRequestReviewsMeta   | <10^5                               | ✅                                      | ✅                           |
| ---------------------------------  | Graphql Only (Default)              | -----------------------                 |                              |
| CollectIssueMeta                   | ≈10^4                               | ❌                                      | ✅                           |
| CollectPrMeta                      | ≈10^3                               | ❌                                      | ✅                           |
| CollectCheckRunMeta                | <10^4                               | ❌                                      | ✅                           |
| CollectAccountMeta                 | ≈10^2                               | ❌                                      | -                            |
| ---------------------------------  | Restful Only (Not by Default)       | -----------------------                 |                              |
| CollectApiIssuesMeta               | ≈10^4                               | ✅                                      | ❌                           |
| CollectApiPullRequestsMeta         | ≈10^2                               | ❌                                      | ❌                           |
| CollectApiPullRequestCommitsMeta   | ≈10^4                               | ✅                                      | ✅                           |
| **CollectApiPrReviewCommentsMeta** | ≈10^4                               | ✅                                      | ✅                           |
| **CollectAccountsMeta**            | ≈10^4                               | ❌                                      | ❌                           |
| **CollectAccountOrgMeta**          | ≈10^4                               | ❌                                      | ❌                           |
| CollectJobsMeta                    | <10^6                               | ❌                                      | ✅                           |
| CollectApiCommitsMeta              | Not enabled                         | -                                       | -                            |
| CollectApiCommitStatsMeta          | Not enabled                         | -                                       | -                            |

### Feishu

| Subtask Name                  | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| ----------------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectMeetingTopUserItemMeta | ≈10^3                           | ❌                                      | ✅                           |

### Bitbucket

| Subtask Name                        | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| ----------------------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| ~~CollectApiRepoMeta~~              | 1                               | ❌                                      | ❌                           |
| CollectApiPullRequestsMeta          | ≈10^3                           | ❌                                      | ❌                           |
| **CollectApiIssuesMeta**            | ≈10^4                           | ❌                                      | ❌                           |
| **CollectApiPrCommentsMeta**        | ≈10^5                           | ❌                                      | ❌                           |
| **\*CollectApiIssueCommentsMeta\*** | ≈10^6                           | ❌                                      | ❌                           |
| **CollectApiPipelinesMeta**         | <10^4                           | ❌                                      | ❌                           |
| CollectApiDeploymentsMeta           | <10^2                           | ❌                                      | ❌                           |

### Gitee

| Subtask Name                         | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| ------------------------------------ | ------------------------------- | --------------------------------------- | ---------------------------- |
| ~~CollectApiRepoMeta~~               | 1                               | ❌                                      | ❌                           |
| CollectApiPullRequestsMeta           | ≈10^3                           | ✅                                      | ❌                           |
| **CollectApiIssuesMeta**             | ≈10^4                           | ✅                                      | ❌                           |
| **CollectCommitsMeta?**              | ≈10^4                           | ✅                                      | ❌                           |
| **CollectApiPrCommentsMeta**         | ≈10^5                           | ❌                                      | ❌                           |
| **\*CollectApiIssueCommentsMeta\***  | ≈10^6                           | ✅                                      | ❌                           |
| **CollectApiPullRequestCommitsMeta** | ≈10^5                           | ❌                                      | ❌                           |
| **CollectApiPullRequestReviewsMeta** | ≈10^5                           | ❌                                      | ❌                           |
| **\*CollectApiCommitStatsMeta\***    | ≈10^6 (Not enable)              | ❌                                      | ❌                           |
### SonarQube

| Subtask Name                 | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| ---------------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectAccounts              | <10^4                           | ❌                                      | ❌                           |
| CollectIssues                | <10^4                           | ❌                                      | ❌                           |
| CollectHotspots              | <10^4                           | ❌                                      | ❌                           |
| CollectFilemetrics           | <10^4                           | ❌                                      | ❌                           |
| CollectAdditionalFilemetrics | <10^4                           | ❌                                      | ❌                           |

### Bamboo

| Subtask Name         | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| -------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectPlanMeta      | <10^4                           | ❌                                      | ❌                           |
| CollectJobMeta       | <10^5                           | ❌                                      | ❌                           |
| CollectPlanBuildMeta | <10^6                           | ❌                                      | ❌                           |
| CollectJobBuildMeta  | <10^6                           | ❌                                      | ❌                           |
| CollectDeployMeta    | 1                               | ❌                                      | ❌                           |

### Zentao

| Subtask Name          | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| --------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectExecutionMeta  | <10^3                           | ❌                                      | ❌                           |
| CollectStoryMeta      | <10^4                           | ❌                                      | ❌                           |
| CollectBugMeta        | <10^4                           | ❌                                      | ❌                           |
| CollectTaskMeta       | <10^4                           | ❌                                      | ❌                           |
| CollectAccountMeta    | ≈10^2                           | ❌                                      | ❌                           |
| CollectDepartmentMeta | ≈10^2                           | ❌                                      | ❌                           |

### Tapd


| Subtask Name                   | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| ------------------------------ | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectWorkitemTypesMeta       | ≈10                             | -                                       | -                            |
| CollectStoryCustomFieldsMeta   | ≈10                             | -                                       | -                            |
| CollectTaskCustomFieldsMeta    | ≈10                             | -                                       | -                            |
| CollectBugCustomFieldsMeta     | ≈10                             | -                                       | -                            |
| CollectStoryCategoriesMeta     | ≈10                             | -                                       | -                            |
| CollectStoryStatusMeta         | ≈10                             | -                                       | -                            |
| CollectStoryStatusLastStepMeta | ≈10                             | -                                       | -                            |
| CollectBugStatusMeta           | ≈10                             | -                                       | -                            |
| CollectBugStatusLastStepMeta   | ≈10                             | -                                       | -                            |
| CollectAccountsMeta            | ≈10^3                           | ❌                                      | ❌                            |
| CollectIterationMeta           | ≈10^4                           | ✅                                      | ✅                            |
| CollectStoryMeta               | ≈10^4                           | ✅                                      | ✅                            |
| CollectBugMeta                 | ≈10^4                           | ✅                                      | ✅                            |
| CollectTaskMeta                | ≈10^4                           | ✅                                      | ✅                            |
| CollectBugChangelogMeta        | ≈10^6                           | ✅                                      | ✅                            |
| CollectStoryChangelogMeta      | ≈10^6                           | ✅                                      | ✅                            |
| CollectTaskChangelogMeta       | ≈10^6                           | ✅                                      | ✅                            |
| CollectWorklogMeta             | ≈10^6                           | ✅                                      | ✅                            |
| CollectBugCommitMeta           | ≈10^6                           | ✅                                      | ✅                            |
| CollectStoryCommitMeta         | ≈10^6                           | ✅                                      | ✅                            |
| CollectTaskCommitMeta          | ≈10^6                           | ✅                                      | ✅                            |
| CollectStoryBugMeta            | ≈10^6                           | ✅                                      | ✅                            |


### Azure Devops

| Subtask Name              | Estimated Max Number of Request | Does It support Incremental Collection? | Does It Support Time Filter? |
| ------------------------- | ------------------------------- | --------------------------------------- | ---------------------------- |
| CollectBuilds             | <10^3                           | ❌                                       | ❌                          |
| CollectJobs               | <10^4                           | ❌                                       | ❌                          |
| CollectPullRequests       | <10^3                           | ❌                                       | ❌                          |
| CollectPullRequestCommits | <10^4                           | ❌                                       | ❌                          |