This document explains how to set environment variables for Apache DevLake and what environment variables can be set.
This environment variable is used to enable or disable the execution of subtasks.
The format is as follows: plugin_name1:subtask_name1:enabled_value,plugin_name2:subtask_name2:enabled_value,plugin_name3:subtask_name3:enabled_value
Guidance on locating the plugin_name and subtask_name:
Example 1: Enable some subtasks that are closed by default
ENABLE_SUBTASKS_BY_DEFAULT="jira:collectIssueChangelogs:true,jira:extractIssueChangelogs:true,jira:convertIssueChangelogs:true,tapd:collectBugChangelogs:true,tapd:extractBugChangelogs:true,tapd:convertBugChangelogs:true,zentao:collectBugRepoCommits:true,zentao:extractBugRepoCommits:true,zentao:convertBugRepoCommits:true,zentao:collectStoryRepoCommits:true,zentao:extractStoryRepoCommits:true,zentao:convertStoryRepoCommits:true,zentao:collectTaskRepoCommits:true,zentao:extractTaskRepoCommits:true,zentao:convertTaskRepoCommits:true"
Example 2: Close some subtasks that are executed by default
ENABLE_SUBTASKS_BY_DEFAULT="github_graphql:Collect Job Runs:false,github_graphql:Extract Job Runs:false,github_graphql:Convert Job Runs:false"
This set of environment variables is used to configure and finetune the behavior of the GitHub GraphQL Job Runs collection process.
| Environment Variable | Description | Default Value |
|---|---|---|
| GITHUB_GRAPHQL_JOB_COLLECTION_MODE | Specifies the mode of job collection. Possible values are BATCHING and PAGINATING | BATCHING |
| GITHUB_GRAPHQL_JOB_BATCHING_INPUT_STEP | Defines the step size for batching mode. | 10 |
| GITHUB_GRAPHQL_JOB_BATCHING_PAGE_SIZE | Defines the limit of jobs to collect in a batch for each run. | 20 |
| GITHUB_GRAPHQL_JOB_PAGINATING_PAGE_SIZE | Defines the page size for paginating mode. | 50 |
These environment variables are particularly useful when dealing with large repositories that have a significant number of job runs. By adjusting these settings, you can optimize the data collection process to better suit your specific needs and infrastructure capabilities. Also this can help to avoid timeouts on the github GraphQL API with too large requests.
BATCHING for GITHUB_GRAPHQL_JOB_COLLECTION_MODE when your workflow runs typically have less than 20 jobs and you want to minimize the number of API calls to GitHub.GITHUB_GRAPHQL_JOB_BATCHING_INPUT_STEP and GITHUB_GRAPHQL_JOB_BATCHING_PAGE_SIZE to control how many jobs are collected in each batch. NOTE: Increasing these values can lead to timeouts if the requests become too large.PAGINATING for GITHUB_GRAPHQL_JOB_COLLECTION_MODE when your workflow runs have a large number of jobs (e.g., more than 50). This mode will only query 1 Workflow run at a time and paginate through the jobs, reducing the risk of timeouts.GITHUB_GRAPHQL_JOB_PAGINATING_PAGE_SIZE to control how many jobs are fetched per page. A smaller page size can help avoid timeouts but may increase the total number of API calls.TLDR: BATCHING is more efficient for smaller workflows, while PAGINATING will guarantee complete collection of jobs for larger workflows.
After setting the environment variable, restart the DevLake service to take effect.
docker-compose down and docker-compose up -d.helm upgrade devlake devlake/devlake --recreate-pods.