dev-docs/dependency-upgrades.adoc

= Dependency upgrades

Solr has lots of 3rd party dependencies, defined in `gradle/libs.versions.toml`.
Keeping them up-to-date is crucial for a number of reasons:

* minimizing the risk of critical CVE vulnerabilities by staying on a recent and supported version
* avoiding "dependency hell", that can arise from falling too far behind

Read the https://github.com/apache/solr/blob/main/dev-docs/gradle-help/dependencies.txt[dev-docs/gradle-help/dependencies.txt] file for an in-depth
explanation of how dependencies are managed.

== Manual dependency upgrades
To upgrade a dependency, you need to run through a number of steps:

1. Identify the available versions from e.g. https://search.maven.org[Maven Central]
2. Update the version in `gradle/libs.versions.toml` file
3. Run `./gradlew resolveAndLockAll --write-locks` to re-generate lockfiles. Note that this may cause a cascading effect
   where the locked version of other dependencies also changes.
4. Run `./gradlew kotlinUpgradeYarnLock` to update the kotlin-js-store lockfile used for the new UI.
   Most of the cases it will not have any changes.
5. In case of a conflict, resolve the conflict according to `dev-docs/gradle-help/dependencies.txt`
6. Update the license and notice files of the changed dependencies. See `dev-docs/gradle-help/dependencies.txt` for details.
7. Run `./gradlew updateLicenses` to re-generate SHA1 checksums of the new jar files.
8. Once in a while, a new version of a dependency will transitively bring in brand-new dependencies.
   You'll need to decide whether to keep or exclude them. See `dev-docs/gradle-help/dependencies.txt` for details.

=== Constraints and Version Alignment

To sync the version of direct and transitive dependencies across the project, we iterate in the `:platform` module
over the libraries defined in `gradle/libs.version.toml` and add them as constraints. Then, we use the module in
main modules like `:solr:api` and `:solr:core` and transitively pass down to all other modules the constraints.

If a new module does not depend on another module that already includes `:platform` as a platform dependency, it should
explicitly add it to sync the versions with the rest of the project. `:solr:server` is one case where this is necessary.

=== Addressing Security Vulnerabilities

When it comes to security vulnerabilities that are found in direct or transitive dependencies, the recommended way to
address them is to update the specific library if there is a new release that solves this issue. For both direct and
transitive dependencies, we simply have to update the version as described above.

In case it is a transitive dependency that is not directly used, you can simply add it to `libs.versions.toml` as you
would with any other dependency. The dependency resolution approach defined in `:platform` will handle the rest.
Don't forget to add a `# @keep` note with a reference to the vulnerable version and CVE that is fixed with the explicit
definition of the library and new version. This way it is easier to keep track of unreferenced dependencies in our
libraries toml file, and we can clean them up once the libraries using the modules are updated.

== Renovate bot Pull Requests

A member of the Solr community operates a Github bot running https://github.com/renovatebot/renovate[Renovate], which
files Pull Requests to Solr with dependency upgrade proposals. The PRs are labeled `dependencies` and do include
changes resulting from the gradle tasks `resolveAndLockAll` and `updateLicenses`.

Community members and committers can then review, and if manual changes are needed, help bring the PR to completion.
For many dependencies, a changelog is included in the PR text, which may help guide the upgrade decision.

The scans are run on a schedule. New PRs are filed once a month (before 9am on the first day of the month), and only
dependency versions that are at least 5 days old are suggested, to guard against bad apples. Since Solr only releases
every couple of months, a monthly cadence keeps the number of open dependency PRs manageable. A Release Manager who
wants to be on the very latest right before a release can temporarily switch to a more frequent schedule (e.g.
`["* * * * 0"]` for weekly) by editing `renovate.json`.

To further cut down on the number of tiny PRs, upgrades are *grouped*. There are a handful of logical groups for
libraries that must stay version-compatible (e.g. Admin UI, Web server stack, Logging), and then two large
catch-all groups - "all non-major dependencies" and "all non-major test dependencies" - that bundle the *minor and
patch* bumps of everything else. *Major* bumps are never grouped; each gets its own PR for individual review. See
`renovate.json` for the exact group definitions.

If an upgrade is decided, simply merge (and backport) the PR. To skip an upgrade, close the PR. If a PR is left open,
it will be re-used and auto updated whenever a newer patch- or minor version gets available. Thus, one can reduce
churn from frequently-updated dependencies by delaying merge until a few weeks before a new release. One can also
choose to change to a less frequent schedule or disable the bot, by editing `renovate.json`.

The `branch_9x` branch has its own `renovate.json` that layers on top of (and inherits from) this one, overriding only
the schedule and the lockfile/license commands. Grouping rules defined here therefore apply to `branch_9x` too.

Please note that Solr version prior to 10.X use a versions resolution plugin that uses `versions.lock` instead of
`libs.version.toml`. Therefore, changes cannot be backported via cherry-pick.

=== Configuring renovate.json

While the bot runs on a https://github.com/solrbot/renovate-github-action[GitHub repo external to the project],
the bot behavior can be tailored by editing `.github/renovate.json` in this project.
See https://docs.renovatebot.com[Renovatebot docs] for available options.