NOTE(areusch): This RFC is a combination of the two above pre-RFCs, and limited copy-editing has been done to preserve the flow of the document.
This RFC proposes to add product roadmaps to TVM.
Roadmaps should be seen as a way of unifying the planning process of TVM, all the way from ideas to PR merges. The roadmaps discussed in this RFC are intentionally designed to integrate with TVM’s existing planning tools (e.g. GitHub tracking issues, RFCs), while adding an additional space for sharing and collaboration earlier in the R&D process.
This proposal includes two categories of roadmaps:
This guide-level explanation will describe the basic structure of the roadmap, without focusing heavily on the intended contents.
The roadmap utilizes GitHub Projects as the underlying tooling mechanism. This maximizes reuse of existing content tracked in GitHub, and is very user-friendly for existing GitHub users.
This RFC proposes to place the TVM roadmaps directly in the apache/tvm
repository, since the roadmap is intended to be a community-focused project, and having the roadmaps directly in TVM would help to maximize visibility to the TVM community.
Above is a sample roadmap for TVM CI and Testing, with annotations to show several important features within the roadmap:
Themes & Goals: The Themes & Goals are listed in the top-most card of each column of the roadmap, and are intended to show how the focus areas of the project are addressed that quarter.
For example, in Q3 of 2021 (the last 3 months of development in TVM), two of the major focus areas in TVM CI and Testing were Stability and Runtime.
Roadmap Items: Each column of development contains roadmap items which contribute to each of the Themes & Goals described above. These items are meant to unify feature development within each project, whether it is coming from an RFC, GitHub task tracking, or directly from within the roadmap. Items on the roadmap generally have feature ownership at least partially identified at a minimum.
This section proposes the Roadmap-RFC process, which aims to encapsulate the processes discussed in the Overview section. The Roadmap-RFC is used to create, modify, or delete roadmaps. Details on the process are listed below.
The following fields provide some contextual information for each roadmap.
For each roadmap, a set of scope and themes should also be defined, in order for TVM‘s community members to quickly learn about a roadmap’s key focus areas.
Establishing Scope
Establishing Themes
programmability
, portability
, and performance
.performance
could be interpreted as tuning times, runtime latency, or a number of other definitions.A roadmap‘s maintainers are the primary folks responsible for adding, modifying, and removing items for a roadmap. This isn’t a hard and fast rule—for example, it may be expedient for other community members to triage new roadmap items into a roadmap's backlog. However, the maintainers should be considered the “owners” of a roadmap, and generally no rigid process is defined around modifying the items in a roadmap.
Roadmaps are defined using GitHub Projects and can include GitHub Issues, Pull Requests, and simple note cards. Maintainers should strive to place mainly GitHub Issues in roadmaps to make it possible for the community to learn more about ongoing work and reduce triage burden.
Each item on a roadmap is intended to track one of these community processes:
pre-RFCs serve as a way to begin discussions on planned work at the earliest stage of maturity. pre-RFCs are typically posted in the TVM Discuss forums in order to solicit TVM Community feedback. For an example of a pre-RFC, see the screenshot of @areusch's proposal to Convert RST Docs to Markdown.
pre-RFCs can be tracked on a Roadmap by preemptively creating a GitHub Task-tracking Issue in tvm-rfcs.
RFCs serve as a way to share feature proposals with the TVM Community. The tvm-rfcs repo is used for the creation and acceptance of RFCs. For some examples of RFCs, see the screenshot of open pull requests in tvm-rfcs below.
Open RFCs can be directly linked into any roadmap. Once an RFC is accepted, please use the GitHub Task-Tracking process to track RFC Execution.
GitHub Task-Tracking Issues are used in tvm to share the progress of midsize features and/or accepted RFCs over time.
For an example of a GitHub Task-Tracking Issue, see the screenshot of @AndrewZhaoLuo's RFC to Add Mixed-Precision Support to TVM below.
These task-tracking issues can be directly linked into any roadmap.
For an example of a midsize feature which could be categorized as a GitHub Task-Tracking Issue, see the screenshot of @FranckQC's Implementation of Common Subexpression Elimination for TIR below.
These features will need a separate GitHub Issue created and linked to the applicable pull requests, so that they can be properly linked into GitHub Projects.
Bugfixes are actionable GitHub Issues not necessarily connected to a Task-Tracking Issue. Generally, they require only 1 PR and the work is clearly specified in the issue. For an example of a bugfix GitHub Issue, see the screenshot of a flaky CI test report below.
You can directly link bugfixes in a roadmap by adding their corresponding GitHub Issues as a card. Generally, it’s the maintainer’s responsibility to combine related bugfixes in order to keep the roadmap concise. For example, if you have 10 flaky tests in CI, it might make sense to have a global tracking issue for flaky CI tests which points to each individual bug.
There were several key considerations made while designing this roadmap, centering around the overall themes of quality content, usability, and maintainability. Below are some of these considerations:
Below is the initial set of roadmaps proposed in this RFC.
Roadmap Name | Scope |
---|---|
TVM Global Roadmap | All of TVM's efforts |
TVM Automation Roadmap | Automated performance improvements across the entire TVM stack |
TVM Community Roadmap | Community efforts across all of TVM, including Developer Tooling, Documentation, Events, Videos, and Tutorials |
TVM Continuous Integration & Testing Roadmap | CI and testing across all of TVM |
TVM Core Compiler Roadmap | Major compiler features and/or refactors with impacts across key APIs and/or multiple IRs in the TVM stack |
TVM Frontend & Interface Roadmap | Major user-facing pieces of TVM, such as importers, TVMC, and user facing APIs |
TVM Graph Computation & High-Level Optimizations Roadmap | All things Relay related |
TVM Scheduler & Low-Level Optimizations Roadmap | All things TIR related |
TVM Hardware Roadmap | All things hardware backend related |
microTVM roadmap | All things microTVM related |
There are a few challenges to having publicly available roadmaps, but overall, we believe that the benefits of having roadmaps for TVM vastly outweigh the negatives.
apache/tvm
via GitHub Projects, roadmap items would be created directly in apache/tvm
as GitHub issues. Currently, GitHub issue usage is very limited in apache/tvm
in order to ensure easier triage and timely closure of issues. Having the TVM roadmaps live in apache/tvm
would increase the overall count of GitHub issues in the repository, but the roadmap items could be labeled with a tvm:roadmap
tag so that they can be properly triaged and filtered.There are several alternatives to this roadmap proposal; however, each of the alternatives has drawbacks of its own.
apache/tvm-roadmap
) might reduce the number of roadmap issues filed directly to apache/tvm
, but it would increase the complexity and fragmentation of the roadmap significantly, since this roadmap repository would have to link to RFC tracking issues in apache/tvm
anyways.This RFC is inspired by the previous work done to improve the tracking mechanisms of TVM. This includes but is not limited to the existing RFC process, release process, and technical visions of TVM.
It is also inspired by the public GitHub roadmap, which is an excellent example of usage for GitHub Projects.
This is a prospective upstreaming timeline for the TVM roadmap.
Milestone Name | Deliverables |
---|---|
TVM Roadmap M1A: Taking the Plunge | All roadmap projects defined Initial prototype roadmap (TVM CI & Testing) upstreamed Skeleton roadmap documentation upstreamed Scope of M1B and M1C clearly defined |
TVM Roadmap M1B: Building Momentum | At least 3 additional roadmaps upstreamed Additional roadmap process documentation upstreamed |
TVM Roadmap M1C: Ready for Prime Time | All initial roadmaps (described in M1A RFC) upstreamed All RFC opens from M1A and M1B addressed Publicize roadmap in documentation and TVM forums |
Questions about the design and contents of the TVM Roadmap will be progressively resolved through the upstreaming timeline shown above. Any unresolved questions at the end of TVM Roadmap Milestone 1 will be created as roadmap items in the TVM Community Roadmap.
Thanks so much for copyediting, @electriclilies! :hugs: