From 750833e9c4650b1703e377cf99da99f2c808636c Mon Sep 17 00:00:00 2001 From: Nick Cameron Date: Fri, 18 Aug 2017 12:06:42 +1200 Subject: [PATCH] Documents managing tools in the Rust repo --- tools-in-rust-repo.md | 141 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 tools-in-rust-repo.md diff --git a/tools-in-rust-repo.md b/tools-in-rust-repo.md new file mode 100644 index 0000000..8f466a0 --- /dev/null +++ b/tools-in-rust-repo.md @@ -0,0 +1,141 @@ +# Maintaining tools in the Rust repository + +This document contains information for tool maintainers and Rust contributors +who need to work with tools that are part of the Rust distribution and +repository. For information on requirements for such tools see the +[stability docs](https://github.com/nrc/dev-tools-team/blob/master/stability/README.md). + +When adding tools or making significant changes, please be sure to cc @nrc and +@alexcrichton on PRs. Adding tools to the Rust repo requires approval from the +dev-tools team and sign-off from the core team. + + +## Repository layout + +Tools should live in their own repository which should be in an official +organisation (e.g., [rust-lang-nursery](https://github.com/rust-lang-nursery)). +Tools should be included into the Rust repository as a Git submodule in the +[src/tools](https://github.com/rust-lang/rust/tree/master/src/tools) directory. + + +## Constraints on tools + +It is essential that any commit used as a submodule in the Rust repo is +preserved forever (i.e., is *never* deleted or changed). This ensures that we +will continue to be able to build any historic release of Rust (which is +important for bisecting regressions, etc). In order to do this, any commit +pulled into the Rust repo as a submodule must be from a Rust organisation repo, +not an author's. If the commit is only on a branch, that branch cannot be +deleted. Whether the commit is on a branch or master, it cannot be rebased +(which essentially deletes the commit and creates a new one). + +Tools used in the Rust repo cannot use Cargo workspaces. This is because the +tool itself will be treated as a workspace crate and workspaces cannot be +nested. + + +## Making breaking changes + +If making a change to a 'core' part of Rust causes breakage in any of the tools, +then there is a bit of a dance to perform to land the Rust PR. + +Communication is key here! If you're a compiler dev, communicate with tool devs +as soon as possible (and vice versa). + +At the end of the day, the tool's maintainers are responsible for ensuring that +their tool continues to build and pass tests in the Rust repo. For non-trivial +fixes, tool devs should be happy to help fix breaking changes. + +How to land a breaking change: + +* Make your changes to the Rust repo +* Make changes to the tool (against its own repo, not the submodule in the Rust + repo) to address the breaking changes. + - check that the tool's tests pass inside the Rust repo + - either push a branch to the tool's repo or submit a PR +* Point the Rust submodule at the new tool commit +* Land the PR +* Wait for the PR to be included in a nightly release... +* Merge the tool commit to the tool's master branch + - if the commit was changed or rebased, **do not delete** the commit's branch + (See https://github.com/llvm-mirror/compiler-rt/branches for example of this policy) +* Create a new PR to the Rust repo pointing the tool submodule back to tool's + master tip; land it. + +When changing the tool submodule, make sure that any other commits are OK to +land (i.e., they are not work in progress and tests pass, etc.). + + +### Exceptions + +Rarely, a breaking change might be too complex to manage in the above fashion. +In such cases it is OK to temporarily stop building and testing the tool while +the tool devs fix the issue. + +* If you think this is necessary, you must communicate this to the tool's + maintainers, the dev-tools team, and the compiler team before landing the PR + (and preferably before starting the work). +* Try and do this early in the release cycle to give the maximum amount of time + to fix the tool before the beta uplift. +* It is never OK to do this on beta or stable branches. +* Avoid doing this, if you can. + +The steps to follow are similar to the previous steps, but you must comment out +code in the build system as needed, rather than change the submodule, when +landing your initial PR. + + +## Building and testing a tool + +After adding the tool as a submodule, you'll need to make some changes to the +build system to build and test the tool, and to have it distributed with Rustup. + +For all these changes, it is probably easiest to see how it is done for an +existing tool and try and copy that where possible. You can try grepping for +`rls` and `Rls` inside [src/bootstrap](https://github.com/rust-lang/rust/tree/master/src/bootstrap). + +To build the tool you'll need to add the tool as a workspace to `src/Cargo.toml` +and make several changes to `src/bootstrap/tool.rs`. + +To run a tool's tests, you'll need to make changes to `src/bootstrap/check.rs`. + +You probably want the 'tidy' script not to check your tool, to do that you'll +need to add the tool directory to the skip list in `src/tools/tidy/src/lib.rs`. + + +## Distribution + +To include a tool as a Rustup component you have to ensure that a tarball of the +compiled tool is created and put in the right place, and that there is an entry +in the Rustup manifest (generated by the build system). You don't need to make +any changes to Rustup itself. You'll need to edit `src/bootstrap/dist.rs` and +`src/bootstrap/install.rs` to create the tarball, and +`src/tools/build-manifest/src/main.rs` to make an entry in the manifest. + + +## Tips + +Rustbuild will update submodules as part of its build process. If you don't +commit the change, it will be overwritten. Therefore, you should either commit +after updating a submodule and before building, or set `submodules = false` in +`config.toml`. + +After making changes to `Cargo.toml`, you need to build to ensure `Cargo.lock` +is updated. Changes to `Cargo.lock` must be committed along with `Cargo.toml` +changes. + +To run tests for a tool, use `./x.py test src/tools/rls`. Note that it is +possible for tool tests to pass in their own CI, but fail when run inside the +Rust repo because of the different environment. + + +## People who can help you + +Usernames are for irc/GitHub. + +* Any questions - nrc/@nrc or ask in #rust-dev-tools +* Rustbuild - simulacrum/@Mark-Simulacrum, acrichto/@alexcrichton, or ask in #rust-infra +* Compiler issues - ask in #rustc, if you need a specific person try mw or nmatsakis. +* Rust distribution - acrichto/@alexcrichton +* Rustfmt or RLS - nrc/@nrc +* Clippy - Manishearth/@Manishearth or oli-obk/@oli-obk