44 lines
2.8 KiB
Markdown
44 lines
2.8 KiB
Markdown
|
- *Cargo is not a build system*. This is even in the Cargo book. But everyone treats it as such (including me).
|
||
|
|
- *Cargo downloads dependencies, compiles packages, makes them distributable packages, and uploads them to crates.io*.
|
||
|
|
- their tests must:
|
||
|
|
- build a sandbox binary for executing webassembly
|
||
|
|
- build a webassembly program
|
||
|
|
- post-process the webassembly program
|
||
|
|
- build and execute a test binary that launches the sandbox binary, sends the webassembly program to the sandbox, and interacts with the program
|
||
|
|
- this test flow requires three cargo commands, and thus cannot be represented as a single cargo directive
|
||
|
|
- they saw no improvement from using [sscache](https://github.com/mozilla/sccache)
|
||
|
|
- they tried using Nix
|
||
|
|
- *requires fine-grained derivations*: each rust package must be turned into a derivation. They used cargo2nix to get there.
|
||
|
|
- developers were uncomfortable. Only the experts could modify the build rules.
|
||
|
|
- they crashed when the deployment team insisted on not using Nix.
|
||
|
|
- their builds became unbearably slow and flaky
|
||
|
|
- *developers were using the nix shell, which diverged from the CI*
|
||
|
|
- bazel
|
||
|
|
- easy to define and wire build targets
|
||
|
|
- adding new build rules requires expertise
|
||
|
|
- build files are verbose and boring
|
||
|
|
- gracefully handles linux and macos binaries, webassembly programs, os images, docker containers, etc, etc
|
||
|
|
- has aggressive caching
|
||
|
|
- remote caching
|
||
|
|
- distributed builds
|
||
|
|
- all tests are bazel tests, and so every developer can run any test locally
|
||
|
|
- migration took months
|
||
|
|
- built a prototype. sample repository mimicked the features.
|
||
|
|
- created a bazel rule called `cargo_build`, which continued to treat Cargo as a black box.
|
||
|
|
- Bazel builds binaries from Rust directly and ignores cargo.
|
||
|
|
- Added the bazel test job as soon as they possibly could
|
||
|
|
- they then started at the bottom of the stack and swapped the Cargo file out for a BUILD file one crate at a time
|
||
|
|
- they also used a visualization to show how much progress they were making
|
||
|
|
- Cargo discovers tests automatically. Bazel has to be told explicitely. They wrote a tool that compared the cargo test output and the bazel test output to ensure that all tests made it through
|
||
|
|
- remaining problems
|
||
|
|
- they've never gotten a good replacement for cargo check
|
||
|
|
- rust-analyzer support is insufficient, so they have to keep cargo files around
|
||
|
|
- there's no replacement for cargo publish yet
|
||
|
|
-
|
||
|
|
|
||
|
|
---
|
||
|
|
- [Scaling Rust builds with Bazel](https://mmapped.blog/posts/17-scaling-rust-builds-with-bazel.html)
|
||
|
|
|
||
|
|
## Bazel
|
||
|
|
|
||
|
|
I have made several attempts at using Bazel, however the rust toolchain has always failed. I have not recorded the results of the most recent failure, but it is disheartening given that Bazel is Google's tool, and Google definitely has active Rust development happening.
|