90 lines
4.3 KiB
Markdown
90 lines
4.3 KiB
Markdown
Tools for running builds using Nix in a monorepo.
|
|
|
|
Goal: Using one coherent build tool, I can develop multiple projects with common dependencies.
|
|
|
|
# Background
|
|
|
|
I do a lot of Rust development. I also like to attach other programming languages so that I can, for instance, write a [go application that works in a web browser](https://git.luminescent-dreams.com/savanni/tools/src/branch/main/kifu/pwa). And, I tend to work in a [monorepo](https://git.luminescent-dreams.com/savanni/tools/), because that helps me keep all of my disparate interests somewhat in sync with one another.
|
|
|
|
Finally, I use Nix to manage my development tools.
|
|
|
|
However, what I cannot do is use Nix to actually build my projects. Most of my projects, admittedly, are Rust projects and so they are pretty easily built with a simple Cargo.toml. However, as soon as I want to do something complicated, such as a web application, I have to dig extensively to find documentation on how to actually lay out the application and get all of the tools in place. Plus, I have to use a different build tool and get all of the build tools to work together.
|
|
|
|
What I actually want to do is this.
|
|
|
|
```
|
|
> nix run .#kifu-gtk
|
|
```
|
|
|
|
And I want you to be able to just say this...
|
|
|
|
```
|
|
> nix run https://git.luminescent-dreams.com/savanni/tools/#.kifu-gtk
|
|
```
|
|
|
|
## Monorepos
|
|
|
|
For the purpose of this project, a monorepo as a single Git repository which contains potentially many libraries and applications, in separate packages (rust Crates, node Modules, etc), which interdepend and may be developed simultaneously with one another.
|
|
|
|
## Past experiences
|
|
|
|
I don't have many past experiences.
|
|
|
|
- I have the 1Password Monorepo, which is mostly held together with Makefiles which invoke platform-specific build tools. These were put together before I arrived at the company, and they are extensive. The developer build environment does not match the CI build environment.
|
|
- I have tried Bazel, but have not been able to build a purely Rust project even while following the tutorials.
|
|
- I have my [Tools Monorepo](https://git.luminescent-dreams.com/savanni/tools/) where I use a bunch of Makefiles to invoke platform-specific build commands. This doesn't work too badly, but I've hand-rolled every boilerplate Makefile, and should I want to add something like cross-compilation to Raspberry Pi, I am in for a world of pain.
|
|
|
|
I have evaluated all but one of the [[Rust build tools in Nix]], and found significant drawbacks in each. The biggest drawback, though, is that the tools are all for building an application. Binding two applications together is never a consideration with any of the build tools, and so something like the Kifu PWA application simply cannot be built with Nix as a build tool.
|
|
# Requirements
|
|
|
|
- select the toolchains and versions thereof for the project
|
|
- configure cross-compilation
|
|
- allow multiple languages
|
|
- co-exist with language ecosystems. For example, rust-analyzer needs cargo in order to understand Cargo.toml. However, it should be easier to fully build the project with Nix
|
|
- every module is compiled into a derivation that downstream modules can import. Each module derivation is cached separately so that Nix only rebuilds the changed module.
|
|
- Nix can generate a closure for an executable, even a cross-compiled one, to copy to the remote system
|
|
|
|
# Languages and Bindings
|
|
|
|
Languages:
|
|
- C
|
|
- Rust
|
|
- Typescript
|
|
- Javascript
|
|
|
|
Bindings:
|
|
- C -> Rust
|
|
- Rust -> Typescript
|
|
- Rust -> Javascript
|
|
|
|
Backends
|
|
|
|
| | Linux x86 | Linux ARM | WASM | AVR | ESP32 |
|
|
| ---------- | --------- | --------- | ---- | --- | ----- |
|
|
| C | yes | yes | | yes | |
|
|
| Rust | yes | yes | yes | yes | yes |
|
|
| Typescript | | | yes | | |
|
|
| Javascript | | | yes | | |
|
|
|
|
- [[Mononix, C library]]
|
|
- [[Mononix, Rust library]]
|
|
|
|
# Build targets
|
|
|
|
- build
|
|
- run
|
|
- tests
|
|
- code coverage profiling
|
|
- linting
|
|
- performance testing
|
|
|
|
Each of these is a separate Nix command that builds or runs the desired result.
|
|
|
|
---
|
|
- [[Nix For Development]]
|
|
- [[Cross-compiling]]
|
|
- [[Scaling Rust Builds with Bazel]]
|
|
- [[Makefiles and Monorepos]]
|
|
- [Experiment in cross-compiling](https://github.com/jraygauthier/jrg-rust-cross-experiment/blob/master/simple-static-rustup-target-windows/shell.nix)
|
|
- [Building Nix flakes from Rust workspaces - Tweag](https://www.tweag.io/blog/2022-09-22-rust-nix/)
|