diff options
author | EuAndreh <eu@euandre.org> | 2024-11-18 08:21:58 -0300 |
---|---|---|
committer | EuAndreh <eu@euandre.org> | 2024-11-18 08:44:57 -0300 |
commit | 960e4410f76801356ebd42801c914b2910a302a7 (patch) | |
tree | 615d379416f72956d0c1666c63ce062859041fbe /_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md | |
parent | Remove jekyll infrastructure setup (diff) | |
download | euandre.org-960e4410f76801356ebd42801c914b2910a302a7.tar.gz euandre.org-960e4410f76801356ebd42801c914b2910a302a7.tar.xz |
Diffstat (limited to '_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md')
-rw-r--r-- | _articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md | 201 |
1 files changed, 0 insertions, 201 deletions
diff --git a/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md b/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md deleted file mode 100644 index c922589..0000000 --- a/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md +++ /dev/null @@ -1,201 +0,0 @@ ---- - -title: "swift2nix: Run Swift inside Nix builds" - -date: 2020-10-05 1 - -layout: post - -lang: en - -ref: swift2nix-run-swift-inside-nix-builds - ---- - -While working on a Swift project, I didn't find any tool that would allow Swift -to run inside [Nix][nix] builds. Even thought you *can* run Swift, the real -problem arises when using the package manager. It has many of the same problems -that other package managers have when trying to integrate with Nix, more on this -below. - -I wrote a simple little tool called [swift2nix] that allows you trick -Swift's package manager into assuming everything is set up. Here's the example -from swift2nix's README file: - -``` -let - niv-sources = import ./nix/sources.nix; - pkgs = import niv-sources.nixpkgs { }; - src = pkgs.nix-gitignore.gitignoreSource [ ] ./.; - swift2nix = pkgs.callPackage niv-sources.swift2nix { - package-resolved = ./Package.resolved; - }; -in pkgs.stdenv.mkDerivation { - inherit src; - name = "swift-test"; - buildInputs = with pkgs; [ swift ]; - phases = [ "unpackPhase" "buildPhase" ]; - buildPhase = '' - # Setup dependencies path to satisfy SwiftPM - mkdir .build - ln -s ${swift2nix.env.dependencies-state-json} .build/dependencies-state.json - ln -s ${swift2nix.env.checkouts} .build/checkouts - - # Run the tests - swift test - touch $out - ''; -} -``` - -The key parts are lines 15~17: we just fake enough files inside `.build/` that -Swift believes it has already downloaded and checked-out all dependencies, and -just moves on to building them. - -I've worked on it just enough to make it usable for myself, so beware of -unimplemented cases. - -[nix]: https://nixos.org/ -[swift2nix]: https://euandre.org/static/attachments/swift2nix.tar.gz - -## Design - -What swift2nix does is just provide you with the bare minimum that Swift -requires, and readily get out of the way: - -1. I explicitly did not want to generated a `Package.nix` file, since - `Package.resolved` already exists and contains the required information; -2. I didn't want to have an "easy" interface right out of the gate, after - fighting with "*2nix" tools that focus too much on that. - -The final actual code was so small (46 lines) that it made me -think about package managers, "*2nix" tools and some problems with many of them. - -## Problems with package managers - -I'm going to talk about solely language package managers. Think npm and cargo, -not apt-get. - -Package managers want to do too much, or assume too much, or just want to take -control of the entire build of the dependencies. - -This is a recurrent problem in package managers, but I don't see it as an -intrinsic one. There's nothing about a "package manager" that prevents it from -*declaring* what it expects to encounter and in which format. The *declaring* -part is important: it should be data, not code, otherwise you're back in the -same problem, just like lockfiles are just data. Those work in any language, and -tools can cooperate happily. - -There's no need for this declarative expectation to be standardized, or be made -compatible across languages. That would lead to a poor format that no package -manager really likes. Instead, If every package manager could say out loud what -it wants to see exactly, than more tools like swift2nix could exist, and they -would be more reliable. - -This could even work fully offline, and be simply a mapping from the lockfile -(the `Package.resolved` in Swift's case) to the filesystem representation. For -Swift, the `.build/dependencies-state.json` comes very close, but it is internal -to the package manager. - -Even though this pain only exists when trying to use Swift inside Nix, it sheds -light into this common implicit coupling that package managers have. They -usually have fuzzy boundaries and tight coupling between: - -1. resolving the dependency tree and using some heuristic to pick a package - version; -2. generating a lockfile with the exact pinned versions; -3. downloading the dependencies present on the lockfile into some local cache; -4. arranging the dependencies from the cache in a meaningful way for itself inside - the project; -5. work using the dependencies while *assuming* that step 4 was done. - -When you run `npm install` in a repository with no lockfile, it does 1~4. If you -do the same with `cargo build`, it does 1~5. That's too much: many of those -assumptions are implicit and internal to the package manager, and if you ever -need to rearrange them, you're on your own. Even though you can perform some of -those steps, you can't compose or rearrange them. - -Instead a much saner approach could be: - -1. this stays the same; -2. this also stays the same; -3. be able to generate some JSON/TOML/edn which represents the local expected - filesystem layout with dependencies (i.e. exposing what the package manager - expects to find), let's call it `local-registry.json`; -4. if a `local-registry.json` was provided, do a build using that. Otherwise - generate its own, by downloading the dependencies, arranging them, *etc.* - -The point is just making what the package manager requires visible to the -outside world via some declarative data. If this data wasn't provided, it can -move on to doing its own automatic things. - -By making the expectation explicit and public, one can plug tools *à la carte* -if desired, but doesn't prevent the default code path of doing things the exact -same way they are now. - -## Problems with "*2nix" tools - -I have to admit: I'm unhappy with most of they. - -They conflate "using Nix" with "replicating every command of the package manager -inside Nix". - -The avoidance of an "easy" interface that I mentioned above comes from me -fighting with some of the "\*2nix" tools much like I have to fight with package -managers: I don't want to offload all build responsibilities to the "*2nix" -tool, I just want to let it download some of the dependencies and get out of the -way. I want to stick with `npm test` or `cargo build`, and Nix should only -provide the environment. - -This is something that [node2nix] does right. It allows you to build -the Node.js environment to satisfy NPM, and you can keep using NPM for -everything else: - -```shell -ln -s ${node2nix-package.shell.nodeDependencies}/lib/node_modules ./node_modules -npm test -``` - -Its natural to want to put as much things into Nix as possible to benefit from -Nix's advantages. Isn't that how NixOS itself was born? - -But a "*2nix" tool should leverage Nix, not be coupled with it. The above -example lets you run any arbitrary NPM command while profiting from isolation -and reproducibility that Nix provides. It is even less brittle: any changes to -how NPM runs some things will be future-compatible, since node2nix isn't trying -to replicate what NPM does, or fiddling with NPM's internal. - -**A "*2nix" tool should build the environment, preferably from the lockfile -directly and offload everything else to the package manager**. The rest is just -nice-to-have. - -swift2nix itself could provide an "easy" interface, something that allows you to -write: - -```shell -nix-build -A swift2nix.release -nix-build -A swift2nix.test -``` - -The implementation of those would be obvious: create a new -`pkgs.stdenv.mkDerivation` and call `swift build -c release` and `swift test` -while using `swift2nix.env` under the hood. - -[node2nix]: https://github.com/svanderburg/node2nix - -## Conclusion - -Package managers should provide exact dependencies via a data representation, -i.e. lockfiles, and expose via another data representation how they expect those -dependencies to appear on the filesystem, i.e. `local-registry.json`. This -allows package managers to provide an API so that external tools can create -mirrors, offline builds, other registries, isolated builds, *etc.* - -"\*2nix" tools should build simple functions that leverage that -`local-registry.json`[^local-registry] data and offload all the rest back to the -package manager itself. This allows the "*2nix" to not keep chasing the package -manager evolution, always trying to duplicate its behaviour. - -[^local-registry]: This `local-registry.json` file doesn't have to be checked-in - the repository at all. It could be always generated on the fly, much like - how Swift's `dependencies-state.json` is. |