aboutsummaryrefslogtreecommitdiff
path: root/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md
diff options
context:
space:
mode:
authorEuAndreh <eu@euandre.org>2024-11-18 08:21:58 -0300
committerEuAndreh <eu@euandre.org>2024-11-18 08:44:57 -0300
commit960e4410f76801356ebd42801c914b2910a302a7 (patch)
tree615d379416f72956d0c1666c63ce062859041fbe /_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md
parentRemove jekyll infrastructure setup (diff)
downloadeuandre.org-960e4410f76801356ebd42801c914b2910a302a7.tar.gz
euandre.org-960e4410f76801356ebd42801c914b2910a302a7.tar.xz
v0 migration to mkwbHEADmain
Diffstat (limited to '_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md')
-rw-r--r--_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md201
1 files changed, 0 insertions, 201 deletions
diff --git a/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md b/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md
deleted file mode 100644
index c922589..0000000
--- a/_articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md
+++ /dev/null
@@ -1,201 +0,0 @@
----
-
-title: "swift2nix: Run Swift inside Nix builds"
-
-date: 2020-10-05 1
-
-layout: post
-
-lang: en
-
-ref: swift2nix-run-swift-inside-nix-builds
-
----
-
-While working on a Swift project, I didn't find any tool that would allow Swift
-to run inside [Nix][nix] builds. Even thought you *can* run Swift, the real
-problem arises when using the package manager. It has many of the same problems
-that other package managers have when trying to integrate with Nix, more on this
-below.
-
-I wrote a simple little tool called [swift2nix] that allows you trick
-Swift's package manager into assuming everything is set up. Here's the example
-from swift2nix's README file:
-
-```
-let
- niv-sources = import ./nix/sources.nix;
- pkgs = import niv-sources.nixpkgs { };
- src = pkgs.nix-gitignore.gitignoreSource [ ] ./.;
- swift2nix = pkgs.callPackage niv-sources.swift2nix {
- package-resolved = ./Package.resolved;
- };
-in pkgs.stdenv.mkDerivation {
- inherit src;
- name = "swift-test";
- buildInputs = with pkgs; [ swift ];
- phases = [ "unpackPhase" "buildPhase" ];
- buildPhase = ''
- # Setup dependencies path to satisfy SwiftPM
- mkdir .build
- ln -s ${swift2nix.env.dependencies-state-json} .build/dependencies-state.json
- ln -s ${swift2nix.env.checkouts} .build/checkouts
-
- # Run the tests
- swift test
- touch $out
- '';
-}
-```
-
-The key parts are lines 15~17: we just fake enough files inside `.build/` that
-Swift believes it has already downloaded and checked-out all dependencies, and
-just moves on to building them.
-
-I've worked on it just enough to make it usable for myself, so beware of
-unimplemented cases.
-
-[nix]: https://nixos.org/
-[swift2nix]: https://euandre.org/static/attachments/swift2nix.tar.gz
-
-## Design
-
-What swift2nix does is just provide you with the bare minimum that Swift
-requires, and readily get out of the way:
-
-1. I explicitly did not want to generated a `Package.nix` file, since
- `Package.resolved` already exists and contains the required information;
-2. I didn't want to have an "easy" interface right out of the gate, after
- fighting with "*2nix" tools that focus too much on that.
-
-The final actual code was so small (46 lines) that it made me
-think about package managers, "*2nix" tools and some problems with many of them.
-
-## Problems with package managers
-
-I'm going to talk about solely language package managers. Think npm and cargo,
-not apt-get.
-
-Package managers want to do too much, or assume too much, or just want to take
-control of the entire build of the dependencies.
-
-This is a recurrent problem in package managers, but I don't see it as an
-intrinsic one. There's nothing about a "package manager" that prevents it from
-*declaring* what it expects to encounter and in which format. The *declaring*
-part is important: it should be data, not code, otherwise you're back in the
-same problem, just like lockfiles are just data. Those work in any language, and
-tools can cooperate happily.
-
-There's no need for this declarative expectation to be standardized, or be made
-compatible across languages. That would lead to a poor format that no package
-manager really likes. Instead, If every package manager could say out loud what
-it wants to see exactly, than more tools like swift2nix could exist, and they
-would be more reliable.
-
-This could even work fully offline, and be simply a mapping from the lockfile
-(the `Package.resolved` in Swift's case) to the filesystem representation. For
-Swift, the `.build/dependencies-state.json` comes very close, but it is internal
-to the package manager.
-
-Even though this pain only exists when trying to use Swift inside Nix, it sheds
-light into this common implicit coupling that package managers have. They
-usually have fuzzy boundaries and tight coupling between:
-
-1. resolving the dependency tree and using some heuristic to pick a package
- version;
-2. generating a lockfile with the exact pinned versions;
-3. downloading the dependencies present on the lockfile into some local cache;
-4. arranging the dependencies from the cache in a meaningful way for itself inside
- the project;
-5. work using the dependencies while *assuming* that step 4 was done.
-
-When you run `npm install` in a repository with no lockfile, it does 1~4. If you
-do the same with `cargo build`, it does 1~5. That's too much: many of those
-assumptions are implicit and internal to the package manager, and if you ever
-need to rearrange them, you're on your own. Even though you can perform some of
-those steps, you can't compose or rearrange them.
-
-Instead a much saner approach could be:
-
-1. this stays the same;
-2. this also stays the same;
-3. be able to generate some JSON/TOML/edn which represents the local expected
- filesystem layout with dependencies (i.e. exposing what the package manager
- expects to find), let's call it `local-registry.json`;
-4. if a `local-registry.json` was provided, do a build using that. Otherwise
- generate its own, by downloading the dependencies, arranging them, *etc.*
-
-The point is just making what the package manager requires visible to the
-outside world via some declarative data. If this data wasn't provided, it can
-move on to doing its own automatic things.
-
-By making the expectation explicit and public, one can plug tools *à la carte*
-if desired, but doesn't prevent the default code path of doing things the exact
-same way they are now.
-
-## Problems with "*2nix" tools
-
-I have to admit: I'm unhappy with most of they.
-
-They conflate "using Nix" with "replicating every command of the package manager
-inside Nix".
-
-The avoidance of an "easy" interface that I mentioned above comes from me
-fighting with some of the "\*2nix" tools much like I have to fight with package
-managers: I don't want to offload all build responsibilities to the "*2nix"
-tool, I just want to let it download some of the dependencies and get out of the
-way. I want to stick with `npm test` or `cargo build`, and Nix should only
-provide the environment.
-
-This is something that [node2nix] does right. It allows you to build
-the Node.js environment to satisfy NPM, and you can keep using NPM for
-everything else:
-
-```shell
-ln -s ${node2nix-package.shell.nodeDependencies}/lib/node_modules ./node_modules
-npm test
-```
-
-Its natural to want to put as much things into Nix as possible to benefit from
-Nix's advantages. Isn't that how NixOS itself was born?
-
-But a "*2nix" tool should leverage Nix, not be coupled with it. The above
-example lets you run any arbitrary NPM command while profiting from isolation
-and reproducibility that Nix provides. It is even less brittle: any changes to
-how NPM runs some things will be future-compatible, since node2nix isn't trying
-to replicate what NPM does, or fiddling with NPM's internal.
-
-**A "*2nix" tool should build the environment, preferably from the lockfile
-directly and offload everything else to the package manager**. The rest is just
-nice-to-have.
-
-swift2nix itself could provide an "easy" interface, something that allows you to
-write:
-
-```shell
-nix-build -A swift2nix.release
-nix-build -A swift2nix.test
-```
-
-The implementation of those would be obvious: create a new
-`pkgs.stdenv.mkDerivation` and call `swift build -c release` and `swift test`
-while using `swift2nix.env` under the hood.
-
-[node2nix]: https://github.com/svanderburg/node2nix
-
-## Conclusion
-
-Package managers should provide exact dependencies via a data representation,
-i.e. lockfiles, and expose via another data representation how they expect those
-dependencies to appear on the filesystem, i.e. `local-registry.json`. This
-allows package managers to provide an API so that external tools can create
-mirrors, offline builds, other registries, isolated builds, *etc.*
-
-"\*2nix" tools should build simple functions that leverage that
-`local-registry.json`[^local-registry] data and offload all the rest back to the
-package manager itself. This allows the "*2nix" to not keep chasing the package
-manager evolution, always trying to duplicate its behaviour.
-
-[^local-registry]: This `local-registry.json` file doesn't have to be checked-in
- the repository at all. It could be always generated on the fly, much like
- how Swift's `dependencies-state.json` is.