6 files changed, 279 insertions, 0 deletions
diff --git a/src/content/blog/2020/10/05/cargo2nix-demo.tar.gz b/src/content/blog/2020/10/05/cargo2nix-demo.tar.gz
new file mode 100644
index 0000000..281a91c
--- /dev/null
+++ b/src/content/blog/2020/10/05/cargo2nix-demo.tar.gz
diff --git a/src/content/blog/2020/10/05/cargo2nix.adoc b/src/content/blog/2020/10/05/cargo2nix.adoc
new file mode 100644
index 0000000..1db3d0c
--- /dev/null
+++ b/src/content/blog/2020/10/05/cargo2nix.adoc
@@ -0,0 +1,80 @@
+= cargo2nix: Dramatically simpler Rust in Nix
+
+date: 2020-10-05 2
+
+layout: post
+
+lang: en
+
+ref: cargo2nix-dramatically-simpler-rust-in-nix
+
+---
+
+In the same vein of my earlier post on
+[swift2nix]({% link _articles/2020-10-05-swift2nix-run-swift-inside-nix-builds.md %}), I
+was able to quickly prototype a Rust and Cargo variation of it:
+[cargo2nix].
+
+
+The initial prototype is even smaller than swift2nix: it has only
+37 lines of code.
+
+[cargo2nix]: https://euandre.org/static/attachments/cargo2nix.tar.gz
+
+Here's how to use it (snippet taken from the repo's README):
+
+```nix
+let
+  niv-sources = import ./nix/sources.nix;
+  mozilla-overlay = import niv-sources.nixpkgs-mozilla;
+  pkgs = import niv-sources.nixpkgs { overlays = [ mozilla-overlay ]; };
+  src = pkgs.nix-gitignore.gitignoreSource [ ] ./.;
+  cargo2nix = pkgs.callPackage niv-sources.cargo2nix {
+    lockfile = ./Cargo.lock;
+  };
+in pkgs.stdenv.mkDerivation {
+  inherit src;
+  name = "cargo-test";
+  buildInputs = [ pkgs.latest.rustChannels.nightly.rust ];
+  phases = [ "unpackPhase" "buildPhase" ];
+  buildPhase = ''
+    # Setup dependencies path to satisfy Cargo
+    mkdir .cargo/
+    ln -s ${cargo2nix.env.cargo-config} .cargo/config
+    ln -s ${cargo2nix.env.vendor} vendor
+
+    # Run the tests
+    cargo test
+    touch $out
+  '';
+}
+```
+
+That `cargo test` part on line 20 is what I have been fighting with every
+"\*2nix" available for Rust out there. I don't want to bash any of them. All I
+want is to have full control of what Cargo commands to run, and the "*2nix" tool
+should only setup the environment for me. Let me drive Cargo myself, no need to
+parameterize how the tool runs it for me, or even replicate its internal
+behaviour by calling the Rust compiler directly.
+
+Sure it doesn't support private registries or Git dependencies, but how much
+bigger does it has to be to support them? Also, it doesn't support those **yet**,
+there's no reason it can't be extended. I just haven't needed it yet, so I
+haven't added. Patches welcome.
+
+The layout of the `vendor/` directory is more explicit and public then what
+swift2nix does: it is whatever the command `cargo vendor` returns. However I
+haven't checked if the shape of the `.cargo-checksum.json` is specified, or
+internal to Cargo.
+
+Try out the demo (also taken from the repo's README):
+
+```shell
+pushd "$(mktemp -d)"
+wget -O- https://euandre.org/static/attachments/cargo2nix-demo.tar.gz |
+	tar -xv
+cd cargo2nix-demo/
+nix-build
+```
+
+Report back if you wish.
diff --git a/src/content/blog/2020/10/05/cargo2nix.tar.gz b/src/content/blog/2020/10/05/cargo2nix.tar.gz
new file mode 100644
index 0000000..8a9985a
--- /dev/null
+++ b/src/content/blog/2020/10/05/cargo2nix.tar.gz
diff --git a/src/content/blog/2020/10/05/swift2nix-demo.tar.gz b/src/content/blog/2020/10/05/swift2nix-demo.tar.gz
new file mode 100644
index 0000000..f688572
--- /dev/null
+++ b/src/content/blog/2020/10/05/swift2nix-demo.tar.gz
diff --git a/src/content/blog/2020/10/05/swift2nix.adoc b/src/content/blog/2020/10/05/swift2nix.adoc
new file mode 100644
index 0000000..84f4d34
--- /dev/null
+++ b/src/content/blog/2020/10/05/swift2nix.adoc
@@ -0,0 +1,199 @@
+= swift2nix: Run Swift inside Nix builds
+
+date: 2020-10-05 1
+
+layout: post
+
+lang: en
+
+ref: swift2nix-run-swift-inside-nix-builds
+
+---
+
+While working on a Swift project, I didn't find any tool that would allow Swift
+to run inside [Nix][nix] builds. Even thought you *can* run Swift, the real
+problem arises when using the package manager. It has many of the same problems
+that other package managers have when trying to integrate with Nix, more on this
+below.
+
+I wrote a simple little tool called [swift2nix] that allows you trick
+Swift's package manager into assuming everything is set up. Here's the example
+from swift2nix's README file:
+
+```
+let
+  niv-sources = import ./nix/sources.nix;
+  pkgs = import niv-sources.nixpkgs { };
+  src = pkgs.nix-gitignore.gitignoreSource [ ] ./.;
+  swift2nix = pkgs.callPackage niv-sources.swift2nix {
+    package-resolved = ./Package.resolved;
+  };
+in pkgs.stdenv.mkDerivation {
+  inherit src;
+  name = "swift-test";
+  buildInputs = with pkgs; [ swift ];
+  phases = [ "unpackPhase" "buildPhase" ];
+  buildPhase = ''
+    # Setup dependencies path to satisfy SwiftPM
+    mkdir .build
+    ln -s ${swift2nix.env.dependencies-state-json} .build/dependencies-state.json
+    ln -s ${swift2nix.env.checkouts} .build/checkouts
+
+    # Run the tests
+    swift test
+    touch $out
+  '';
+}
+```
+
+The key parts are lines 15~17: we just fake enough files inside `.build/` that
+Swift believes it has already downloaded and checked-out all dependencies, and
+just moves on to building them.
+
+I've worked on it just enough to make it usable for myself, so beware of
+unimplemented cases.
+
+[nix]: https://nixos.org/
+[swift2nix]: https://euandre.org/static/attachments/swift2nix.tar.gz
+
+## Design
+
+What swift2nix does is just provide you with the bare minimum that Swift
+requires, and readily get out of the way:
+
+1. I explicitly did not want to generated a `Package.nix` file, since
+   `Package.resolved` already exists and contains the required information;
+2. I didn't want to have an "easy" interface right out of the gate, after
+   fighting with "*2nix" tools that focus too much on that.
+
+The final actual code was so small (46 lines) that it made me
+think about package managers, "*2nix" tools and some problems with many of them.
+
+## Problems with package managers
+
+I'm going to talk about solely language package managers. Think npm and cargo,
+not apt-get.
+
+Package managers want to do too much, or assume too much, or just want to take
+control of the entire build of the dependencies.
+
+This is a recurrent problem in package managers, but I don't see it as an
+intrinsic one. There's nothing about a "package manager" that prevents it from
+*declaring* what it expects to encounter and in which format. The *declaring*
+part is important: it should be data, not code, otherwise you're back in the
+same problem, just like lockfiles are just data. Those work in any language, and
+tools can cooperate happily.
+
+There's no need for this declarative expectation to be standardized, or be made
+compatible across languages. That would lead to a poor format that no package
+manager really likes. Instead, If every package manager could say out loud what
+it wants to see exactly, than more tools like swift2nix could exist, and they
+would be more reliable.
+
+This could even work fully offline, and be simply a mapping from the lockfile
+(the `Package.resolved` in Swift's case) to the filesystem representation. For
+Swift, the `.build/dependencies-state.json` comes very close, but it is internal
+to the package manager.
+
+Even though this pain only exists when trying to use Swift inside Nix, it sheds
+light into this common implicit coupling that package managers have. They
+usually have fuzzy boundaries and tight coupling between:
+
+1. resolving the dependency tree and using some heuristic to pick a package
+   version;
+2. generating a lockfile with the exact pinned versions;
+3. downloading the dependencies present on the lockfile into some local cache;
+4. arranging the dependencies from the cache in a meaningful way for itself inside
+   the project;
+5. work using the dependencies while *assuming* that step 4 was done.
+
+When you run `npm install` in a repository with no lockfile, it does 1~4. If you
+do the same with `cargo build`, it does 1~5. That's too much: many of those
+assumptions are implicit and internal to the package manager, and if you ever
+need to rearrange them, you're on your own. Even though you can perform some of
+those steps, you can't compose or rearrange them.
+
+Instead a much saner approach could be:
+
+1. this stays the same;
+2. this also stays the same;
+3. be able to generate some JSON/TOML/edn which represents the local expected
+   filesystem layout with dependencies (i.e. exposing what the package manager
+   expects to find), let's call it `local-registry.json`;
+4. if a `local-registry.json` was provided, do a build using that. Otherwise
+   generate its own, by downloading the dependencies, arranging them, *etc.*
+
+The point is just making what the package manager requires visible to the
+outside world via some declarative data. If this data wasn't provided, it can
+move on to doing its own automatic things.
+
+By making the expectation explicit and public, one can plug tools *à la carte*
+if desired, but doesn't prevent the default code path of doing things the exact
+same way they are now.
+
+## Problems with "*2nix" tools
+
+I have to admit: I'm unhappy with most of they.
+
+They conflate "using Nix" with "replicating every command of the package manager
+inside Nix".
+
+The avoidance of an "easy" interface that I mentioned above comes from me
+fighting with some of the "\*2nix" tools much like I have to fight with package
+managers: I don't want to offload all build responsibilities to the "*2nix"
+tool, I just want to let it download some of the dependencies and get out of the
+way. I want to stick with `npm test` or `cargo build`, and Nix should only
+provide the environment.
+
+This is something that [node2nix] does right. It allows you to build
+the Node.js environment to satisfy NPM, and you can keep using NPM for
+everything else:
+
+```shell
+ln -s ${node2nix-package.shell.nodeDependencies}/lib/node_modules ./node_modules
+npm test
+```
+
+Its natural to want to put as much things into Nix as possible to benefit from
+Nix's advantages. Isn't that how NixOS itself was born?
+
+But a "*2nix" tool should leverage Nix, not be coupled with it. The above
+example lets you run any arbitrary NPM command while profiting from isolation
+and reproducibility that Nix provides. It is even less brittle: any changes to
+how NPM runs some things will be future-compatible, since node2nix isn't trying
+to replicate what NPM does, or fiddling with NPM's internal.
+
+**A "*2nix" tool should build the environment, preferably from the lockfile
+directly and offload everything else to the package manager**. The rest is just
+nice-to-have.
+
+swift2nix itself could provide an "easy" interface, something that allows you to
+write:
+
+```shell
+nix-build -A swift2nix.release
+nix-build -A swift2nix.test
+```
+
+The implementation of those would be obvious: create a new
+`pkgs.stdenv.mkDerivation` and call `swift build -c release` and `swift test`
+while using `swift2nix.env` under the hood.
+
+[node2nix]: https://github.com/svanderburg/node2nix
+
+## Conclusion
+
+Package managers should provide exact dependencies via a data representation,
+i.e. lockfiles, and expose via another data representation how they expect those
+dependencies to appear on the filesystem, i.e. `local-registry.json`. This
+allows package managers to provide an API so that external tools can create
+mirrors, offline builds, other registries, isolated builds, *etc.*
+
+"\*2nix" tools should build simple functions that leverage that
+`local-registry.json`[^local-registry] data and offload all the rest back to the
+package manager itself. This allows the "*2nix" to not keep chasing the package
+manager evolution, always trying to duplicate its behaviour.
+
+[^local-registry]: This `local-registry.json` file doesn't have to be checked-in
+    the repository at all. It could be always generated on the fly, much like
+    how Swift's `dependencies-state.json` is.
diff --git a/src/content/blog/2020/10/05/swift2nix.tar.gz b/src/content/blog/2020/10/05/swift2nix.tar.gz
new file mode 100644
index 0000000..bfab3f1
--- /dev/null
+++ b/src/content/blog/2020/10/05/swift2nix.tar.gz