1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
|
#
msgid ""
msgstr ""
msgid ""
"title: \"swift2nix: Run Swift inside Nix builds\"\n"
"date: 2020-10-05 1\n"
"layout: post\n"
"lang: en\n"
"ref: swift2nix-run-swift-inside-nix-builds\n"
"category: mediator"
msgstr ""
msgid ""
"While working on a Swift project, I didn't find any tool that would allow "
"Swift to run inside [Nix](https://nixos.org/) builds. Even thought you *can*"
" run Swift, the real problem arises when using the package manager. It has "
"many of the same problems that other package managers have when trying to "
"integrate with Nix, more on this below."
msgstr ""
msgid ""
"I wrote a simple little tool called "
"[swift2nix](https://git.euandreh.xyz/swift2nix/about/) that allows you trick"
" Swift's package manager into assuming everything is set up. Here's the "
"example from swift2nix's README file:"
msgstr ""
msgid ""
"The key parts are lines 15~17: we just fake enough files inside `.build/` "
"that Swift believes it has already downloaded and checked-out all "
"dependencies, and just moves on to building them."
msgstr ""
msgid ""
"I've worked on it just enough to make it usable for myself, so beware of "
"unimplemented cases. Patches welcome."
msgstr ""
msgid "Design"
msgstr ""
msgid ""
"What swift2nix does is just provide you with the bare minimum that Swift "
"requires, and readily get out of the way:"
msgstr ""
msgid ""
"I explicitly did not want to generated a `Package.nix` file, since "
"`Package.resolved` already exists and contains the required information;"
msgstr ""
msgid ""
"I didn't want to have an \"easy\" interface right out of the gate, after "
"fighting with \"*2nix\" tools that focus too much on that."
msgstr ""
msgid ""
"The final [actual "
"code](https://git.euandreh.xyz/swift2nix/tree/default.nix?id=2af83ffe43fac631a8297ffaa8be3ff93b2b9e7c)"
" was so small (46 lines) that it made me think about package managers, "
"\"*2nix\" tools and some problems with many of them."
msgstr ""
msgid "Problems with package managers"
msgstr ""
msgid ""
"I'm going to talk about solely language package managers. Think npm and "
"cargo, not apt-get."
msgstr ""
msgid ""
"Package managers want to do too much, or assume too much, or just want to "
"take control of the entire build of the dependencies."
msgstr ""
msgid ""
"This is a recurrent problem in package managers, but I don't see it as an "
"intrinsic one. There's nothing about a \"package manager\" that prevents it "
"from *declaring* what it expects to encounter and in which format. The "
"*declaring* part is important: it should be data, not code, otherwise you're"
" back in the same problem, just like lockfiles are just data. Those work in "
"any language, and tools can cooperate happily."
msgstr ""
msgid ""
"There's no need for this declarative expectation to be standardized, or be "
"made compatible across languages. That would lead to a poor format that no "
"package manager really likes. Instead, If every package manager could say "
"out loud what it wants to see exactly, than more tools like swift2nix could "
"exist, and they would be more reliable."
msgstr ""
msgid ""
"This could even work fully offline, and be simply a mapping from the "
"lockfile (the `Package.resolved` in Swift's case) to the filesystem "
"representation. For Swift, the `.build/dependencies-state.json` comes very "
"close, but it is internal to the package manager."
msgstr ""
msgid ""
"Even though this pain only exists when trying to use Swift inside Nix, it "
"sheds light into this common implicit coupling that package managers have. "
"They usually have fuzzy boundaries and tight coupling between:"
msgstr ""
msgid ""
"resolving the dependency tree and using some heuristic to pick a package "
"version;"
msgstr ""
msgid "generating a lockfile with the exact pinned versions;"
msgstr ""
msgid ""
"downloading the dependencies present on the lockfile into some local cache;"
msgstr ""
msgid ""
"arranging the dependencies from the cache in a meaningful way for itself "
"inside the project;"
msgstr ""
msgid "work using the dependencies while *assuming* that step 4 was done."
msgstr ""
msgid ""
"When you run `npm install` in a repository with no lockfile, it does 1~~4. "
"If you do the same with `cargo build`, it does 1~~5. That's too much: many "
"of those assumptions are implicit and internal to the package manager, and "
"if you ever need to rearrange them, you're on your own. Even though you can "
"perform some of those steps, you can't compose or rearrange them."
msgstr ""
msgid "Instead a much saner approach could be:"
msgstr ""
msgid "this stays the same;"
msgstr ""
msgid "this also stays the same;"
msgstr ""
msgid ""
"be able to generate some JSON/TOML/edn which represents the local expected "
"filesystem layout with dependencies (i.e. exposing what the package manager "
"expects to find), let's call it `local-registry.json`;"
msgstr ""
msgid ""
"if a `local-registry.json` was provided, do a build using that. Otherwise "
"generate its own, by downloading the dependencies, arranging them, *etc.*"
msgstr ""
msgid ""
"The point is just making what the package manager requires visible to the "
"outside world via some declarative data. If this data wasn't provided, it "
"can move on to doing its own automatic things."
msgstr ""
msgid ""
"By making the expectation explicit and public, one can plug tools *à la "
"carte* if desired, but doesn't prevent the default code path of doing things"
" the exact same way they are now."
msgstr ""
msgid "Problems with \"*2nix\" tools"
msgstr ""
msgid "I have to admit: I'm unhappy with most of they."
msgstr ""
msgid ""
"They conflate \"using Nix\" with \"replicating every command of the package "
"manager inside Nix\"."
msgstr ""
msgid ""
"The avoidance of an \"easy\" interface that I mentioned above comes from me "
"fighting with some of the \"\\*2nix\" tools much like I have to fight with "
"package managers: I don't want to offload all build responsibilities to the "
"\"*2nix\" tool, I just want to let it download some of the dependencies and "
"get out of the way. I want to stick with `npm test` or `cargo build`, and "
"Nix should only provide the environment."
msgstr ""
msgid ""
"This is something that [node2nix](https://github.com/svanderburg/node2nix) "
"does right. It allows you to build the Node.js environment to satisfy NPM, "
"and you can keep using NPM for everything else:"
msgstr ""
msgid ""
"Its natural to want to put as much things into Nix as possible to benefit "
"from Nix's advantages. Isn't that how NixOS itself was born?"
msgstr ""
msgid ""
"But a \"*2nix\" tool should leverage Nix, not be coupled with it. The above "
"example lets you run any arbitrary NPM command while profiting from "
"isolation and reproducibility that Nix provides. It is even less brittle: "
"any changes to how NPM runs some things will be future-compatible, since "
"node2nix isn't trying to replicate what NPM does, or fiddling with NPM's "
"internal."
msgstr ""
msgid ""
"\\**A \"*2nix\" tool should build the environment, preferably from the "
"lockfile directly and offload everything else to the package manager**. The "
"rest is just nice-to-have."
msgstr ""
msgid ""
"swift2nix itself could provide an \"easy\" interface, something that allows "
"you to write:"
msgstr ""
msgid ""
"The implementation of those would be obvious: create a new "
"`pkgs.stdenv.mkDerivation` and call `swift build -c release` and `swift "
"test` while using `swift2nix.env` under the hood."
msgstr ""
msgid "Conclusion"
msgstr ""
msgid ""
"Package managers should provide exact dependencies via a data "
"representation, i.e. lockfiles, and expose via another data representation "
"how they expect those dependencies to appear on the filesystem, i.e. `local-"
"registry.json`. This allows package managers to provide an API so that "
"external tools can create mirrors, offline builds, other registries, "
"isolated builds, *etc.*"
msgstr ""
msgid ""
"\"\\*2nix\" tools should build simple functions that leverage that `local-"
"registry.json`[^local-registry] data and offload all the rest back to the "
"package manager itself. This allows the \"*2nix\" to not keep chasing the "
"package manager evolution, always trying to duplicate its behaviour."
msgstr ""
msgid ""
"[^local-registry]: This `local-registry.json` file doesn't have to be "
"checked-in the repository at all. It could be always generated on the fly, "
"much like how Swift's `dependencies-state.json` is."
msgstr ""
|
