1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
* Tasks
** DONE Provision DigitalOcean's droplet from Terraform
CLOSED: [2019-05-25 Sat 13:29]
** DONE Properly provision Ubuntu droplet
CLOSED: [2019-05-25 Sat 17:50]
** DOING Automate deployment of updates
*** DOING Subtasks
**** DONE Fix Debian import of GPG keys
CLOSED: [2019-05-26 Sun 14:34]
While NixOS image isn't fixed, use Debian instead.
The GPG data was all in a single line. I copied and pasted it properly and it was identified correctly.
**** CANCELLED Properly install Nix on Debian image
NixOS patch was applied.
**** DONE Fix NixOS GPG key importing in builds.sr.ht
CLOSED: [2019-05-26 Sun 17:37]
See patch and discussion in [[https://lists.sr.ht/~sircmpwn/sr.ht-dev/%3C20190526162135.1646-1-eu%40euandre.org%3E][sr.ht-dev mailing list]].
**** DOING Use ssh configuration from environment instead of creating and alias for =ssh=
**** TODO Use DigitalOcean's Floating IP in front of the droplet
**** TODO Test provisioning DNS entries with other DNS registrars
**** DONE Namecheap whitelist IP limitation
CLOSED: [2019-05-26 Sun 17:14]
Namecheap requires you to specifically whitelist an IP that can perform changes to their API.
[[https://lists.sr.ht/~sircmpwn/sr.ht-discuss/%20%3CCAJk2QMbq8uE1pcG3Uy6w37HUY7W15cQ+sHoj-UBWN-W11AtcrA%40mail.gmail.com%3E][builds.sr.ht]] don't guarantee any specific IP, so whitelisting it isn't an option.
The best candidate so far is using DigitalOcean's Floating IP feature to link a hardcoded IP to a droplet, while the droplet's IP may change. This way any new deployment wouldn't change the public IP of the box, and wouldn't require me to change the DNS A and AAAA records on Namecheap.
This has also an advantage of allowing the email server to keep it's IP address.
The downside is that the deployment of DNS registries isn't fully automated: whenever I have to change a DNS entry, either for adding a new CNAME record or changing an AAAA record, I'll have to:
1- get my own IP;
2- whitelist it on Namecheap's web interface;
3- run a separate Terraform recipe.
The upside is that this should happen less often than a deployment, but still not ideal. The ideal would be to run the Terraform recipe every time, and Terraform would realize that there was no DNS related change and do nothing.
*** Limitations
During build, decrypt content of files and update the deployment.
How does Terraform tfstate file can be handled in this case?
UPDATE:
Terraform does support the so called "backends" to coordinate lock and usage of the =.tfstate= files. On this regard there are no restrictions on continously deploying with Terraform from the CI pipelines.
However the current applications do *not* properly support blue/green deployment, like email, Nextcloud, etc.
We could try to share a shared volume, but that would be a consistency nightmare.
The other option is to always recreate everything, with downtime. The advantage is that we get actual immutable deployments with stateful storage, but there would be downtime for every deployment. This is due to the nature of most of the packaged applications being single node *only*.
There's also the IP reputation issue: recreating everything from scratch every time would lead to new droplets with new IP addresses, which is not a good thing to be changing in a server box.
A reasonable alternative would be to redeploy everything on a different node, with a different TLD, and manually check that. But that would be just like an staging environment, with all of it's downsides too.
In this situation, I if go on with automating the deployment I'd rather pick the downtime option.
I'll start with other services other than email and consider alternatives later.
** TODO Use Digital Ocean's Volumes for persistent extended storage
** WAITING Configure DNS from Terraform
* Must
** Fully deployable from code
Use NixOps and Terraform to fully automate all of the configuration.
* Services
** DONE =euandreh.org=: Static webhosting
CLOSED: [2019-05-26 Sun 10:17]
** DONE =wallabag.euandreh.org=: Wallabag
CLOSED: [2019-05-25 Sat 18:02]
** TODO =mail.euandreh.org=: Email + webmail
** TODO =cloud.euandreh.org=: Nextcloud: storage, calendar, contacts, notes
** TODO =hydra.euandreh.org=: Hydra
Does Hydra support release management?
I'd like to release both pre-compiled binaries and Docker images.
** TODO =annex.euandreh.org=: Public content from Git Annex repositories
Only an static file server, with folders for individual assets.
** TODO =pi-hole.euandreh.org=: Pi-hole
** TODO =pwk.euandreh.org=: Piwik
** TODO =git.euandreh.org=: CGit or GitWeb
https://github.com/iconoeugen/docker-gitweb
** CANCELLED =perkeep.euandreh.org=: Perkeep
I'm already covered by using Git Annex for almost everything.
** WAITING =matrix.euandreh.org=: Matrix Synapse server
I'm not using IRC a lot right now. Wait for me to interact more with mailing lists and gauge the need of IRC.
* Questions
** DONE Do I want or need Docker? Should I use it?
CLOSED: [2019-05-25 Sat 18:1980]
It was a better path than sticking with NixOps and nixcloud-webservices. It's more widespread and has more things done for it.
** CANCELLED How to share the Nix store across services?
** DONE How to leverage DigitalOcean's block storage?
CLOSED: [2019-05-25 Sat 18:19]
Provision it using Terraform, and use it's path as the =$VOLUME_HOME= variable for containers.
This was I can compartimentalize the data storage to easily backup and duplicate, but also destroy a running droplet and create a new one.
* Nice to have
** =*.euandreh.org=
** Nix Terraform provisioning
* Resources
** [[https://github.com/mail-in-a-box/mailinabox][Mail-in-a-Box]]
** [[https://sealedabstract.com/code/nsa-proof-your-e-mail-in-2-hours/][NSA-proof your e-mail in 2 hours]]
** [[https://www.iredmail.org/][iRedMail]]
** [[https://blog.harveydelaney.com/hosting-websites-using-docker-nginx/][Hosting Multiple Websites with SSL using Docker, Nginx and a VPS]]
** [[https://github.com/sovereign/sovereign/][Sovereign]]
** [[https://github.com/nixcloud/nixcloud-webservices][nixcloud-webservices]]
** [[https://github.com/Kickball/awesome-selfhosted#email][Awesome-Selfhosted: Email]]
|
