[tor-bugs] #31239 [Internal Services/Tor Sysadmin Team]: automate installs

[Tor Bug Tracker & Wiki]

#31239: automate installs
-------------------------------------------------+-------------------------
 Reporter:  anarcat                              |          Owner:  anarcat
     Type:  enhancement                          |         Status:
                                                 |  assigned
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by anarcat):

 in #32902, hiro and I played with draw.io to draw diagrams of what the
 current install process looks like. it was a fun exercise, and showed a
 few interesting things:

  * too much duplication between the two disk formatters, which should be
 resolved
  * duplication between the disk formatters and luks-setup
  * inconsistencies between sites: hrobot writes authorized-keys in
 /root/.ssh, hcloud in /etc/ssh/userkeys/, one uses grml-debootstrap, the
 other debootstrap

 I'm leaning towards scrapping the current install process and converging
 towards a simpler process that would be basically:

  1. pick IP address, hostname and other static parameters
  2. create metal/cloud upstream
  3. get a console (ssh, web console, whatever)
  4. use [https://manpages.debian.org/setup-storage setup-storage] to
 partition the disk, based on well-defined templates
  5. mount everything
  6. run debootstrap
  7. setup network, including hostname (maybe reusing gnt-network stuff?)
  8. populate LDAP
  9. bootstrap Puppet in the chroot
  10. reboot

 Every remaining manual step can then be done in Puppet, as it runs before
 the first boot. Those steps, currently done manually, are already done by
 Puppet so automating this is just a matter of ordering:

  * SSH daemon and keys configuration
  * automated upgrades (part of the larger #31957)
  * /etc/hosts management?

 Those would need some coding work in Puppet:

  * root password management (trocla? abandon?)
  * swapfile (move to setup-storage?)
  * kernel and grub setup?
  * mdadm.conf, fstab and crypttab config (setup-storage?)
  * dropbear-initramfs setup
  * mandos setup
  * net.ifnames=0

 Those steps would stay manual until they are configured in Puppet.

 So the next step seems to be to experiment with changing the order of the
 install process to bootstrap Puppet earlier and see what happens. We
 should also experiment with a different partionning tool, probably setup-
 storage.

 TL;DR: next steps:

  1. test setup-storage
  2. bootstrap Puppet earlier

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/31239#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online