[tor-bugs] #33406 [Internal Services/Tor Sysadmin Team]: automate reboots
Tor Bug Tracker & Wiki
blackhole at torproject.org
Thu Feb 20 23:52:11 UTC 2020
#33406: automate reboots
-------------------------------------------------+---------------------
Reporter: anarcat | Owner: tpa
Type: project | Status: new
Priority: Low | Milestone:
Component: Internal Services/Tor Sysadmin Team | Version:
Severity: Major | Resolution:
Keywords: tpa-roadmap-march | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-------------------------------------------------+---------------------
Comment (by anarcat):
note that this may very well mean just removing tsa-misc/reboot-host and
tsa-misc/reboot-guest, and documenting the multi-tool better. :)
i just tried `./torproject-reboot-rotation` and ``./torproject-reboot-
simple` and the unattended operation isn't great... it fires up all those
reboots, and doesn't show clearly what it did. for example, it seems to
have queued reboots on a bunch of hosts, but it doesn't say which.
after further inspection (with `cumin '*' 'screen -ls | grep reboot-
job'`), i have found it has scheduled reboots on
* static-master-fsn.torproject.org
* cdn-backend-sunet-01.torproject.org
* web-fsn-01.torproject.org
* onionoo-frontend-01.torproject.org
* orestis.torproject.org
* nutans.torproject.org
* chives.torproject.org
* onionbalance-01.torproject.org
* listera.torproject.org
* peninsulare.torproject.org
Most of those are okay and should return unattended. But in some cases,
those could have been covered by a libvirt reboot (i had performed those
before, in this case, so non were). Eventually though, that point is moot
because we'll all be running under ganeti and will separate host and guest
reboot procedures.
one host is problematic in there (chives) as it needs a specific warning
to users. maybe chives should be taken out of "justdoit" rotation...
i also wonder, in general, if we should warn users about those reboots, as
part of the reboot script.
then i don't know which hosts are left to do manually, but i guess that,
with time, nagios will let us know. it would be nice to have a scenario
for those as well.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33406#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list