[tor-bugs] #33406 [Internal Services/Tor Sysadmin Team]: automate reboots
Tor Bug Tracker & Wiki
blackhole at torproject.org
Fri Feb 21 14:51:51 UTC 2020
#33406: automate reboots
-------------------------------------------------+---------------------
Reporter: anarcat | Owner: tpa
Type: project | Status: new
Priority: Low | Milestone:
Component: Internal Services/Tor Sysadmin Team | Version:
Severity: Major | Resolution:
Keywords: tpa-roadmap-march | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-------------------------------------------------+---------------------
Comment (by anarcat):
just for future reference, ganeti-reboot-cluster, as we have in our puppet
repo, doesn't work in our cluster, because it relies on assumptions
specific to the DSA clusters (namely that the last node is an empty
spare). so it fails with:
{{{
fsn-node-03.torproject.org not empty.
}}}
apparently, the latest version of the script might fix that with the
`crossmigratemany` function:
https://salsa.debian.org/dsa-team/mirror/dsa-
puppet/raw/master/modules/ganeti2/files/ganeti-reboot-cluster
for now, i'll just do the reboot by hand.
in theory, rebooting a ganeti node is to:
1. migrate all the primaries off of the node: `ssh $master gnt-migrate
$node`
2. if it's a master, promote another master: `ssh $notmaster gnt-cluster
master-failover` (optional, only if we can't afford having the master down
during the reboot)
3. reboot the node `ssh $node reboot`
... for each node.
i'm testing that procedure on fsn-node-03 now.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33406#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list