[tor-project] minutes from the sysadmin meeting

[Antoine Beaupré]

Hello everyone!

We held our usual meeting a week late in April. But still, here are the
minutes, preceeded by a usual, carefully hand-crafted variable header by
your favorite editor (the human, not the software).

Stay safe, inside, wash your hands, and take care of each other. :)

# Roll call: who's there and emergencies

anarcat, hiro and weasel are present (gaba late)

# Roadmap review

We changed our meeting template to just do a live roadmap review from
Trac instead of listing all the little details of all the things we did
in the last month. The details are in Trac.

So we reviewed the roadmap at:

<https://trac.torproject.org/projects/tor/wiki/org/teams/SysadminTeam>

SVN, Solr got postponed to april. kvm3 wasn't completed either but
should be by the end of the week. hopefully kvm4 will be done by the
end of the month but is also likely to be postponed.

We might need to push on the buster upgrade schedule if we don't want
to miss the "pre-LTS" window.

We also note that we don't have a good plan for the GitLab deployment,
on the infrastructure side of things. We'll need to spend some time to
review the infra before anarcat leaves.

# Voice meetings

Anarcat and hiro have started doing weekly checkups, kind of
informally, on the last two mondays, and it was pretty amazing. We
didn't want to force a voice meeting on everyone without first
checking in, but maybe we could just switch to that model it's mostly
just hiro and anarcat every week anyways.

The possibilities considered were:

 1. we keep this thing where some people checkin by voice every week,
    but we keep a monthly text meeting
 2. we switch everything to voice
 3. we end the voice experiment completely and go back to
    text-monthly-only meetings

Anarcat objected to option 3, naturally, and favored 2. Hiro agreed to
try, and no one else objected.

A little bit of the rationale behind the discussion was discussed in
the meeting. IRC has the advantage that people can read logs if they
don't come. But we will keep minutes of the monthly meetings even if
they are by voice, so people can read those, which is better than
reading a backlog, because it's edited (by yours truly). And if people
miss the meeting, it's their responsability: there are announcements
and multiple reminders before the meeting, and they seem to have
little effect on attendance. So meetings are mostly hiro and anarcat,
with gaba and weasel sometimes joining in. So it makes little sense to
force IRC on those two workers to accomodate people that don't get
involved as much. Anarcat also feels the IRC meetings are too slow:
this meeting took 30 minutes to evaluate the roadmap, and did not get
much done. He estimates this would have taken only 10 minutes by voice
and the end result would have been similar, if not better: the tickets
would have been updated anyways.

So the plan for meetings is to have weekly checkins and a monthly
meeting, by voice, on Mumble.

 * weekly checkins: timeboxed to 15 minutes, with an optional 45
   minutes worksession after if needed
 * monthly meetings: like the current IRC meetings, except by
   voice. timeboxed to 60 minutes still, replacing the weekly checkin
   for that week

We use Mumble for now, but we could consider other
platforms. (Somewhat off-topic: Anarcat wrote a [review of the Mumble
UX][] that was somewhat poorly received by the Mumble team, so don't
get your hopes up about the Mumble UI improving.)

 [review of the Mumble UX]: https://anarc.at/blog/2020-04-09-mumble-dreams/

# Other discussions

No other discussion was brought up.

# Next meeting

Next "first monday of the month", which is 2020-05-04 15:00UTC
(11:00:00EDT, 17:00CET).

# Metrics of the month

 * hosts in Puppet: 76, LDAP: 80, Prometheus exporters: 123
 * number of apache servers monitored: 31, hits per second: 168
 * number of nginx servers: 2, hits per second: 2, hit ratio: 0.89
 * number of self-hosted nameservers: 6, mail servers: 10
 * pending upgrades: 1, reboots: 0
 * average load: 1.03, memory available: 322.57 GiB/1017.81 GiB,
   running processes: 460
 * bytes sent: 211.97 MB/s, received: 123.01 MB/s
 * completion time of stretch major upgrades: 2020-07-16

Upgrade prediction graph still lives at https://help.torproject.org/tsa/howto/upgrades/

Now also available as the main Grafana dashboard. Head to
<https://grafana.torproject.org/>, change the time period to 30 days,
and wait a while for results to render.

-- 
Antoine Beaupré
torproject.org system administration
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20200414/11baae93/attachment.sig>