[tor-project] GitLab Runner updates

[Jim Newsome]

On 6/16/22 08:55, Antoine Beaupré wrote:
<snip>
> In general, you shouldn't really *trust* GitLab or GitLab CI for
> anything else than running tests. Builds should be verified out of band
> with reproducible builds. You can reproduce a local GitLab CI
> environment by installing gitlab-runner and executing jobs locally,
> without having to trust the entire GitLab installation or foreign
> runners. As a reminder, it is your responsibility to ensure the
> integrity of your code and artifacts, see those links for a further
> discussion:
> 
> https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/81
> https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/gitlab#git-repository-integrity-solutions
> https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/git#security-concerns
<snip>

We also had some discussion about reproducing gitlab-CI builds in 
https://gitlab.torproject.org/tpo/core/tor/-/issues/40615.

While it's fairly straightforward to install a gitlab-runner and execute 
locally, as far as I can tell a malicious GitLab installation could 
still send a modified "script" (post-processed .gitlab-ci.yml) or repo 
checkout down to the runner. Maybe there's some way to audit this, but I 
couldn't find an obvious one. Maybe configuring the runner to log at 
debug level would record enough? 
https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section

For that issue I ended up hacking together a small python script that 
processes the .gitlab-ci.yml into something to feed directly through 
Docker. It's currently a bit hacky and specialized for the Debian tor 
package build. I think it could be generalized further to be reusable if 
that's of interest (maybe using Docker Compose to orchestrate jobs 
within a pipeline), but am still thinking about whether there's a better 
way... 
https://gitlab.torproject.org/jnewsome/reproduce-tor-debian-build/-/blob/main/reproduce_pipeline.py

Right now my top candidate we haven't tried yet is to install a full 
local GitLab in addition to a local gitlab-runner; maybe using their 
published Docker images https://docs.gitlab.com/ee/install/docker.html. 
This seems like the least engineering effort (~none) but a bit more work 
for every individual wanting to do such a local build.

Keeping as much logic out of the .gitlab-ci.yml as possible so that the 
gitlab yml is trivial to manually reproduce outside of gitlab (e.g. run 
`./build.sh`) is probably ideal, though gives up some gitlab 
functionality. IIUC this is the approach we're using for the tor 
tarballs. https://gitlab.torproject.org/tpo/core/tor-ci-reproducible