I prefer to use git over ssh with key-based authentication, but this causes trouble with putting gitlab in docker. The problem is a simple one - I have ssh on the host, and in the container, but I want to treat them equally. git@example.com should go to gitlab, me@example.com should go to the system ssh, and I do not want to explain the difference to my users.

There's two easy answers to this - move the host's ssh onto a different port, or expose the container's ssh on different port. If it's just me, this is easily accomodated by using my ssh config to send two names to two ports. But because I don't want to confuse everyone with those instructions, I took the third method, which is so messy, I need to write it down incase I ever have to replicate it.

How ssh into gitlab works

Since this is the mechanism I exploit, it's worth taking two minutes to understand the execution path here. In normal usage (non-docker) usage, the system's ssh daemon accepts connections for the git user. When gitlab populates the list of ssh keys for the git user, it adds a stanza to force the connection to use a specific command. So in ~git/.ssh/authorized_keys, we get something like:

command="/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell key-1",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty key-type AAAAtheactualkeyhere

So when a connection is accepted using this key, it's forced into the gitlab-shell process, which presents git the interface it's expecting. On the face of it then, the task is simple - I want gitlab-shell inside the container, to accept connections to the system ssh.

Step 0

This is accidentally easy. I want the ownership of the 'git' account inside & outside to match. Luckily gitlab's docker images are using an Ubuntu base, and so is my host - so git is uid 998 inside & out. If this assumption is ever broken, it'll need to be reconstructed before we continue.

Step 1

I want gitlab to be able to populate the authorized_keys for my system git, and the easiest way I've managed this is to mount the outside file as a volume inside. So in docker-compose,

volumes:
  - /home/git/.ssh/authorized_keys:/var/opt/gitlab/.ssh/authorized_keys

Docker likes to assume volumes are folders, so restarting now is problematic. If you have an existing gitlab install, copy the existing file out:

docker cp gitlab_gitlab_1:/var/opt/gitlab/.ssh/authorized_keys /a/local/file

Or touch the local path to ensure it exists. Either way, don't forget to fix permissions, the file should be 0600 and owned by git. ssh does not forgive permissions, if this is incorrect it'll act as though this file does not exist.

Step 2

Now ssh will accept connections for git@example.com, it'll match keys gitlab provides, and then it'll attempt to execute command=...gitlab-shell. This creates two needs. We need ...gitlab-shell to exist, and we need it to act as it should. One method I was tempted to explore is using docker exec to run gitlab-shell inside the existing container - but giving the git user permissions to the docker socket scares me, so I'm going to continue hijacking ssh.

So create a script at /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell as follows:

#!/bin/bash
ssh -i /home/git/.ssh/id_rsa -p 2222 -o StrictHostKeyChecking=no git@127.0.0.1 "SSH_ORIGINAL_COMMAND=\"$SSH_ORIGINAL_COMMAND\" $0 $@"

This does pretty much exactly what it says it does. Whatever we try to do in the outside gitlab-shell gets sent to gitlab's ssh. But it doesn't work just yet, there's a few things in there we haven't setup yet. the port (-p 2222) and the key (-i path).

Step 3

Port first, since it's super easy. I don't want to expose this, so I define it as a port instead of an expose. So back to docker-compose:

ports:
  - "127.0.0.1:2222:22"

Easy. :2222 on localhost is :22 inside.

Step 4

The last hurdle, I'm still not sure I have the best answer to. git@host needs to authenticate to git@container, and I can't find a good way to re-use the same key the user just came in with. So what I'm doing is create a keypair got git@host as normal (ssh-keygen), and then mount the pubkey in the container as:

volumes:
  - /home/git/.ssh/id_rsa.pub:/gitlab-data/ssh/authorized_keys

I don't like this as I have no idea why it works. I have no idea what users this key is acceptable for. I'm not entirely convinced that someone with more clue than me can't re-use this key to access any user in the container. I can't find this path referenced from /etc/ssh/*, but .. it works.

Done?

This should be everything to put it together. The system ssh is accepting connections made with the keys gitlab is populating. ssh invokes gitlab-shell, which outsources the to ssh inside the container. Nothing has been modified inside the container, so we can update freely without breaking anything too much. Unless they change the uid for git. I live in fear.