Jenkins and docker agents on Windows

I quite like the Gitlab way of working with runners, the runner is a small docker container that is managing jobs in other containers, easy to install and maintain. It would be nice if Jenkins could do it too. Well, it actually can, but if need to run jobs in Windows containers, the journey to working setup took me a while.

The quick solution is to install the agent directly on the host, but maybe you are running your CI in docker swarm, maybe you want to use docker for easier upgrades, so here's how to do it in a more complicated manner and run Jenkins agent inside a windows docker container.

Jenkins server

If you don't have a Jenkins server running, you can spawn you own quickly:

 docker run -it -d -u root -p 8080:8080 -p 50000:50000 -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker -v /var/jenkins_home:/var/jenkins_home --name jenkins jenkins/jenkins:lts

I mounted the docker.sock and docker binary (it's not present in the image) to the container so Jenkins can access the docker running on the host and spawn new containers for scheduled jobs. You could avoid mounting the docker binary by creating a custom container with docker installed, but this was faster :).

To log into the server, run docker logs jenkins and note the setup password, now you can open your browser at localhost:8080 and configure your Jenkins instance. You can create some docker test job to test everything works.

Jenkins agent on Linux

Let's start with something simple, running Jenkins agent as docker executor inside docker on Linux is easy:

docker run -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker --name jenkins_agent --init jenkins/inbound-agent -url http://jenkins-server:port -workDir=/home/jenkins/agent <secret> <agent name>

As with the Jenkins server on linux, I'm mounting some docker stuff to allow container to use docker directly. If you don't know your agent secret and name, you probably need to create it in Jenkins (manage agents and clouds -> new agent -> Permanent node).

Windows Jenkins agent

Running jenkins agent inside docker on Windows is pain. It's quite easy to start a simple agent in docker with no support for running jobs in docker (oh man, this sentence is strange):

docker run --isolation=hyperv  --name jenkins-agent jenkins/inbound-agent:windowsservercore-1809 `
   -Url http://jenkins-server:port `
   -WorkDir=C:/Users/jenkins/Agent `
   -Secret <SECRET> `
   -Name <AGENTNAME>

The --isolation=hyperv is needed if you run in same windows version as the agent image.

If you want to run the docker jobs, you'll need to build a custom docker image with docker tools preinstalled. Take this Dockerfile and replace the FROM image with jenkins/inbound-agent:windowsservercore-1809. Build the image and start it as:

docker run --isolation=hyperv -v "//./pipe/docker_engine://./pipe/docker_engine" --name jenkins-agent mycustomjenkinsagentcontainer
   -Url http://jenkins-server:port 
   -WorkDir=C:/Users/jenkins/Agent
   -Secret <SECRET> 
   -Name <AGENTNAME>

Jenkins docker cloud

There's a different way to do it, maybe a bit cleaner, you can use the Jenkins docker cloud plugin. By using this plugin, you can connect Jenkins directly to the docker hosts. The route is not that straightforward, you'll need to generate TLS certs on each docker host, push them to Jenkins server, enable some ports on Windows Firewall, etc.

I made it working back then, but the cloud doesn't support the isolation=hyperv I needed (toolchain running under different windows version than the server).