Automation lies at the heart of DevOps. Various automation tools and techniques have truly enabled the concept of continuous integration and continuous delivery. These tools have evolved over the years rapidly but one name that seems to be here forever is Jenkins.
We will neither discuss introductory concepts of CI-CD in this post nor waste time showing Jenkins installation steps. If you are new to Jenkins, you can check out the Offical Installation Docs to get started with Jenkins. Therefore, the purpose of this post is to discuss how to setup Jenkins controller-agent architecture (aka master-slave architecture) and address some of the issues that arise when doing it. This is because, the process can be tedious and if you haven't done it in a while, you may end up wasting a few hours.
Why Jenkins Controller-Agent Architecture?
The controller(master) node is Jenkin's brain, it's where the Jenkins application runs. If we do too much work on the controller node (or it crashes), the entire application may become unavailable. Therefore, we want the master to be as available as possible. This can be done by delegating work to agent nodes (slave nodes). So in a Jenkins Controller-Agent architecture, the jobs are scheduled and assigned to agents by the controller. The controller also keeps track of whether the slaves are online, retrieve their responses of the build results, and outputs the build results on the console. Thus, the master node is more available and hence the overall performance of our Jenkins server is improved using this design.
Another advantage of this architecture is that we can only install a minimum set of tools on the controller node and we can install the heavier tools (required by the jobs) on the agent nodes. This keeps the controller lightweight and also allows us to organize our jobs based on the agents that should execute them. In the above example, we have a Jenkins controller along with 4 agents. Each of the agents can serve a specific purpose.
- For example, if we need to run tests and build jobs of javascript-based applications we can restrict these jobs to be executed on the agent on the far left.
- Similarly, if we need to build some .NET applications we can setup a Jenkins agent with a windows host and restrict these jobs to be executed on the far right.
- Furthermore, we can improve performance by balancing the loads based on our system's requirements. Let's say, we are setting up CI-CD of a system that has hundreds of microservices among which, there are twice as many services written using python based stack as any other stack. In this situation, we can setup two Jenkins agents with python based tools installed and the Jenkins controller can balance the load between these two agents.
Setup
Step 1: Spin up a Jenkins Controller(master) Container
We can use the official jenkins docker container. Here's an example docker-compose file that you can use.
version: '3.8'services: jenkins_controller: image: jenkins/jenkins:lts-jdk11 privileged: true user: root container_name: $CONTAINER_NAME ports: - 50001:8080 - 50002:50000 volumes: - $JENKINS_HOME:/var/jenkins_home - /var/run/docker.sock:/var/run/docker.sock
Now, we need to spin up the container and we need to install the required tools for the Jenkins controller node. We can write a simple bash script to achieve that.
#!/bin/bashset -o nounsetJENKINS_CONTAINER_NAME=$1JENKINS_HOME=`pwd`/jenkins/jenkins_home echo "JENKINS HOME: $JENKINS_HOME"CONTAINER_NAME=$JENKINS_CONTAINER_NAME JENKINS_HOME=$JENKINS_HOME docker-compose -f docker-compose-controller.yaml up --build -dsleep 10# Here we have just installed git for our controller node, install as many tools as you requiredocker exec $JENKINS_CONTAINER_NAME bash -c "apt-get update -y -q && apt-get upgrade -y -q && apt-get install -y -q git"
- Jenkins uses apache jetty which uses port 8080 by default, We mapped our host machine's port 50001 with the container's 8080. So, entering http://host:50001 should take you to the Jenkins web dashboard.
- Check container logs for the first time admin password and create a new admin user.
Step 2: Set up a Jenkins Agent (slave)
We can now setup our agent(s). Since, our Jenkins controller will communicate with the agents using SSH, we need to generate the SSH keys. In this case, the Jenkins master node will act as the SSH client and the agent(s) will act as SSH servers. So, we need to set it up accordingly.
- Generate Keys
ssh-keygen -t rsa -f jenkins_agent_1
- Goto Jenkins Dashboard > Manage Jenkins > Manage Credentials > Add 'System' scoped Credential for enabling SSH into a Jenkins Agent
System Credential vs Global Credential
System: Only available on Jenkins server (not visible by jenkins job)
Global: Accessible everywhere including jenkins job
Fill up the form with the appropriate values. Here's an example,
Username: jenkins # we want to ssh into the agent as 'jenkins' user which already exists by default in the jenkins-agent container we will be using
ID: An Unique ID for the credential that can be used to refer to the credential
Private Key: SSH Private Key file contents (e.g: jenkins_agent_1)
Step 3: Spin up a Jenkins Agent(slave) Container
We can use the official jenkins-ssh-agent docker container. Here's an example docker-compose file that you can use.
version: '3.8'services: jenkins_agent: image: jenkins/ssh-agent:jdk11 privileged: true user: root container_name: $CONTAINER_NAME expose: - 22 environment: - JENKINS_AGENT_SSH_PUBKEY=$JENKINS_AGENT_SSH_PUBKEY
Notice that, we must set the environment variable JENKINS_AGENT_SSH_PUBKEY which in this case we are doing from a bash variable. We also need to install the required tools in our Jenkins agent. We can achieve all that using a simple bash script like the one below,
#!/bin/bashset -o nounsetJENKINS_CONTAINER_NAME=$1JENKINS_AGENT_SSH_PUBKEY=$2CONTAINER_NAME=$JENKINS_CONTAINER_NAME JENKINS_AGENT_SSH_PUBKEY=$JENKINS_AGENT_SSH_PUBKEY docker-compose -f docker-compose-agent.yaml up --build -dsleep 10# Here we have just installed tools that help us create a python virtual environment for our agent node, install as many tools as you requiredocker exec $JENKINS_CONTAINER_NAME bash -c "apt-get update -y -q && apt-get upgrade -y -q && apt-get install -y -q git python3 python3-venv"
Step 4: Configure an Agent from the Jenkins Controller
Goto Jenkins Dashboard > Manage Jenkins > Manage Nodes and Clouds > New Node
Fill up the form using the appropriate values. Here's an example,
Name: JenkinsAgent1
NumberOfExecutors: 1-2
RemoteRootDirectory: /home/jenkins/agent
Labels: linux, python # Space separated values, Can be useful to restrict jobs to run on a particular agent
Usage: Use this node as much as possible
Launch Method: Launch agents via SSH
Host: jenkins_agent # Agent's Hostname or IP to connect. (docker-compose service name if controller and agent is on the same machine)
Credentials: Select the Credential created in Step 2
HostKeyVerificationStrategy: Non verifying Verification Strategy
Launch Method > Advanced
ConnectionTimeoutInSeconds: 60
MaximumNumberOfRetries: 10
SecondsToWaitBetweenRetries: 15
There are some other options you might wanna look into, but the ones i discussed should help you get started.
We can create as many agents as we require using the process discussed above.
Step 5: Create jobs and run
Now, our agent should be discovered by the controller and we can start delegating our jobs to the agents. We can restrict a job to run on a particular agent by using the labels we assigned when creating an agent.
That's done :D. Let me know, if missed anything. You can also find the source files here: ashiqursuperfly/Jenkins-Controller-Agent-Setup
If you found this post helpful, CLICK BELOW 👇 
