Using Azure Kubernetes Service (AKS) for your VSTS build agents

Sometimes hosted build agents in VSTS dont cut the mustard and you want full control over your build environment. That’s where self hosted build agents come in. The problem is … do you don’t want to run VMs ideally and if you are getting into Kubernetes then your dev cluster is probably sitting there idle 90%+ of the time with all those CPU cycles being wasted.

We decided to do something with that extra capacity and run a set of VSTS linux build agents (good for Nodejs and .net Core builds etc…) in our dev AKS cluster! We can scale them up for more concurrent builds really easily.

What you will need:

Lets go …

Helm is a tool that helps you install apps in your kubernetes environment. Helm charts are templates for your application. They define what your app needs and what containers should be deployed etc… Fortunately Microsoft make their linux build agent available as a Docker image that we can use in a helm chart to get it deployed. https://hub.docker.com/r/microsoft/vsts-agent/

This means all we need to do is deploy it (or many of them) to Kubernetes … and helm charts can help with that! We wrote a basic one to get things going.

Setup

First you will need to get our helm chart.

git clone [email protected]:Hyperfish/vsts-build.git

Next open up the values.yaml file and update the following properties:

  • VSTS_ACCOUNT – this is the name of your VSTS account e.g. “contoso”.
  • VSTS_POOL – this is the name of the agent pool you would like your agents registered in.
  • VSTS_TOKEN – this is your personal access token from VSTS that has been given at least the Agent Pools (read, manage) scope.
  • replicaCount – set this to how many agents you want deployed.

Note: for more information about these see the vsts agent docker image documentation.

Deploy

Once you have updated the values.yaml file you are ready to deploy!

Ensure you are in the /vsts-agent folder and have kubectl connected to the kubernetes cluster you want to deploy the application to. (tip: run “kubectl cluster-info” to check you are connected)

Deploy the chart:

helm install .

Once complete the agent will be started in your kubernetes cluster.

helm ls

This will show you the apps you ahve deployed and you should see the vets-agent chart deployed.

Check your VSTS build pool that you specified in the values.yaml file. You should see your new agents listed.

Troubleshooting:
If you don’t see them listed then its likely that the values you set are incorrect. You can check the logs of your agents using:

kubectl logs <pod name>

You might see something like “error: missing VSTS_ACCOUNT environment variable”

Summary

Kubernetes is a great way to deploy multiple vsts build agents! Deploying with a Helm chart is even nicer! It gives you are simple way to deploy and manage your agents in kubernetes.

Enjoy!

-CJ

2 thoughts on “Using Azure Kubernetes Service (AKS) for your VSTS build agents

  1. Mike

    Hey, nice article!
    Tried to follow it and stuck on 1 problem:
    when I run “kubectl logs” I get

    Error from server (BadRequest): container “vsts-build-agent” in pod “knobby-tuatara-vsts-build-agent-74ff468dc-67vng” is waiting to start: ContainerCreating

    And inside the kubernetes dashboard for this pod I see this error:

    network is not ready: [runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: Kubenet does not have netConfig. This is most likely due to lack of PodCIDR]

    Any hints on how to fix it?
    Thanks!

    Reply

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.