CI/CD Pipelines And Caching Of Dependencies On Azure DevOps — Apisero

Authors: Rahul Kumar & Kushagra Kakkar

Introduction:

In this blog we’ll be taking you through a brief explanation of CI/CD Pipelines in Azure DevOps and how to implement caching of maven dependencies in the pipelines while deploying your Mule application to Cloudhub.

What is CI/CD?

CI, short for Continuous Integration, is a software development practice in which all developers merge code changes in a central repository multiple times a day. CD stands for Continuous Delivery, which on top of Continuous Integration adds the practice of automating the entire software release process.

What is Azure Devops?

Azure DevOps provides developer services for allowing teams to plan work, collaborate on code development, and build and deploy applications. It provides following integrated features that you can access through your web browser or IDE client:

• Azure Repos
• Azure Boards
• Azure Pipelines
• Azure Artifacts
• Azure Test Plans

We’ll be using Azure Pipelines and Azure Repos for this demonstration. Azure Repos is a git based repository and Azure Pipelines as the name suggests, we’ll use to implement our CI/CD pipeline.

Prerequisites for Azure CI/CD pipeline:

• Anypoint Studio to develop your application.
• Azure DevOps account with access to hosted parallelism.
• An active Anypoint Platform account.
• Maven and Git installed on your local system.

Tutorial:

1. The first step after you create you Azure devops account is to raise a request for hosted parallelism, else whenever you execute your pipeline you will get the below error:

a. To raise a request go to https://aka.ms/azpipelines-parallelism-request and fill the form with the correct organization name. It will take around 2 business days to get this request approved.

2. Now create a project in Anypoint studio which you wish to deploy on your cloudhub. For this tutorial we’ll take a simple example which consists of an HTTP Listener and a Logger component.

3. Now configure your project’s pom.xml to enable the deployment of your project on cloudhub via maven. This uses mule-maven-plugin which is already a part of your pom.xml generally, in case it is missing just add the dependency and use the following configuration:

<plugin>
<groupId>org.mule.tools.maven</groupId>
<artifactId>mule-maven-plugin</artifactId>
<version>3.5.4</version>
<extensions>true</extensions>
<configuration>
<cloudHubDeployment>
<uri> https://anypoint.mulesoft.com</uri>
<muleVersion>${app.runtime}</muleVersion>
<username>${username}</username>
<password>${password}</password>
<applicationName>${cloudhub.application.name}</applicationName>
<environment>${environment}</environment>
<region>${region}</region>
<workers>${workers}</workers>
<workerType>${workerType}</workerType>
<properties>
<key>value</key>
</properties>
</cloudHubDeployment>
</configuration>
</plugin>

These variables can be defined at runtime, or can be directly given in the pom.xml, For further reading on these properties or any additional flags, you can visit the official documentation on https://docs.mulesoft.com/mule-runtime/4.4/deploy-to-cloudhub.

Remember to keep your application name unique, otherwise you’ll get an error while pipeline execution.

4. Be sure to check your Maven settings.xml file has all the correct credentials and the required repositories are added there, otherwise your pipeline would fail. This settings.xml file is present at your Maven home in your local system.

5. Now let’s create a repository on Azure Repos.

a. Go to Repos in your Project -> Click on initialize.

b. Now clone your repository into your local system by getting the clone URL from your Azure Repo.

c. Now using this URL execute the git clone command on your local system to clone your repository.

d. It will ask for your devops credentials while cloning. Alternatively, you may use git credentials for HTTPS/SSH according to your requirement.

e. Now commit and push your code into the Azure repository it would look like this after successful code push:

6. We’ll now proceed to create the Pipeline:

a. Click on Create pipeline and then go to classic editor:

b. As we are using Azure Repos, we’ll select the Azure Repository here. You may proceed with any other option if you have already set up your repository somewhere else:

c. Now click on Empty Job. If you have exposure to write pipelines using YAML to create and configure your pipelines you may select the YAML option but if you are looking for ease of use, just go with an empty job.

d. If you have any specific agent requirements i.e. you want your pipeline to run on ubuntu/windows you may select that in Agent Specification. The default is windows-2019 which we are using for our demo.

e. Now click on + Button to start adding agent jobs:

f. We need to add 3 jobs, Cache, Maven and Download Secure File. The order should be Cache then Maven and in the end the DownloadSecure file, otherwise the Cache won’t work.

g. For Cache, provide the below properties:

We are providing pom.xml as our key, so that the cache would check for any changes in pom.xml and would download the dependencies in case there is a cache miss otherwise it would use the dependencies present on the Path specified. For us we have Provided $(MAVEN_CACHE) as our variable.

For the first time execution of your pipeline, the cache will always be a miss.

h. Now to provide values to the variable go to the Variables section and define the value. For $(MAVEN_CACHE) the value should be $(Pipeline.Workspace)/.m2/repository. This is the location where the maven dependencies are stored on Azure. $(Pipeline.Workspace) is a predefined variable.

Also, remember to provide all the variables which you want to provide here itself which will be used in your maven command.

We have an option of creating variable groups in case you want to create some secure properties, such as your username and password. You can provide them in Library->Variable Groups

i. After this group is created you have to Link it in the Pipelines-> Variables section.

j. Now go to Secure files in Library and add your Maven settings.xml from your local system here.

k. Go to your pipeline-> Download Secure File in your agent job and provide the settings.xml from the dropdown under Secure file. In the Output Variable section, provide a variable name and copy the full variable from the Variable list which will be used to access the secure file via maven command.

l. Now go back to the pipeline and go to your Maven Task and provide the maven details such as the location of the pom file, your goals and the command.

7. You can enable Trigger as well so that you can leverage the Continuous Integration ability of your Pipeline. Whenever you push your code the pipeline will automatically Execute. Go to Pipeline-> Triggers and check Enable continuous integration.

8. Now click on Save & queue and your pipeline will execute. In case there is any error that will be displayed in the pipeline itself.

9. Now you may check the Runtime Manager on cloudhub, and if the job was successful and all the parameters were correct you should see the application there!

Happy Learning

Do reach out to us if you have any questions/suggestions.

Rahul- rahul.p.kumar@apisero.com
Kushagra- kushagra.n.kakkar@apisero.com

Originally published at https://apisero.com on June 10, 2022.