Unit Tests is an important part of any codebase. It helps capture bugs quickly even before going into the QA cycle, reducing the time in the bug life cycle. Since “Go To Market” is an important factor in running a business, we tend to miss writing Unit Tests sometimes and push the code to production, leaving chances for the system to fail in a live environment.
In this blog post, we will cover how to capture Unit Test coverage continuously and propagate the results to a Dashboard that gives overall visibility of the Unit Test coverage.
Before deep-diving into the topic, let us understand a couple of basic terminologies:
Unit Test
A Unit Test is basically testing the smallest piece of code which is isolated logically in a system. It can be a simple line of code, a function, or a class. The main objective of unit testing is to isolate written code to test and determine if it works as intended.
Code Coverage vs Test Coverage
| Code Coverage | Test Coverage | |
| Definition | White-box testing technique is performed to verify the extent to which the code has been executed. | Test Coverage is a black-box testing technique performed to verify the product’s functionality. |
| Purpose | Code coverage scripts generate a report that details how much of the application code has been executed | It monitors the number of tests that have been executed. Test cases are written to ensure maximum coverage of requirements outlined in multiple documents. |
| Execution Done By | Developers | SDETs maintain Test Cases in documents which makes it easy to track |
But to track Code Coverage, there are fewer tools. SonarQube comes into the picture here which solves our problem of tracking Code Coverage Metrics. It provides us with an interface and also APIs which can be used to track the Code Coverage Metrics.
Now, as the number of microservices increases, there has to be some holistic view where we can see all the Unit Test Code Coverage metrics in one place. At Cashfree Payments, we have found an innovative way to achieve the same using sonar APIs and Google Data Studio.
Related Post: UPI Payments For Businesses
Let us break this entire setup into four simple steps for better and easy understanding.
These four steps are:
- Setting up SonarQube
- Using SonarQube APIs to fetch project coverage metrics
- Capturing project metrics in a Google Sheet using the above SonarQube APIs
- Building a Dashboard using the source as Google Sheet
Step 1: Setting up SonarQube
Here, we will be considering the service to be a Java Maven application but the process remains similar for any application. Below are the steps:
- Installing SonarQube and generating a token:
The latest version of SonarQube can easily be installed from the open-source download page and set up the server and project using the documentation they provide. Once the server is up and a project was created, we generated the token key and stored it, as we will be using it to capture our future results.
Here is how the page looked once we generated a token in the Sonar UI:
Analyzing a Java project using maven:
We can simply run the below maven command from the project folder and all the details about the project like code coverage, security issues, vulnerability, etc will be captured.
| mvn clean verify sonar:sonar -Dsonar.host.url=$SONAR_URL -Dsonar.login=$LOGIN_TOKEN |
The analysis summary looks like this once the above command is executed:
- Making Sonar analysis part of Jenkins Pipeline:
We can automate the above process to capture the analysis just before deployment in a JenkinsFile. Create 2 stages: one for running the tests and the second for capturing the results in sonar. This will ensure all the tests are run and the unit test coverage details are analyzed and captured in Sonar in an automated way.
stage('Run test') {
container('maven') {
configFileProvider(
[configFile(fileId: 'global-maven', variable: 'MAVEN_SETTINGS')]) {
sh 'mvn -s $MAVEN_SETTINGS test'
}
}
}
stage('Sonar') {
container('maven') {
withCredentials([string(credentialsId: 'sonar', variable: 'TOKEN')]) {
sh """
mvn clean verify sonar:sonar -Dsonar.host.url=$SONAR_URL -Dsonar.login=$TOKEN
"""
}
}
}Now that we have set up SonarQube and automated the flow to capture unit test coverage, it is still a tedious task to view the code coverage for 100s of projects/services in the Sonar UI.
To solve this problem of viewing the unit test coverage details in one place, we have come up with an innovative workflow at Cashfree Payments using AWS Cron, Java Spring Boot Service, Sonar APIs, GoogleSheets Library, and DataStudio.
Step 2: Using SonarQube APIs to fetch Project Coverage Metrics
SonarQube provides us with a few APIs where we can get all projects and the metrics of the projects as a JSON Response. Using these APIs, we can get all projects that we have analyzed, get metrics of each project, and store/update them in GoogleSheets in a regular manner where all the metrics can be seen in one place. All the metric-related details and definitions are provided here.
Below are the 2 Sonar APIs used:
- Get Projects:
curl --location --request GET 'http://sonar.orgname.com/api/components/search?qualifiers=TRK&ps=1' \
--header 'Authorization: <Sonar Auth>'The above cURL gives the below response where we get a list of all projects based on page size:
{
"paging": {
"pageIndex": 1,
"pageSize": 1,
"total": 110
},
"components": [
{
"organization": "default-organization",
"key": "com.orgname:svcname",
"name": "svcname",
"qualifier": "TRK",
"project": "com.orgname:svcname"
}
]
}We can make use of the Page Size (ps) param to get as many projects as we want. Now using the project key we can get the metrics with the below cURL.
- Get Project Metrics:
curl --location --request GET 'http://sonar.orgname.com/api/measures/component?component=<project>&metricKeys=coverage,branch_coverage,line_coverage' \
--header 'Authorization: <Sonar Auth>'The above cURL gives us the below metrics:
{
"component": {
"id": "<projectId>",
"key": "com.orgname.svcname",
"name": "svcname",
"description": "Service Description",
"qualifier": "TRK",
"measures": [
{
"metric": "line_coverage",
"value": "65.3",
"bestValue": false
},
{
"metric": "coverage",
"value": "62.1",
"bestValue": false
},
{
"metric": "branch_coverage",
"value": "46.4",
"bestValue": false
}
]
}
}Related Post: SEPA Transfers
Step 3: Capturing project metrics in a Google Sheet using the above SonarQube APIs
Let’s see how this is achieved from the below workflow:
- Individual Services: Each service will capture the coverage results and push them to sonar as explained before.
- Kubernetes Cron: The cron hits the SpringBoot Service endpoint twice a day which does the job of getting project metrics using the above Sonar APIs and storing them in a Google Sheet.
- SpringBoot Service: I have exposed a GET endpoint/capture-unit-tests which does the following:
- Get all Projects and store them in a list using the Get Projects Sonar API.
- Get each project from the above list and get metrics for the project using the Get Project Metrics Sonar API. Aggregate all the metrics in another list.
- Check if there is existing data present in the target Google Sheets. To make sure no duplicate data gets added, delete all existing data from the sheet (if present) and update the sheet with the latest aggregated metrics list. We have made use of the google-API-client library to achieve the same.
- Now the above-aggregated details can be used to store the metrics in a Google Sheet in the below format:
Step 4: Building a Dashboard using the source as Google Sheet
Now after step 3, we have all the project metrics present in the Google Sheet. We can integrate this into a Dashboard using Google Data Studio.
Let’s see how to build a Dashboard on top of the above Google Sheet using Data Studio:
- Go to https://datastudio.google.com/
- Create a new Data Source and choose Google Sheets.
- Select the above sheet where we have captured the metrics and connect.
- Verify the columns that are required and click on Create Report.
- Here we can be as innovative as possible to create the Dashboard. Below is a sample Report created from the data captured in the above Google Sheet.
- We are all set to view the UT Coverage metrics in the above Data Studio Report. Now, whenever the cron runs periodically and updates the Google Sheet, Data Studio will automatically refresh and we can see the latest coverage data up to date at any time.
Did you find this interesting?
With the above workflow, we can keep a track of the unit test coverage and analyze which services to improve. Add UT coverage tasks to a given sprint based on the above data and improve the code coverage so that we can write good quality code and deploy new features faster and safer.
Apart from the above-used metric keys (coverage, branch_coverage, line_coverage), there are many more parameters also to play around with, like critical_violations, file_complexity, code_smells, etc that can be self-explored. A list of all Sonar APIs can be found here.
Now, what are you waiting for? Go ahead and try this out!
Does this sound like something that tickles all the right brain cells? Then we have some exciting opportunities lined up for engineers just like you!
