Enable Functional Monitoring using CI CD Pipeline

This practical guide demonstrates how to leverage the Functional Monitoring BAT CLI for executing integration tests both locally and on a schedule. Additionally, it outlines integrating functional monitoring into your CI/CD workflow, with a focus on using Jenkins for automation.

Introduction

The article provides a walkthrough on using the Browser Automation Toolkit (BAT) Command Line Interface (CLI) to perform integration tests and to set up scheduled functional tests on the Anypoint Platform. Although Jenkins is the primary CI/CD tool discussed, the showcased commands are adaptable to any CI/CD system.

Scope

The guide focuses on integrating functional monitoring into a CI/CD pipeline, offering an illustrative example of the process.

Article Highlights

Folder and File Structure: An explanation of the necessary directory layout and files for setting up functional monitoring.
Command Line Instructions:
- Running a test suite manually.
- Creating and managing a scheduled test suite on a private location within the Anypoint Platform.
Jenkinsfile Usage:
- Step-by-step guidance on constructing a Jenkinsfile to manage API lifecycle steps, such as:
  - Promoting the API through various stages.
  - Deploying the API to designated environments.
  - Executing integration tests post-deployment.
  - Scheduling functional tests to ensure ongoing performance and reliability.
Source Code Repository: A GitHub repository link for accessing the sample files and scripts.

Implementation Steps for Jenkins

Set Up Project Structure: Organize your functional tests in a source-controlled repository with a clear folder structure.
Write Test Scripts: Develop your integration test scripts using the BAT framework.
Configure Jenkins: Set up a Jenkins project and configure it to track your source control repository for changes.
Create Jenkinsfile: Develop a Jenkinsfile that defines the pipeline stages, including test execution and scheduling.
Build and Test: Use Jenkins to build your project and run integration tests using the BAT CLI commands.
Deploy and Monitor: After successful integration testing, deploy your API and set up functional monitoring tests as part of the deployment pipeline.
Schedule Tests: Define cron jobs in Jenkins to trigger the functional monitoring tests at desired intervals.
Results and Reporting: Configure Jenkins to collect and display test results, sending notifications for any failures or performance issues.

Prerequisites

Before trying out the steps in this article, I would suggest you read the following articles/links to ensure you are aware of the topic

Understanding of Jenkins Pipelines – Continuous Delivery with Jenkins Pipeline
Functional Monitoring with the BAT CLI – https://docs.mulesoft.com/api-functional-monitoring/bat-top
Mule OAuth 2.0 Provider – https://docs.mulesoft.com/api-manager/2.x/mule-oauth-provider-landing-page
API Instance with the following policies,
- Mule OAuth 2.0 Access Token enforcement
- Client ID Enforcement Policy

Expected Folder Structure and Files

Since we will be focusing on using the BAT CLIU for the remainder of this article, it is important to understand the expected folder structure and the type of files typically associated with each folder. There is a great command that will help scaffold the main folders and files, which we can then update in subsequent steps. Open up your terminal window and navigate to your desired location.

1. You will first start by installing the BAT CLI (the below command can be executed on a Mac).

curl -o- 'https://s3.amazonaws.com/bat-wrapper/install.sh' | bash

2. Then you verify the installation by checking the version (if installed correctly you will see the “BAT Wrapper Version” and “BAT Version”)

bat --version
BAT Wrapper Version: 1.1.14
BAT Version: 1.1.32

3. Next we can run a few simple commands to help scaffold your initial folder structure and default files (we will updating these later on)

4. You then should see something like the following when you view the folders in Finder

The file “bat.yaml” is your main file that contains all of the related assets (e.g. which Test Suites need to be executed)
The folder “config” will contain all of your environment configurations (e.g. variables for DEV like endpoints).
The folder “tests” will contain all of your Test Suites.

5. Now we are going to update the files. Let’s start with the “bat.yaml” file. Below is what we will be using in this example.

suite:
  name: afm-demo-flights-api
files:
 -
  file: tests/afmDemoFlights.dwl
reporters:
 -
  type: HTML
  outFile: /tmp/index.html

The “reporters” section can be used to send any failed test results to several third-party applications like New Relic. Please view this link for more details

6. Next we are going to update the “config” filers, I suggest you amend them to reflect the environments in your Anypoint Platform. For demonstration I have included only one environment (dev.dwl), note that I have some values that are going to be replaced with real data later on.

%dw 2.0
---
{
  base64: 'oauth.client.credentials',
  url: 'https://afm-demo-flights-api-dev-v1.au-s1.cloudhub.io',
  client_id: 'jenkins.demo.client.id',
  client_secret: 'jenkins.demo.client.secret',
  env:'Development'
}

7. Next we need to create a new folder called “data” and create a file (flights.json) that will be used as part of the POST test case

{
    "airline": "Delta",
    "flightCode": "ER0945",
    "fromAirportCode": "PDX",
    "toAirportCode": "CLE",
    "departureDate": "June 1, 2016",
    "emptySeats": 24,
    "totalSeats": 350,
    "price": 450,
    "planeType": "Boeing 747"
}

8. Finally we can replace the test file with the one we will execute as part of this article. I called my test file “afmDemoFlights.dwl”.

import * from bat::BDD
import * from bat::Assertions
import * from bat::Mutable
 
var context = HashMap()
var oauth_base64 = config.base64
var app_client_id = config.client_id
var app_client_secret = config.client_secret
var api_endpoint = config.url
 
---
describe ("AFM-Demo-Flights-API-Suite") in [
  it must 'Create Access Token' in [
    POST `https://mule-oauth2-provider-mp-v1.au-s1.cloudhub.io/aes/external/access-token` with {
      "headers": {
          "Content-Type": "application/x-www-form-urlencoded",
          "Authorization": "Basic $(oauth_base64)"
        },
        "body": "grant_type=client_credentials&scope=READ" as Binary {encoding: "UTF-8"}
      } assert [
        $.response.status mustEqual 200,
        $.response.mime mustEqual "application/json"
      ] execute [
          context.set('access_token', $.response.body.access_token)
        ]
    ],
  it must 'Get Flights' in [
    GET `$(api_endpoint)/api/flights?code=SFO&airline=united` with {
      "headers": {
        "client_id": "$(app_client_id)",
        "client_secret": "$(app_client_secret)",
        "Authorization": "Bearer $(context.get('access_token'))"
      }
      } assert [
        $.response.status mustEqual 200,
        $.response.mime mustEqual "application/json"
      ]
  ],
  it must "Get Ping" in [
    GET `$(api_endpoint)/api/ping` with {
      "headers": {
        "client_id": "$(app_client_id)",
        "client_secret": "$(app_client_secret)",
        "Authorization": "Bearer $(context.get('access_token'))"
      }
    } assert [
        $.response.status mustEqual 200,
        $.response.mime mustEqual "application/json"
      ]
  ],
  it must "Post Flights" in [
    POST `$(api_endpoint)/api/flights` with {
      "headers": {
        "client_id": "$(app_client_id)",
        "client_secret": "$(app_client_secret)",
        "Authorization": "Bearer $(context.get('access_token'))"
      },
      body: readUrl('classpath://data/flights.json', 'application/json')
    } assert [
        $.response.status mustEqual 201,
        $.response.mime mustEqual "application/json"
      ]
  ]
]

Note that you use the “context.set” function in the response of the first API test to capture the access_token, we then reuse it with the remaining API test by using the “context.get” function.
The BAT Playground is an excellent resource to get more familiar with the BDD Test writing syntax

Sample Commands to Execute your Test Suite

Let’s first start by executing your Test Suite to validate that your configuration and BDD Test have been set up correctly.

Run a Test Suite

If you recall, we have some dummy values in the environment configuration files, so we will use the “sed” command to replace the variables before running the BAT tests

Run the following “sed” commands (please replace them with values relevant to your implementation)

sed -i -e "s/base64:.*\$/base64: '',/g" bat-tests/config/test.dwl
sed -i -e "s/client_id:.*\$/client_id: '',/g" bat-tests/config/test.dwl
sed -i -e "s/client_secret:.*\$/client_secret: '',/g" bat-tests/config/test.dwl

2. This step is optional but recommended when executing from your CI/CD pipeline. It is to ensure that you are always running the latest version of BAT (basically it will update your BAT install if you have an older version)

bat --version

3. Now you can run your BAT tests, it is a good idea to use the “–config” option to indicate which environment you want to test against

bat --config=test bat-tests

4. If you have completed all of the previous setups correctly, you will see your test results. If you recall you did define the “HTML” reporter as part of the “bat.yaml” file. Towards the bottom of your test results, you will see the path to your HTML page. This is something that can be published as part of your CI/CD pipeline.

#  Reporter: bat/Reporters/HTML.dwl >> /tmp/index.html
#  Reporter: bat/Reporters/JSON.dwl >> /var/folders/2t/fn23_m092jbfd1p7kmgw8d94b9tm8d/T/bat_report_20200922220344.json
#  Reporter: bat/Reporters/HTML.dwl >> /var/folders/2t/fn23_m092jbfd1p7kmgw8d94b9tm8d/T/bat_report_20200922220344.html

Create a Scheduled Test Suite running on a Private Location

When you schedule a Test Suite you have the option of selecting either a Public or Private location. I have chosen a Private location to ensure that my tests are executing within the Sydney region.

You need to have an entitlement to Anypoint VPC to create a Private location. A private location is a CloudHub worker that runs on your selected environment.

1. You need to login to be able to access your Anypoint Platform to carry out the following steps

bat login --username="replace with Username" --password="replace with Password"

2. Next you need to switch to the environment where you want to create your Private location

bat environment switch "replace with environment name"

The expected response will look something like below

Environment switched to Development.

3. Execute this command to create your location, take note of the “id” as you will need it when you create the schedule

bat location create

The expected response will look something like below

Creating location, please wait...
 
Location Created:
  Name: dummy-dev-location
  Location Id: 16c0f80b-b039-444d-951c-a8975ef3cd1e
  Organization: a50e8dcf-b730-43ea-8017-c9557806180e
  Environment Id: 0fefc835-3ba7-440c-95cf-0923182a36c2

4. This step is optional but recommended when executing from your CI/CD pipeline. It is to ensure that you are always running the latest version of BAT (basically it will update your BAT install if you have an older version)

bat --version

5. Next we may need to run the “sed” commands again to (please replace with values relevant to your implementation)


sed -i -e "s/base64:.*\$/base64: '"replace with base64 encoded Username:Password"',/g" bat-tests/config/prod.dwl
sed -i -e "s/client_id:.*\$/client_id: '"replace with Client ID"',/g" bat-tests/config/prod.dwl
sed -i -e "s/client_secret:.*\$/client_secret: '"replace with Client Secret"',/g" bat-tests/config/prod.dwl

Introduction

Scope

Article Highlights

Implementation Steps for Jenkins

Prerequisites

Expected Folder Structure and Files

Sample Commands to Execute your Test Suite

Run a Test Suite

Create a Scheduled Test Suite running on a Private Location

Recent Posts

Recent Comments

Subscribe To Our Newsletter

You have Successfully Subscribed!