This tutorial will demonstrate Keptn, a CNCF Incubator Project’s integration with k6. Keptn is a cloud-native application delivery and operations platform. We will use the Job Executor Service to execute k6 performance testing in a Keptn project. We'll start with running a k6 script and how the logs look. And then we'll modify the k6 script to see the behavior when it fails.
A demo of this tutorial can be found in YouTube.
About Keptn
Keptn is an event-driven orchestration engine connecting observability with cloud-native applications operations. The project uses a declarative approach to build scalable automation for delivery and operations, evaluates Service Level Indicators (SLOs), and provides a dashboard, alerts, and auto-remediation for them. It allows us to define multi-stage delivery pipelines declaratively.
Keptn allows you to pick a use case and automate & integrate it in a more general way. Depending on the use case, you bring a configuration. For instance:
- Quality Gates <> SLI/SLO config
- Progressive Delivery <> Shipyard file
- Remediation <> Runbook for fixing system
Keptn & k6 Test Workflow
This will the workflow of events in this tutorial:
Shipyard: a Shipyard file is declared on a project Level in Keptn. All the services in the project will share the same shipyard definition. Shipyard defines the stages which each deployment would have to go through, e.g. dev, staging, production stage. It allows us to declare multi-stage delivery workflows by defining dedicated strategies in them.
Stage: A stage is declared by its name. This name will be used for the branch in the Git repository and Kubernetes namespace to which services at this stage will be deployed to.
Sequence: After defining stages, sequences can be added to a stage. A sequence is an ordered list of tasks that are triggered sequentially. K6 will use the **test(()) sequence reserved in Keptn for testing any microservice.
Job Executor Service: Job Executor Service allows us to run customizable tasks with Keptn as Kubernetes Jobs. This will allow us to integrate k6 testing easily with Keptn. You can read more about it at keptn-contrib/job-executor-service.
Setup
Install Keptn
Setup Keptn from the quickstart guide.
Install Job Executor Service
The Job Executor Service can be installed using this command:
For more information regarding the latest version of Job Executor Service, please follow this keptn-contrib/job-executor-service. In this tutorial, the Job Executor Service should listen to sh.keptn.event.test.triggered CloudEvent, which can be configured in the shipyard file.
If you have Job Executor Service installed already, please directly subscribe to sh.keptn.event.test.triggered event from Bridge after creating Keptn Project.
Local Files Setup
Clone this repository keptn-sandbox/k6-service from GitHub, and find all the files here in docs/k6-jes-example folder. Please enter into that directory to find all the files path in commands given in this tutorial.
Creating Project
Create a new Keptn project using the following command:
This command will create the project k6-jes and in the mentioned GIT_REPO, have a shipyard.yaml file in the master branch and initialize the production branch based on the stage mentioned in the file.
Creating Service
Create a microserviceA service using the command:
This command will create a microserviceA service. It will have a job config file for the k6 testing command and file path.
Adding Resources
Next, we'll add config files for microserviceA services using the commands:
This will add config.yaml and k6 test file to the production branch on GIT_REPO.
Make sure the resources have been added successfully to the git repo for the execution of the test.
Alternative Approach
Users can skip this step and upload to Git directly if they choose. Users need to upload into the {branch-name-which-matches-the-stage} and path {keptn-service-name}/{resourceUri-path}.
For example: Inside the production branch, in the folder microserviceA/job/config.yaml.
Understanding Resources
Config
The config.yaml for service microserviceA looks like:
K6 docker image is pulled from grafana/k6 and used for execution using the k6 run command. Inside the container, files are placed inside /keptn/ path. So, <resource-uri> in Git becomes accessible from /keptn/<resource-uri>.
Any custom k6 Docker image could be used here, along with k6 binary created using k6 extensions. A common example would be grafana/xk6-output-prometheus-remote. We'll take a look at k6 extensions in the next tutorial.
K6 test
A simple k6 test is used here:
This k6 test would be used for performance testing.
Trigger Sequence
Let's trigger the sequence using the command:
You can trigger the sequence from Keptn Bridge or using Keptn API too.
Success Trigger
The Sequence has been successfully triggered can be seen by the log of job-executor-service started in the below image:
K6 Execution Logs
The microserviceA service will execute with a zero exit code, therefore it will be finished successfully. We can view the logs given by the k6 run command.
Failing a k6 Test
We'll see how the service fails when the k6 performance test fails. In the k6_test.js file, we'll change the threshold from 500ms to 5ms... Forcing it to fail!
We'll have to update the resources on Git Upstream. You can do that manually in the production branch or by using this command:
Here, the k6_test_fail.js will replace the k6_test.js in the Git repo.
Trigger Sequence Again
Let's re-run the Service using the same command to trigger the sequence:
K6 Fail Logs
Due to the change in the k6 test script, now the service microserviceA will execute with a non-zero exit code, hence it will be finished as a failure.
Debug
Some of the steps which might help in debugging some common issues are:
Make sure the resources are in the correct location in the git repo. As reference, you can use jainammm/keptn-k6-jes-tutorial.
View the logs of Job Executor Service by using the command:
See also
This tutorial has shown how to run on Keptn, a standard k6 test with the core k6 functionality. But you can also enrich k6 with other testing protocols or result outputs by incorporating k6 extensions.
Check out the following tutorials that provide instructions for other cases:
- Using a k6 extension (Prometheus Extension): shows how to use a k6 extension in the k6 Job Executor Service.
- Evaluate k6 metrics with Keptn Quality Gates: shows how to send k6 metrics to Prometheus and use Keptn's Quality Gates evaluation in k6 metrics for SLO compliance.