I delighted to bring to your attention the course I have created powered by Udemy for creating a test automation framework for gRPC Testing. The course comes with a fully functioning gRPC service and Proto files ready for you to author kick ass tests against and master your craft. If you have any of the following questions then this is the course for you:
- How do you automate a gRPC test?
- How do you performance test gRPC?
- Can I store my gRPC tests in source control
- How to do gRPC testing?
- Is there a framework for testing gRPC?
This course covers the following areas in practical labs.
Load gRPC Testing
Load testing is the ability to provide your services with the normal expected amount of load over a given period. This give you the confidence that your system will still meet your existing user or customer expectation and no changes have degraded or regressed the system in a negative way.
Spike gRPC Testing
Spiking your system is very important as it allows you to prepare for both exceptional traffic demand (Remember the supermarkets through lockdown) and something a little more nefarious such as a denial-of-service attack. We speak about he process that you would undertake to simulate a spike, such as bringing your tests up to a normal daily load, then suddenly you want to ramp up your virtual users to simulate an extreme load.
This kind of test will let you know what will happen and will be one of the scenarios below:
The service will stop and not recover | In this scenario your service may need human interaction to bring it back to life, and this may be ok depending on your service level agreements. |
The service wont process all requests | This can be almost as bad if not worse than above. For example say you have a booking form with five service calls in it, allowing the user to complete to the third one and then it fails, causing severe customer disappointment. |
The service will limp along at a slower rate | In this scenario your service will limp along at a slower rate, but will process all of the calls. This may be ok and could be managed by a queue, but it’s not an ideal scenario. |
The service will perform without problem | This is what everyone wants, however without auto scaling and other expensive hardware, this may be out of reach. |
Once you have the above knowledge, you can:
- Present this information to the customer or department to see if it is acceptable.
- Create a queueing system to cope with the traffic levels. This can in itself add to the problem because if the queue is made up of real and fake users, then fake users may take priority if they are ahead in the queue. At the very least your customers will know your site is busy.
- You can give the information to developers to ensure that they have done everything they can such as caching and minimising unnecessary service calls. Maximising performance from a configurable point of view.
- If you have monitoring in place with alerts, you can set this just before the spike is due to meet the threshold that you have from your test. Then you can take evasive action to avoid disaster such as putting up message to your customers.
- Allocate more resources to to ensure that the system is more resilient, including vertical and horizontal scaling. Caution with this option must be taken as you may end up paying for redundant resources and code/configuration redundancies should always be sought first.
Soak gRPC Testing
Soak testing, perhaps because I’m a swimmer, I like the sound of this one the best, however it is nothing to do with the lake or beach. Soak gRPC testing is the ability to test your system at normal load for a prolonged period and reveals:
- Memory leaks
- Database connections not closed
- Code bugs that don’t offer themselves up under normal testing.
- Response time differences: comparison from the first test to a test after near the end of the run.
Smoke gRPC Testing
Smoke gRPC testing is the ability to hit a specific part of the system. This gives you a very temporary reassurance that the system is running at the point that the test was run. Making this repeatable through automation with k6 at it’s heart and over many tests gives you the ability to test your whole system on demand.
Customer asks, boss asks, you ask, PM/PO asks, end users ask, B2B ask is it stable? The answer to this can be yes or no with confidence in a matter of minutes if you have your smoke tests automated. Automated smoke tests can be run on demand, on merge to master branch, just after release or on schedule (say nightly). All of this is possible in a heartbeat with a smoke testing framework. You can even run the same framework over multiple environments.
The beauty of k6 is that you can store your JavaScript code in source control so many developers can work on your framework at the same time, allowing tests to be added as part of users stories development lifecycle. Combine this with other technologies and you can build up a complete framework like .NUnit for .net and JUnit for java.
You can add counters for failure tests and success tests, you can also make the whole test abort on first failure, if it’s mission critical code tests and you can have longer and shorter test runs just by creating another test cycle.
My Udemy course focuses on this in great detail, casting a spotlight on the following features:
- N number of tests (as many as you want).
- Negative tests with positive outcome.
- Standalone tests.
- Tests that rely on other tests results via session.
- Data driving these gRPC tests from JSON.
We also talk about how you can run these tests from a JSON array.
Sharing session data between calls
I touched on this in the framework section but say you have a booking form and you have an id provided by the server on the first call. This id has then to be provided by all subsequent calls. We use an external library to achieve this, and what this allows us to do is to pass data between stages. This also gives us statistics for each call individually and as a group.
In additional to running this kind of test as part of the framework containing n number of smoke test you can run this as a load, spike or soak test. You can also name your stages for easy identification.
Adding Extra Exception Handling to your K6 gRPC Tests
As of 0.42.0 of K6 there does not seem to be any error statistics on connection to gRPC services, in particular when it comes to load, soak and spike tests. I help you find these errors and normalise your statistics so every call is represented at the end.
In our advanced topics we then take this information and talk about retrying connections on failure for a given number of attempts per virtual user and learning how to abort the whole test run if we get too many errors.
Response & Assertions using K6 gRPC Tests
The response object is what is returned from your gRPC service and is different to what is returned from a typical K6 web API request. We analyse what is available to consume and use as part of our assertions statements, which is important because if you don’t know what you are getting from your invocation in k6, then how can you test for it?
“Check” is the first method we examine, for checking our response object and we explore adding many assertion statements.
“Describe” is the second method we explore, this is a more human readable version of assertions. We explore the common pitfalls to using this library compared to the check library when creating our assertions.
Sharing Your K6 connections, gRPC Iteration Tests
This is not something that comes out of K6 without a work around. I talk through how to achieve both:
- Sharing a connection over all iterations of a load, spike and soak test.
- Sharing one connection per virtual user (more like reality in most systems.)
Randomise gRPC Arguments
In this topic we cover off creating randomised input data into our gRPC Load, spike and soak tests. It’s important to ensure that your tests are as realistic as possible and at no point will the payloads be identical in live, so why test this way?
Jenkins deployments and gRPC Tests
Jenkins is arguably one of the most common build and deployment systems in the industry. We speak about how to set this up using Docker and how to deploy it with environment variables, set up a pipeline and make the results display without colour.
Kubernetes deployment with Minikube and gRPC Tests
Finally, we look at deploying safely on Kubernetes, using a combination of sealed secrets and config maps and environments variables. We also look at how to run our test as part of a Job and briefly as part of a CRON job. We also explore how to debug the k6 docker image, should you come unstuck.
Debugging
At all stages in this course I do not provide perfect content, the main objective of this course is that you see the errors as they happen and what i do to resolve these. Including grabbing the line number and taking a hands on look at what went wrong. I hope you see this as a learning experience as there will be a time when your code will not work and you will need to have experience of debugging the shell/Javascript.
gRPC Testing The Rarer Types
I also touch on some of the rarer types such as “Any” and “oneOf” types. We have real examples of these being adopted in our test gRPC service, so you can get practical example of working with these types. For sample content on the Any type please look at my other blog article K6 and Google Protobuf Any successfully Demystified