consumer driven contracts

How we do “Quality” at ThoughtWorks Germany

/ Finn Lorbeer / Leave a comment

Thanks to the other QAs in ThoughtWorks Germany for contributing thoughts to this topic over the past months: Sarah, @DizMario, @Nadineheidrich and @bratlingQA

What is testing?

Testing is a method to analyze the quality of a given software. It is a method that is applied after the software is developed. If the software has an insufficient level of quality, yet another cycle of development & testing is needed to increase and measure the quality again.

Testing is bug detection.

Metaphor: Testing is like putting chocolade on a muffin, but exclusively after baking it.

What is QA?

There are other methods that can improve / increase the quality of a software while it is developed. Those methods decrease the amount of cycles of development & testing that are required  to reach a certain level of quality.

QA is bug prevention.

10797887872_IMG_1353
We want to consider chocolade while baking and only put some additional on top.

How we are testing

In addition to the tools and processes that allow us to build high quality software from the first line of code (see below), we have the highest standards for testing software. As mentioned before,  we are well aware that testing can only analyze a software’s current state and show the presence of issues/defects. Another cycle of development is needed to actually improve the quality. That leads to the known problem: whenever the testing is “successful”, the cycle time of stories increases and the delivery of a new feature needs to be postponed.

To minimize this delay, we apply the most efficient testing approaches to provide fast feedback for developers. This reduces the overall time to market of new features. With these methods we are able to reduce the cycle times significantly while improving the overall quality of the software in different projects:

  • Our tests are fast and effective. We run Unit-, Integration and End-to-End tests in a well shaped testing pyramid. This allows us to quickly check if our application behaves as expected. Writing the right tests on the right level reduces the time we need for regression tests from days to minutes. This includes – amongst other things – a 100% automation of regression tests.
  • Of all the tests in the pyramid, we take special care of the integration tests (of different services) to assure the architecture’s resilience. One of our favorites are the consumer driven contract tests. They allow different teams to work independent with loosely coupled services while ensuring that the entire system behaves well altogether.
  • For us, testing is an integrated activity within the software development team and not an independent, separate discipline. There are two ways to get a story tested:
    1. When a story needs QA attention, you move the ticket into a “QA” or “ready for QA” column. The person who is in the role of the Quality Analyst then picks up the story as soon as possible.
      (this is push & role-focused ⇒ that is the Scrum-way with experts in the team)
    2. When a story needs QA attention, you look out for the capability in the team. Any person with some capabilities in testing (often but not always the QA) rotates into the story.
      (this is pull & capability-focused ⇒ that is the Kanban-way with cross-functional people in the team)

      Guess what. I prefer the 2nd approach. The 2nd one is real team-play. And it decreases the cycle time of a single story and thus increases your velocity! Devs will learn (more) about testing and QAs can pair on the programming part, eg. to sort unit- and integration tests into the pyramid ⇒ baking the chocolade inside.

  • For exploratory testing we do not always apply the same standard methods but acknowledge the individual context we are in. Only then can we make use of the various advantages of different test methods. We find all kinds of tools in our box: Behaviour Driven Testing, Acceptance Testing, End-To-End Testing, Scenario based Testing, User Journey Testing, Integration Testing, System Testing, Risk based Testing, Penetration (Security) Testing, UX testing, performance Testing, Guerrilla Testing…

How we do QA

As mentioned before, we really need to learn all about testing. And its a mastery to study. However, its only a (small) part of our job and every day live. Besides testing we look into other things, as we know that good software is only the first step towards a high quality product.

We create a culture in the team, where the aspect of quality is an important asset for each team member. In this context we can address the team’s current needs with a wide set of processes, frameworks and tools that we as ThoughtWorkers already use or create if they do not yet exist (e.g. Selenium).

We acknowledge that we cannot build defect free software.  Hence, we focus on defect prevention and establishing an overall quality mindset. We assure a high level of quality in software through tools and processes that allow us to prevent defects and find errors fast:

  • Well designed services to ensure a resilient architecture. There are so many things to work on if you want to improve the resilience. You can have the best software without bugs. It wont help you if your servers are down for the bigger part of the day. From a QA point of view, we are interested in circuit breaker (self-healing systems), feature toggles, well designed APIs and a kick-ass monitoring:
  • Monitoring! This is so important. And so many people think that “only” Ops should care. What a misunderstanding. Constant monitoring of all services and environments is a shared discipline to be able to react quickly on any arising issue. No matter how good you test (see above), some defects will slip through to production. The best way to reduce their impact is a combination of a good monitoring and quick deployment. If we are able to release a fix fast (= best case: 20 min after a bug is found), we can reduce the impact significantly. This is what monitoring is for. Learn more in this podcast.
Screen Shot 2019-02-15 at 20.12.49
  • We have a strong focus on Continuous Integration and Continuous Deployments best practices, such as fully automated regression testing. You can read all about it in the other post as well as talk in Ljubljana.
  • Test Driven Development for high test coverage and fast feedback during development is also an important thing to notice. While most developers know that a tests are written first in TDD, not all know about the testing pyramid, much less of its benefits. Hence, a pairing of QA and dev while practicing TDD can be of high value. This is how you bake quality in!
10798083424_IMG_1367
  • I just mentioned the consistent pair programming. Pairing allows best designs and fewer defects from the beginning. Make sure to rotate frequently and across the roles. Pairing is a general activity for most tasks in a team. Pair-programming is just one of them.
  • We love Feature Toggles™. They give us maximal control over the features in production and canary releases. A very easy method I usually use is to give chocolate while the standup to the pair that implemented a toggle the previous day. This is a fun way to talk about it, remember it and give a sweet incentive to build it in when it was forgotten. If you find the time to use them it will make your lives much easier. You will not need rollback strategies any more and it is a very, very good safety net. Quality “assurance” at its best!

The mix for the win.

Of course we combine the two aspects of QA and testing. And this is the biggest challenge for us. Where to focus at what point of time. Where do we need more attention and what part of the application / system / team is running smooth? Ultimately,  we try to pick the right tools and create the right mindset to build a high quality product.

How To: Consumer Driven Contract Tests

/ Finn Lorbeer / Leave a comment

In my last post I wrote about how to test micro services. When I wrote about the most difficult part of the testing, the integration, I said that it would blow the size of a post to go into the necessary details. I am sorry and I want to make up for it here.

When we design and create the interaction (and thus integration) of different services with each other we rely very much on the principles of “Consumer Driven Contracts“, in short CDCs. These Contracts can be tested. While the basic idea seems simple (“let other test the things of our service they think they need”) it implies a change in the way how we think of who-tests-what. So I want to spend some time explaining the concept step by step in many pictures:

Imagine you are part of a team (the blue team) and work with another team (the purple team). The purple team’s service will ask the service of the blue team about the email address of a specific user (ID). They agreed – for the sake of the argument – to communicate in json format via a RESTfull interface. In other words: the consumer (purple team) has a contract with the provider (blue team).

Now our friends in the purple team want to make sure, that the blue team will always give them an email as an answer. The first important thing is that the purple team needs to trust the blue team in figuring out the right email address for the given user. If the purple team starts to validate the blue team’s results for the actual content you have a lack of trust. This is a different, much more severe problem. Writing CDCs does not help you to solve this.

Hence, the purple team tests that they get any answer and that it contains an email address (at all). They will not test which one.

Screen Shot 2019-02-15 at 20.46.36.png

So assuming the blue team does it all right, the purple team would then just write a test that sends any ID and gets back any email in a valid json.

Now, whenever the purple team wants to know if the contract is still valid they let the test run. But then think about when the contract would potentially break: in this picture the contract will only be broken if the blue team introduces (unintended, not communicated) changes to their service. So isn’t it a much better idea to run this specific test whenever there is a change to the service of the blue team?

Screen Shot 2019-02-15 at 20.47.55.png

And this is what we do: the purple team – the consumer – drives the the contract to make sure it holds up to the agreed functionality of the blue team’s service. In this way – having the test of the purple team in the CI-System of the blue team – the blue team can make sure to not accidentally break the contract.

A few days later, when everything works smoothly and both teams feel comfortable, the blue team does not need the purple team any more to do a release. The blue team knows that the test they received from the purple team discovers all flaws. The purple team can concentrate on something else.

Screen Shot 2019-02-15 at 20.50.14.png

This is it! This is the basic idea of the CDC. The purple team writes and maintains the test. But the blue team executes it. If the request would go the other way around (the blue team sends an email and expects back an ID), then the test would be written by the blue team and executed by the purple team.

But this is only the start. Of course, the blue team has more than only one service. And there is not just the other purple team, but many teams doing requests to the different services of the blue team. Now the blue team members have to be quite disciplined and ask the other teams for all the CDCs. This sound a bit like an overhead and quite exhausting. But here comes the big benefit: the blue team is in all its work (and more important: deployments) independent of the other teams and independent of the availability of the other teams services in test environments.

Anyone who tried to get multiple services of various teams running in different (test) environments will smile now knowing of all the pain this usually inflicts. In this picture the blue team mitigated this risk/problem.

Screen Shot 2019-02-15 at 20.48.57.png

While the blue team is doing quite well there is more to it. Being independent of other teams and their services is nice. But still you have dependencies between your own single services. You should also resolve those. Cover the connections between the services of the team with CDC tests, too. It will give you:

  • fast feedback (surprise!)
  • independent deploys
  • reduced complexity

Each of those three points on its own leads to a quicker development of features and a reduce in cycle time. As I get the combined benefit in this situation there is nothing I would argue against, so lets do it!

If all tests are in place, each service of the blue team is totally independent. It can be deployed at any time and still make sure the other services work properly.

Screen Shot 2019-02-15 at 20.49.46.png

And did you realize what this means? We do not need a big, heavy, error prone and expensive set of selenium end-to-end tests. We mitigated so many risks already and can be sure of interactions that are working.

You cannot test anything here, but certainly a lot. In any case, improving a lower layer of the testing pyramid usually implies that you can reduce something at the top. That means: less end-2-end test => less time for maintenance => more time to grow out of the role as a classic test manager.

Let’s embrace this!