Better, Faster, Stronger – Delivering High Quality Products

Did you know you can enable your team to build better software faster while having a stronger team culture? Too good to be true?
In recent years, agile has influenced early involvement of testing in the development cycle. With this more and more testers are testing new functionality as soon as a commit is pushed. Yet such teams still fail to deliver high quality software. Why? What is missing?
Working with various diverse teams across multiple projects, Finn realised that testing doesn’t actually improve software quality. It’s just a bar assuring a certain level of quality that already exists. In order to actually improve we must get involved into much more than simply testing and think about the product as a whole.
In this session, Finn will share specific examples of how engaging with the business, engineering, process optimisation as well as the entire cross-functional team can lead to significant improvements in the product’s quality. At the end of the talk, you will know how to start with a holistic approach to improving product quality throughout the entire software delivery lifecycle.

— this is a talk I am currently doing on various conferences. Some people asked me if I can share the slides, but the files are just a bit too large. So instead I decided to include a recording of the talk (with the slides) here, as some of them only play well with the presentation.

How changes to your process increase the quality of your software.

When we have a close look at the process steps of a typical agile software delivery team, we will realize what is going on there.

Screen Shot 2019-02-07 at 10.01.57.png

In the first one, “in Analysis”, we plan to create value. Once we have a plan how to add value for a user to our product we but this story into “Ready for Dev”. And here we are doing literally nothing, or, in other words: wasting time. In the best case, the story is still good to play – in the best case! Often stories are a bit outdated already before they are picked up and if they were lying around a bit too long they may even be completely degraded. In the next column “in dev” we are actually adding the value to the product and in the next “Ready for… “ process step we are – you guessed it – wasting time too. Until someone can check for the planned value and finally deliver the value. If we find a bug while testing, the deliverable software is in the state of analysis all over again – the bug is prioritized against other features and maybe fixed and maybe not.

As QAs we have little influence over the “Ready for Dev” column, but the “Ready for QA” is ours. Removing this column can have some positive impact on the velocity and quality if you pair it with another tool: work in progress (WIP) limits.

The idea is the following: if there is no “Ready for QA” column there is no place for a developer to just drop a ticket/story before picking the next piece of work. Unless the story was put directly into “in QA” (without anyone actually working on it). If the team then agrees on WIP-Limits, one could argue that a single QA person can work on one or max two stories at the same time. Thus, if there are already two stories “in QA” another one cannot be added. Thus, the story cannot move, thus the devs cannot start a new one (the “in dev” column should also have a WIP). The best way is to help the QA do the job and thus everyone gets more involved into testing – a big win for quality.

This will decrease the time to market for new features drastically. As a plus, bug-fixes can be delivered faster, too. Even in the standard process.

Screen Shot 2019-02-07 at 10.07.02.png

We applied these measure on different projects in different context. In one case, we were able to decrease the cycle time (= time from “Analysis” to “Done”) for stories from 13 days to 4 days – without anyone having to work “harder”. The restrictive WIP limits have a very good effect: if you can’t have so many stories in “in dev” you do not have to perform any context switches. You focus on getting one thing done before the next. Surprise: that actually helps to get things done! And being more focused on one specific task leads to fewer mistakes, thus fewer defects.

In another example where we applied the same technique, we were able to increase our velocity by a bit more than 30% (!) without any impact on the quality of the software.

These are some techniques to truly bake chocolate into the muffin: reduce waste in order to increase your focus on the really important things. Spend your available time on the urgent matters and “suddenly” you end up with a higher quality product.

How we do “Quality” at ThoughtWorks Germany

Thanks to the other QAs in ThoughtWorks Germany for contributing thoughts to this topic over the past months: Sarah@DizMario, @Nadineheidrich and @bratlingQA


What is testing?

Testing is a method to analyze the quality of a given software. It is a method that is applied after the software is developed. If the software has an insufficient level of quality, yet another cycle of development & testing is needed to increase and measure the quality again.

Testing is bug detection.

Metaphor: Testing is like putting chocolade on a muffin, but exclusively after baking it.

What is QA?

There are other methods that can improve / increase the quality of a software while it is developed. Those methods decrease the amount of cycles of development & testing that are required  to reach a certain level of quality.

QA is bug prevention.

10797887872_IMG_1353
We want to consider chocolade while baking and only put some additional on top.

How we are testing

In addition to the tools and processes that allow us to build high quality software from the first line of code (see below), we have the highest standards for testing software. As mentioned before,  we are well aware that testing can only analyze a software’s current state and show the presence of issues/defects. Another cycle of development is needed to actually improve the quality. That leads to the known problem: whenever the testing is “successful”, the cycle time of stories increases and the delivery of a new feature needs to be postponed.

To minimize this delay, we apply the most efficient testing approaches to provide fast feedback for developers. This reduces the overall time to market of new features. With these methods we are able to reduce the cycle times significantly while improving the overall quality of the software in different projects:

  • Our tests are fast and effective. We run Unit-, Integration and End-to-End tests in a well shaped testing pyramid. This allows us to quickly check if our application behaves as expected. Writing the right tests on the right level reduces the time we need for regression tests from days to minutes. This includes – amongst other things – a 100% automation of regression tests.
  • Of all the tests in the pyramid, we take special care of the integration tests (of different services) to assure the architecture’s resilience. One of our favorites are the consumer driven contract tests. They allow different teams to work independent with loosely coupled services while ensuring that the entire system behaves well altogether.
  • For us, testing is an integrated activity within the software development team and not an independent, separate discipline. There are two ways to get a story tested:
    1. When a story needs QA attention, you move the ticket into a “QA” or “ready for QA” column. The person who is in the role of the Quality Analyst then picks up the story as soon as possible.
      (this is push & role-focused ⇒ that is the Scrum-way with experts in the team)
    2. When a story needs QA attention, you look out for the capability in the team. Any person with some capabilities in testing (often but not always the QA) rotates into the story.
      (this is pull & capability-focused ⇒ that is the Kanban-way with cross-functional people in the team)

      Guess what. I prefer the 2nd approach. The 2nd one is real team-play. And it decreases the cycle time of a single story and thus increases your velocity! Devs will learn (more) about testing and QAs can pair on the programming part, eg. to sort unit- and integration tests into the pyramid ⇒ baking the chocolade inside.

  • For exploratory testing we do not always apply the same standard methods but acknowledge the individual context we are in. Only then can we make use of the various advantages of different test methods. We find all kinds of tools in our box: Behaviour Driven Testing, Acceptance Testing, End-To-End Testing, Scenario based Testing, User Journey Testing, Integration Testing, System Testing, Risk based Testing, Penetration (Security) Testing, UX testing, performance Testing, Guerrilla Testing…

How we do QA

As mentioned before, we really need to learn all about testing. And its a mastery to study. However, its only a (small) part of our job and every day live. Besides testing we look into other things, as we know that good software is only the first step towards a high quality product.

We create a culture in the team, where the aspect of quality is an important asset for each team member. In this context we can address the team’s current needs with a wide set of processes, frameworks and tools that we as ThoughtWorkers already use or create if they do not yet exist (e.g. Selenium).

We acknowledge that we cannot build defect free software.  Hence, we focus on defect prevention and establishing an overall quality mindset. We assure a high level of quality in software through tools and processes that allow us to prevent defects and find errors fast:

  • Well designed services to ensure a resilient architecture. There are so many things to work on if you want to improve the resilience. You can have the best software without bugs. It wont help you if your servers are down for the bigger part of the day. From a QA point of view, we are interested in circuit breaker (self-healing systems), feature toggles, well designed APIs and a kick-ass monitoring:
  • Monitoring! This is so important. And so many people think that “only” Ops should care. What a misunderstanding. Constant monitoring of all services and environments is a shared discipline to be able to react quickly on any arising issue. No matter how good you test (see above), some defects will slip through to production. The best way to reduce their impact is a combination of a good monitoring and quick deployment. If we are able to release a fix fast (= best case: 20 min after a bug is found), we can reduce the impact significantly. This is what monitoring is for. Learn more in this podcast.
Screen Shot 2019-02-15 at 20.12.49
  • We have a strong focus on Continuous Integration and Continuous Deployments best practices, such as fully automated regression testing. You can read all about it in the other post as well as talk in Ljubljana.
  • Test Driven Development for high test coverage and fast feedback during development is also an important thing to notice. While most developers know that a tests are written first in TDD, not all know about the testing pyramid, much less of its benefits. Hence, a pairing of QA and dev while practicing TDD can be of high value. This is how you bake quality in!
10798083424_IMG_1367
  • I just mentioned the consistent pair programming. Pairing allows best designs and fewer defects from the beginning. Make sure to rotate frequently and across the roles. Pairing is a general activity for most tasks in a team. Pair-programming is just one of them.
  • We love Feature Toggles™. They give us maximal control over the features in production and canary releases. A very easy method I usually use is to give chocolate while the standup to the pair that implemented a toggle the previous day. This is a fun way to talk about it, remember it and give a sweet incentive to build it in when it was forgotten. If you find the time to use them it will make your lives much easier. You will not need rollback strategies any more and it is a very, very good safety net. Quality “assurance” at its best!

The mix for the win.

Of course we combine the two aspects of QA and testing. And this is the biggest challenge for us. Where to focus at what point of time. Where do we need more attention and what part of the application / system / team is running smooth? Ultimately,  we try to pick the right tools and create the right mindset to build a high quality product.

Pure Performance

Episode 21: How ThoughtWorks helped Otto.de transform into a real DevOps Culture

Finn Lorbeer (@finnlorbeer) is a quality enthusiast working for Thoughtworks Germany. I met Finn earlier this year at the German Testing Days where he presented the transformation story at Otto.de. He helped transform one of their 14 “line of business” teams by changing the way QA was seen by the organization. Instead of a WALL between Dev and Ops the teams started to work as a real DevOps team. Further architectural and organizational changes ultimately allowed them to increase deployment speed from 2-3 per week to up to 200 per week for the best performing teams.


Episode 22: Latest trends in Software Feature Development: A/B Tests, Canary Releases, Feedback Loops

In Part II with Finn Lorbeer (@finnlorbeer) from Thoughtworks we discuss some of the new approaches when implementing new software features. How can we build the right thing the right way for our end users?
Feature development should start with UX wireframes to get feedback from end users before writing a single line of code. Feature teams then need to define and implement feedback loops to understand how features operate and are used in production. We also discuss the power of A/B testing and canary releases as it allows teams to “experiment” on new ideas and thanks to close feedback loops will quickly learn on how end users are accepting it.

Are we only Test Manager?

This is a translation of the original blog post that I wrote with Diana Kruse, Natalie Volk and Torsten Mangner. While writing this blog once more just in another language, I took the liberty of adding some personal notes here and there.


In every development team at otto.de there is at least one tester / test manager / QA… or however you would call the person, who is shaping the mindset for quality.

Until recently, “test manager” was the dominating description at Otto – a very rigid and bureaucratic term. Although the intention was good to emphasize that we do not only execute tests, but we also manage them! Meanwhile, even managing tests is only a very small part of the value we deliver.

In a bigger workshop Finn and Natalie, two of our “test managers” picked up on this contradiction and worked things out. We were sure that it would not be sufficient to write “agile” in front of test manager. Hence, we developed a new understanding of our role that looks and feels like this:

coaching1.jpg
We are the teams’ Quality Coaches

We support the teams to understand “quality” as a collective responsibility. We achieve this by working intensively with all roles of the team rather than talk about generic concepts. We establish knowledge and practical approaches regarding the topic of quality.

lifecycle.jpg

We See Through the Entire Story Live Cycle

Together with the team we take care that our high standards of quality are regarded long before the development of our product starts: We suggest alternative solutions during the conception of the story and indicate potential risks. We avoid edge-case problems later on by thinking about them while writing the story. We pair with developers, so that we know that the right things are tested in the right place. Thus, we have more time to talks to our stakeholders and users during the review. With the right monitoring and alerting we are able to observe our software in production.

continuous

We Drive Continuous Delivery / Continuous Deployment

One central goal is to deploy software to the production environment as risk free as possible. Therefore we try to change as little of our codebase as possible and roll out every single commit automatically. We are using feature toggles, to switch on new functionality independent of these deployments. This has two major advantages: we can roll out our software to customers (almost) at the speed of light and get fast feedback for new developed features.

pyramid.jpg

We are Balancing the Test Methods of the Testing Pyramid

We know how to test what on which level of the testing pyramid. We use this concept to create a lot of fast unit tests, a moderate number of integration tests and as few end-to-end tests as possible. This does not only speed up our pipelines but it makes our tests more stable, more reliable and easier to maintain.

Additionally, in our tool box we can find all kinds of tests (acceptance tests, feature tests, exploratory tests), methods (eg. test first, BDD) and frameworks (like Selenium or RSpec). We know how to use those tests, methods and frameworks on all levels of the testing pyramid.

(as a side note: this indeed implies to run eg. Selenium tests on a unit test level if applicable)

agile.jpg

We Help the Team to Choose the Right Methods for a High Quality Product

Being specialists, we know all (dis) advantages of different methods and can help the team to benefit from the advantages. We learned that pairing will enable knowledge transfer, communication, faster delivery and higher quality.  Besides pairing, test driven development is one of the key factors to create a high quality product from the beginning.

Flexible software can only emerge from flexible structures. This is why we are not dogmatic about processes and methods but decide together with the team what mix of processes we really need to get our job done.

pairing.jpg

We are Active in Pairing

We do not only encourage our developers to pair, we also have fun pairing ourselves. In tThis way we can point to problems even while the code is being written. To avoid finding all edge cases only during development we also like to pair and communicate with Business Analysts, UX-Designers and Product Owners. Together with the operations people we will monitor our software in production environment.

The pairing with different people and different roles allows us to further develop our technical as well as domain knowledge.

challenging.jpg

We Represent Different Perspectives

By taking on different positions we prevent unidirectional discussions. We try to avoid typical biases by challenging assumptions about processes, methods, features and architectures. This enables us – from time to time – to show a different solution or an alternative way to solve a given problem. It helps us to reduce systematic errors, money pits and to objectively evaluate risks while developing our software.

communication.jpg

We are Communication Acrobats

We are the information hub for all kinds of things inside and outside the teams. We make special, constructive use of the grapevine, a phenomenon that practically occurs in every company with more that 7 people.

We are enablers for communication. This may be the communication of a pair of developers, between many or all team members or between teams throughout the organization.  By facilitating this coordination we can reduce obscurities about features or integrations of systems and hence get our software into a deliverable state faster.


After developing this role, we engaged more and more of our “agile test managers” with this concept. They were so enthusiastic about it, that they wanted to apply for the job once more right away. The only thing missing was a good name: As in every cross functional team we have different specialists and one of those people is the driving force for high quality we found the perfect name: the Quality Specialist.

specialist.jpg

(Side note: the German term for “quality assurance” (QA) is “QualitätsSicherung” (QS). Using the same abbreviation made it even easier to adopt the new term.)

Quality Specialist is a very well fitting name for this stretched role. Although we are broad generalists, our core value lies in shaping a quality mindset and a culture of quality in a team.

Those were the first steps on a very exciting journey. The next thing to do is talking with other roles in order to find out how this new comprehension of the role changes our daily work. Furthermore, almost no one fulfills this role description today. Thus, we need to grow, level up and reflect on our development. The most fun part is that we can learn a million things in different domains from different people.