Moin MOIA

April 1, 2021April 6, 2021 / Finn Lorbeer

I joined my previous employer, ThoughtWorks, round about 6 years ago. It has been a fantastic journey. I can certainly say that the vibrant (and a bit chaotic) environment helped me, to again and again become a better version of myself. I got an incredible amount of feedback from many different people with vast experience in many different fields of “technology”. This was a key enabler for me over the past years.

I joined ThoughtWorks to improve my “QA Engineering” skill, especially in End-To-End Test Automation. And uhh… how much did I have to learn! Thanks to Dani, Mario, Nadine and Robert (who were the QAs at ThoughtWorks when I joined), I quickly discovered that the scope in which you can increase the level of quality was massive. So much more than I could have imagined before.

I joined Otto as my first project and many more followed. I gathered experience in 15 different teams over the course of those 6 years. They distributed across all kinds of tech stacks, domains, companies, projects, locations… One thing was always the same though: all teams had errors in the software. Since starting my career I’ve seen all kinds of Data Storage (SQL, PostgreSQL, MariaDB, CouchDB, MongoDB, Cassandra, Redis, Elastic Search, Splunk, DynamoDB, S3, …) and worked with all kinds of Programming Languages (Turbo Pascal, Fortran, C++, Ruby/Rails, Javascript, Swift, Java, Clojure, Kolin, Scala, Typescript, …). However, regardless of the technology in use, there were always errors.

And isn’t that odd? Let me repeat that last part: “Regardless of the technology in use …”. That would mean that the technology is not the problem. And since we already make errors when we use technology to solve errors in the real world, let me suggest to not only rely on technology to solve errors in technology. The conclusion, however, is that we should take a step back from “test-automation-everything” and admire the core problem a bit longer. And ultimatley, it is easy to spot: The one common denominator, the one factor which contributed to errors in all those systems is us – us Humans.

Whoever worked with me in the past years will have noticed, that I started to focus not only on the technological part and some kind of governance ( = “the process” ), but to equally look into team dynamics. And to understand how information is transported through the team. How requirements were very clear to everyone – or more often not. Using Conway’s law in reverse, I am trying to build teams that resemble a high quality human-interaction so that their services and software display this very same attribute. The only way to build teams in this way, is to make “Quality” (for whatever “Quality” means in detail for a specific context) an integral part of a team’s culture.

The problem is, that it is difficult to map a job with the Title “team culture enablement to improve the Quality of Software-Products using Convey’s law in reverse” into the QA career path of a high performing global cutting-edge technology consultancy 🙂 In the beginning of my employment with ThoughtWorks, it was a great journey, as a QA, since the environment allowed me to “shift left” and “shift right” and “deep dive” in my role. All at the same time! I am very grateful for this opportunity that was given to me. I am thankful for the room, my colleagues gave to me. I hope I could also deliver on their expectations!

However, the further my journey took me, the more I noticed how limited a “QA Role” was defined for the clients I worked with. Hence, for every new project I had to emancipate from the initial client’s role understanding and evolve this together with the client in a positive and constructive way. After some time, even this small cultural transformation becomes repetitive and then tedious.

But it must have been fate: At some point I joined a client organization, where there were many interesting and wild thoughts in those directions around “Quality”. It was a Hamburg Startup. The Name of this company is MOIA.

However, before I dive into the specifics of MOIA, I have to do the unavoidable detour for a bit of context setting: I joined the MOIA-project about half a year before the Corona Virus hit us. Once we were in the middle of the pandemic, my family reflected how we wanted to shape our future and whether we see this future in a world-famous metropolis or – and this is what we ultimately decided for – rather in the country side. Hence, we shifted gears and started to look for a new home in a small village in the middle of Schleswig-Holstein. The aspect of more Home-Office obviously contributed to paint this picture of our family-future. With Moia having the majority of their tech development in Hamburg, it was a perfect match to shape a new path forward ❤

By the time that we concluded our family plans, I had also established deeper connections (I want to say: roots!!) within MOIA. Back in the days of the first Corona wave, I worked with one of the best and most empathetic teams I ever had the pleasure to work with. And all of those fine folks were from MOIA! Besides, the current set of challenges is extremely interesting: a distributed system of many services and apps and web technologies. And a great demand on Quality for the end product. However, in many places the bi-weekly releases are still served with big amounts of manual regression testing. What a Shocker! Someone needs to do something about this! And I want to believe that those are areas, where I can contirbute to.

And the best thing is: The company has an amazing culture of trustful and constructive collaboration. New ideas are welcome and (sometimes more, sometimes less) enthusiastically implemented 🙂 Appreciation is given towards each other freely and frequently.

Furthermore, MOIA is a company with a great Vision. MOIA participates on the endeavour to change our cities, to very actively contribute to one puzzle piece of the “Verkehrswende” (= transformation of mobility and transportation in Germany). I found a role where I can contribute to the culture of many teams and still have a direct impact on the Quality, the processes around it and the Strategies and Approaches (and also to testing throughout the entire company). This is a huge, difficult and quite complex task.

I am looking forward to truly immerse myself into this task, to really and wholeheartedly fall in Love with the Product that I am am helping to build. To continue to work with great people that I’ve met over the last two years and to contribute to shape a future for MOIA and our cities. And to plant some seeds in a company culture. To grow them and to cater to their needs for many years – and not months.

I am grateful for my journey up until now, which brought me to this point. My thanks go to all the people who contributed to the experiences that I have and who definitely had their share in making me the person that I am.

Better, Faster, Stronger – Delivering High Quality Products

December 7, 2018 / Finn Lorbeer / Leave a comment

Did you know you can enable your team to build better software faster while having a stronger team culture? Too good to be true?
In recent years, agile has influenced early involvement of testing in the development cycle. With this more and more testers are testing new functionality as soon as a commit is pushed. Yet such teams still fail to deliver high quality software. Why? What is missing?
Working with various diverse teams across multiple projects, Finn realised that testing doesn’t actually improve software quality. It’s just a bar assuring a certain level of quality that already exists. In order to actually improve we must get involved into much more than simply testing and think about the product as a whole.
In this session, Finn will share specific examples of how engaging with the business, engineering, process optimisation as well as the entire cross-functional team can lead to significant improvements in the product’s quality. At the end of the talk, you will know how to start with a holistic approach to improving product quality throughout the entire software delivery lifecycle.

— this is a talk I am currently doing on various conferences. Some people asked me if I can share the slides, but the files are just a bit too large. So instead I decided to include a recording of the talk (with the slides) here, as some of them only play well with the presentation.

How establishing a trustful error culture in your team gives you the final boost in quality

December 23, 2017February 15, 2019 / Finn Lorbeer / Leave a comment

As pointed out before: With continuous deployments the time from commit until live for any commit is the time you need until a bug can be fixed – at best! How can you push on this limit?

It usually requires that all people are aware of the code, the business, the infrastructure, the test and the pipeline. If each person has all this knowledge only then can you react quickly, without gathering the “right” people to ship a fix. In other words: you need a high performing and truly cross functional team. Let me emphasise this here once more: from our perspective there is a clear advantage for the quality of the product here: it is about reacting to (breaking) changes (even) faster.

What is the leverage you as quality advocate have? Talking about errors! Your team will do mistakes, this is good. We would be out of jobs if they did not. We are here to find them and to help the team to prevent them going forward. We are the experts on errors – in a way. So make sure that your team does not waste a mistake, but learns from it.

Idea for image originally from: http://ww2.kqed.org/mindshift

There are some mistakes that your team will be doing:

Sloppy mistakes. This is the most common. There are probably a dozen of them in this very blog post. The human brain can only concentrate for about 10-15 minutes, until it needs a mini break. If we skip that break, we tend to do small mistakes.
A typical mitigation strategy for this is pairing of any two people. When one is not fully concentrated for a moment the other one most often is.
Aha-Moment Mistakes. This is when you encounter a (small) new learning by doing a mistake and finding out about it. It may happen if you understood something by reading or even much more so while explaining something.
We also mitigate this by pairing – ideally by different skilled developers. If a more senior person explains a lot to a more junior person both have a higher chance to do this error – and learn something from it right away.
Stretch Mistakes. In this scenario, we are quite away that we step out of our comfort zone. But when you need to or want to try something new you have to at least try, even if you already know that it’s more error prone than business as usual.
Our way around such requirements are so called “spikes”: small, time boxed stories that make sure we can trial-and-error in a save environment. Thus, the result of the spike can be a small prototype that is thrown away. A new service that got some basic things just working fine. Or a branch that is ready for an all-team-code review.
There are also some high-stakes mistakes. Those are equally risky as the stretch-mistakes but usually there is much less to gain. Those are not worth the effort. Prevent your team from doing these.

There are probably many more ways to categorise errors. There are also many more mitigation strategies, i.e. in our case we have the safety nets of our test-driven development. But differentiating the different types from one another is something that is typically quite easy for analysts who mostly work in the field of quality.

By encouraging your team to do errors (in a safe environment) you automatically get the basics right for a great error culture. And as soon as the impact of an error is reduced people will be much less afraid of doing errors. And if you are less afraid at work you are usually more creative. If a broken built is then nothing bad or painful (while still an urgent matter!) you will have a more relaxed, trustworthy and creative atmosphere. And guess what – in the end a lot less error happen in such an environment. Another boost for Quality!

The last fine tuning is to make sure your team has fun. Fun? Really? Yes.
Just like the point before: in an environment where people like to come to work, where they are happy to communicate and interact, where it’s fun to get work done, people will also be more focused and more passionate about what they do. As a result, they will do less errors with smaller impact. The ultimate boost for Quality!

Joint forces of the analysts: improving the quality of software even before its built

December 23, 2017February 15, 2019 / Finn Lorbeer / Leave a comment

Sometimes people say that QAs are the headlights of a project. The developers are the train engine and the BAs are the map of the surrounding area – they know where the train in the night shall ride. And it will somehow get moving. But with headlights that inform you early on about potential obstacles you can maybe find another route on your map.

Source: wallpaperpulse.com

Sometimes people say that QAs and BAs and devs work on the same product – just in different times: The BAs have really good idea what is coming next. They are constantly thinking about the product in one month – rather in the future. Devs are working on the product of the next days, max. next week – thus the immediate future. And you as a QA usually know best how the product behaves today – you know about the present product.

When it behaves unintended you talk to the same stakeholder as a BA who identifies intended behavior changes to be implemented. It is incredible useful to bring these perspectives together!

Usually, QAs know their way around today’s product in details and can point out weaknesses that can be fixed with new requirements as well as shortcuts when introducing new features. That is all incredible valuable for the BA. The outcome for the QA? If you consequently collaborate on analysis and story writing (to some extent, it’s not necessary to type the story together) you can think about weird edge cases and scenarios already long before the story is implemented and discuss the desired behavior with the BA right away. Baking quality in can be so timesaving!

Again, you cannot make up all possibilities in your head, you will still need to conduct some creative, exploratory testing. But by working with BAs at the beginning you reduced this effort by 60, 70 or sometimes 80%. Your work load after development decreases, because quality already increases before the software is developed. Well done!

How to get involved earlier in the software development life cycle: be involved!

December 23, 2017June 5, 2019 / Finn Lorbeer / Leave a comment

Once the process helps us to focus on fewer things and enables us to collaborate and test as a team, I as a “tester” will have more time. And I should spend that time where the product is actually baked: I should get more involved with the developers.

Many QAs are a bit hesitant to move somewhere close to the actual application code. But they shouldn’t be: our main contribution is not that we can code. There are other people who are much more specialised in. We call them developer. They are much better coders. And as such, much better suited to write automated tests. What is then left for a QA?

We bring very strong analytical skills. One of them can analyse all tests on all levels from a strategic point of view. What is the share of unit- integration and end-to-end tests? Where do we cover what business logic, where do we have gaps in our test coverage.

Many QAs who specialize in Front-End-Test-Automation write lots and lots of End-To-End / User-Journey tests. These tests are typically the most flaky, hard to maintain and cost intensive tests you can imagine. Hence, we usually advocate to add as few of them as late as possible.

Screen Shot 2019-02-15 at 20.09.11.png — Picture: 100 End-2-End test to rule them all

Instead QAs should aim to understand the big picture of the system architecture: what services do we have? How are they connected? How are they split? Does each service have a distinct capability? If so what is it and how does it relate to you business?

Once you figured that out you can assess how to test each of these services or domains in your service independently. If this works have a look at the communication between the domains and ensure this. If all of this is covered you may want to add a slight hint of an end-to-end test on top.

Obviously, the second approach is much more difficult – but that is what you are for and what you contribute in the team? You are the one to keep the big picture and consult your team members where to add what test in what way. What is the key assertion in a given test case? What is the right level for it? With a strong coder and your analytical abilities in testing you can ensure that things are working while they are implemented. That does not only improve quality early on, it also significantly decreases the time you need to spend testing (and reporting defects) afterwards.

Still, some defects will be released. No matter how much money you invest, there is no way to ensure bug free software at all (even if you are the NASA and spend more than 320 Million $ on a project). The second thing you can figure out with your dev team is how to identify and catch them. With the lean process (see above) that you established you can be sure to ship a potential fix very fast. The way to detect them is a helpful monitoring setup. This involves to visualise the amount of errors, as much as server/database request (and possibly a deviation to 24hours before). If you go to real professional levels you want to think about anomaly detection so that your system can notify you on its own once something is off.

The last open question is how to react to breaking changes that you may have accidentally released to production. We are running a two fold strategy here. We try to minimize our time-to-market for bug fixes and mitigate risks of other larger issues with feature toggles. Let me go into some details:

Usually, in a classic release management process you have a plan how to do your releases and if there are major problems afterwards you execute a predefined rollback. If this is – for one reason or the other – not possible there is usually a hot-fix-release-branch-deploy process defined by someone somewhere. Here is the problem: If you need a hot fix then your team is probably already on fire. In this very moment you need to follow a process that most people are unfamiliar with which usually bypasses a lot of security measures you have previously established in your release cycle. That is quite a bad habit concerning the production problems one has in this very moment.

Our goal is to make the fasted possible release process our standard process. Thus we drive our teams to deploy every single commit to production. That also means that every commit has to be shippable – with enough tests to make sure our product is still working. This is baking quality in already!

Screen Shot 2019-02-15 at 20.15.03.png — wikimedia.org

Still, things will break. But with a quick way to react and deploy a fix we do not even need rollback strategies any more. But being able to deploy a hot fix very quickly implies that you can also quickly analyse the root cause. But that is not always true. If you know what commit was faulty you can of course deploy a revert. But sometimes a new feature in it’s complexity across stories and commits is just not working right. Thus, we work a lot with feature toggles, making sure that all new functionalities are toggled off in production. We also make sure that we can toggle those features independent of deployments. Thus, we decouple the technical risk of a deploy with the business risk of a feature toggle. This reduces our needs for reverts by about 90% and most deployments run automatically and without problems. Every few days a new feature is toggled on. People are then usually more aware of the situation and monitor the apps behaviour more closely (at least they should). Problems can then be identified and either quickly be fixed with a tiny commit or, if you encounter major blockers, you toggle the feature off again.

In conclusion, we have way fewer way less troublesome releases, while we can activate new features in a very controlled way. Thus, we do not only deliver value fast, we also achieve higher quality at the same time. However, a very experienced and disciplined team is needed to work on such a high level of quality commit by commit.

How changes to your process increase the quality of your software.

December 23, 2017February 15, 2019 / Finn Lorbeer / Leave a comment

When we have a close look at the process steps of a typical agile software delivery team, we will realize what is going on there.

In the first one, “in Analysis”, we plan to create value. Once we have a plan how to add value for a user to our product we but this story into “Ready for Dev”. And here we are doing literally nothing, or, in other words: wasting time. In the best case, the story is still good to play – in the best case! Often stories are a bit outdated already before they are picked up and if they were lying around a bit too long they may even be completely degraded. In the next column “in dev” we are actually adding the value to the product and in the next “Ready for… “ process step we are – you guessed it – wasting time too. Until someone can check for the planned value and finally deliver the value. If we find a bug while testing, the deliverable software is in the state of analysis all over again – the bug is prioritized against other features and maybe fixed and maybe not.

As QAs we have little influence over the “Ready for Dev” column, but the “Ready for QA” is ours. Removing this column can have some positive impact on the velocity and quality if you pair it with another tool: work in progress (WIP) limits.

The idea is the following: if there is no “Ready for QA” column there is no place for a developer to just drop a ticket/story before picking the next piece of work. Unless the story was put directly into “in QA” (without anyone actually working on it). If the team then agrees on WIP-Limits, one could argue that a single QA person can work on one or max two stories at the same time. Thus, if there are already two stories “in QA” another one cannot be added. Thus, the story cannot move, thus the devs cannot start a new one (the “in dev” column should also have a WIP). The best way is to help the QA do the job and thus everyone gets more involved into testing – a big win for quality.

This will decrease the time to market for new features drastically. As a plus, bug-fixes can be delivered faster, too. Even in the standard process.

We applied these measure on different projects in different context. In one case, we were able to decrease the cycle time (= time from “Analysis” to “Done”) for stories from 13 days to 4 days – without anyone having to work “harder”. The restrictive WIP limits have a very good effect: if you can’t have so many stories in “in dev” you do not have to perform any context switches. You focus on getting one thing done before the next. Surprise: that actually helps to get things done! And being more focused on one specific task leads to fewer mistakes, thus fewer defects.

In another example where we applied the same technique, we were able to increase our velocity by a bit more than 30% (!) without any impact on the quality of the software.

These are some techniques to truly bake chocolate into the muffin: reduce waste in order to increase your focus on the really important things. Spend your available time on the urgent matters and “suddenly” you end up with a higher quality product.

Dimensions of Quality

December 23, 2017June 5, 2019 / Finn Lorbeer / Leave a comment

One year ago I introduced the “muffin concept” in a small blog post, “how we do Quality at ThoughtWorks”. Ever since then I have been to various Meetups and Conferences to discuss the idea and all the concepts behind it. After another year and dozens of discussion, there are more thoughts around how to bake quality in. It covers quite different aspects, thus I decided to split this post into a miniseries of 5 posts.

Enjoy the read and please give me feedback: tell me what you think!

People say that quality is like the chocolate on a muffin. Is it? Let’s say the product we build was indeed a muffin. The business analyst brought the recipes, and the developers baked it. Afterwards, the testers put chocolate on top.

If I imagine the muffin, it’s still a bit dull. The muffin is only perceived to be of really high quality if there are some chocolate chunks on top: like testing software.

The only problem is that – just like testing in the software delivery process – the chocolate is only “applied” after baking the major part of the product. It looks good and smells good. But does a muffin with a very few chocolate chunks only on top really taste better?

No, because there is no chocolate inside the muffin, just like testing does not improve the quality of a software:

When you test software you basically analyse a (hopefully) isolated system in a controlled environment. And no matter what you do, that system does not change. You may find behaviours in the system that are unexpected (which are the bugs / defects we are trying to find). But they were in the system already (before you started your test case) and they will be in there afterwards. No system under test does ever change its state (exception: quantum mechanics). Thus, the system does not evolve or improve (in quality) while you test it. Yet, another cycle of development in necessary to actually improve quality.

But that is quite sad. I am a quality analyst. An enthusiast. Caring about the quality of my product is my job description. Usually I am the team member most passionate about it. How can I be the only one who is not able to actually improve the quality?

With this mini-series of blog posts we want to investigate how we can be involved to improve quality in software early on – how to bake chocolate into the muffin!

Usually, a typical day in the life of a tester may look like this, where you pick a new build, deploy it to a test server, run smoke test and your extensive test suite. Possibly its (partly) automated. When no blockers are found one would monitor the production environment, ensure everything is healthy and announce & ship the build to production. Maybe you have a test suite running in production to ensure your delivery there:

However, that is only the last bit of a longer process. Normal, agile software delivery teams have a process that looks similar to the following one:

Each column is often reflected in tools like Mingle, Trello or Jira: “in analysis” is the step where Product Managers or Business Analysts work out the requirements for the projects. Once they are done they move into the next column. That could be a planning meeting where a sprint backlog is filled. We call the backlog “Ready for Dev” column. At some point devs pick up a story, works on it, finish it and put it into “Ready for QA” until a QA picks it up, works on it and ships it. Then a story is finally done.

If a defect is found in the QA work in the best case the ticket needs to go back to the devs or all the way back to in analysis. With these long feedback loops it can take a while until all kinks are out of a new piece of functionality.

Here we want to tighten the feedback loop and get involved earlier. Here is exactly the point where we can improve quality early on and where we can measure it. We identified four different fields where we are usually involved and where we have an actual impact on the quality of our product. You can read about each one of them in an individual (small) post:

Those four points is our recipe to bake quality in: You add some chocolate early on by process improvements. Then we add some technical strawberries along with the right amount of cream in the business space. We finish it off with some colourful sugar toppings in the team culture and voila… we really bake quality in!

With this holistic approach, we also step beyond being pure “Quality Analysts”. We still analyze the quality of software. But we also specialize on so many more things that lead to a better product. Thus, we truly are Product Quality Specialists.

“ThoughtWorks Presents” – Meetup in Hamburg

March 28, 2017February 15, 2019 / Finn Lorbeer / Leave a comment

On Tuesday, March 28th I was invited to Hamburg. ThoughtWorks has a long running series of Meetups in our Hamburg office. I had the honor to be the next person to present there.

For this occasion we talked about how to build a high quality product and what the differences and implications are to talking “only” about a high quality software. We discussed what ways we can think of to improve quality and measure quality and what we should be looking at other than business requirements and bugs in a software.

It was a very nice evening with lots of participants. We had many interesting questions that led to even more interesting discussions. I am looking forward to the next time already. Not only to see old colleges in Hamburg but to continue with all the talks and thoughts.

Link to slides (PDF, opens in new tab): building a high quality product blog

Meetup in Cologne: “ThoughtWorks presents”

March 27, 2017February 15, 2019 / Finn Lorbeer / Leave a comment

Here are the slides from the ThoughtWorks Meetup in Cologne on March, 27th. We had a very interesting discussion around the fields where QAs can be active apart from testing. We also thought about how to find the time for those fields when a lot of manual testing is needed in a project where a lot of changes are not automatically tested.

Link to slides (PDF, opens in new tab): levelup-meetup

How we do “Quality” at ThoughtWorks Germany

December 23, 2016June 5, 2019 / Finn Lorbeer / Leave a comment

Thanks to the other QAs in ThoughtWorks Germany for contributing thoughts to this topic over the past months: Sarah, @DizMario, @Nadineheidrich and @bratlingQA

What is testing?

Testing is a method to analyze the quality of a given software. It is a method that is applied after the software is developed. If the software has an insufficient level of quality, yet another cycle of development & testing is needed to increase and measure the quality again.

Testing is bug detection.

Metaphor: Testing is like putting chocolade on a muffin, but exclusively after baking it.

What is QA?

There are other methods that can improve / increase the quality of a software while it is developed. Those methods decrease the amount of cycles of development & testing that are required to reach a certain level of quality.

QA is bug prevention.

10797887872_IMG_1353 — We want to consider chocolade while baking and only put some additional on top.

How we are testing

In addition to the tools and processes that allow us to build high quality software from the first line of code (see below), we have the highest standards for testing software. As mentioned before, we are well aware that testing can only analyze a software’s current state and show the presence of issues/defects. Another cycle of development is needed to actually improve the quality. That leads to the known problem: whenever the testing is “successful”, the cycle time of stories increases and the delivery of a new feature needs to be postponed.

To minimize this delay, we apply the most efficient testing approaches to provide fast feedback for developers. This reduces the overall time to market of new features. With these methods we are able to reduce the cycle times significantly while improving the overall quality of the software in different projects:

Our tests are fast and effective. We run Unit-, Integration and End-to-End tests in a well shaped testing pyramid. This allows us to quickly check if our application behaves as expected. Writing the right tests on the right level reduces the time we need for regression tests from days to minutes. This includes – amongst other things – a 100% automation of regression tests.
Of all the tests in the pyramid, we take special care of the integration tests (of different services) to assure the architecture’s resilience. One of our favorites are the consumer driven contract tests. They allow different teams to work independent with loosely coupled services while ensuring that the entire system behaves well altogether.
For us, testing is an integrated activity within the software development team and not an independent, separate discipline. There are two ways to get a story tested:
1. When a story needs QA attention, you move the ticket into a “QA” or “ready for QA” column. The person who is in the role of the Quality Analyst then picks up the story as soon as possible.
  (this is push & role-focused ⇒ that is the Scrum-way with experts in the team)
2. When a story needs QA attention, you look out for the capability in the team. Any person with some capabilities in testing (often but not always the QA) rotates into the story.
  (this is pull & capability-focused ⇒ that is the Kanban-way with cross-functional people in the team)
  Guess what. I prefer the 2nd approach. The 2nd one is real team-play. And it decreases the cycle time of a single story and thus increases your velocity! Devs will learn (more) about testing and QAs can pair on the programming part, eg. to sort unit- and integration tests into the pyramid ⇒ baking the chocolade inside.
For exploratory testing we do not always apply the same standard methods but acknowledge the individual context we are in. Only then can we make use of the various advantages of different test methods. We find all kinds of tools in our box: Behaviour Driven Testing, Acceptance Testing, End-To-End Testing, Scenario based Testing, User Journey Testing, Integration Testing, System Testing, Risk based Testing, Penetration (Security) Testing, UX testing, performance Testing, Guerrilla Testing…

How we do QA

As mentioned before, we really need to learn all about testing. And its a mastery to study. However, its only a (small) part of our job and every day live. Besides testing we look into other things, as we know that good software is only the first step towards a high quality product.

We create a culture in the team, where the aspect of quality is an important asset for each team member. In this context we can address the team’s current needs with a wide set of processes, frameworks and tools that we as ThoughtWorkers already use or create if they do not yet exist (e.g. Selenium).

We acknowledge that we cannot build defect free software. Hence, we focus on defect prevention and establishing an overall quality mindset. We assure a high level of quality in software through tools and processes that allow us to prevent defects and find errors fast:

Well designed services to ensure a resilient architecture. There are so many things to work on if you want to improve the resilience. You can have the best software without bugs. It wont help you if your servers are down for the bigger part of the day. From a QA point of view, we are interested in circuit breaker (self-healing systems), feature toggles, well designed APIs and a kick-ass monitoring:
Monitoring! This is so important. And so many people think that “only” Ops should care. What a misunderstanding. Constant monitoring of all services and environments is a shared discipline to be able to react quickly on any arising issue. No matter how good you test (see above), some defects will slip through to production. The best way to reduce their impact is a combination of a good monitoring and quick deployment. If we are able to release a fix fast (= best case: 20 min after a bug is found), we can reduce the impact significantly. This is what monitoring is for. Learn more in this podcast.

We have a strong focus on Continuous Integration and Continuous Deployments best practices, such as fully automated regression testing. You can read all about it in the other post as well as talk in Ljubljana.
Test Driven Development for high test coverage and fast feedback during development is also an important thing to notice. While most developers know that a tests are written first in TDD, not all know about the testing pyramid, much less of its benefits. Hence, a pairing of QA and dev while practicing TDD can be of high value. This is how you bake quality in!

I just mentioned the consistent pair programming. Pairing allows best designs and fewer defects from the beginning. Make sure to rotate frequently and across the roles. Pairing is a general activity for most tasks in a team. Pair-programming is just one of them.
We love Feature Toggles™. They give us maximal control over the features in production and canary releases. A very easy method I usually use is to give chocolate while the standup to the pair that implemented a toggle the previous day. This is a fun way to talk about it, remember it and give a sweet incentive to build it in when it was forgotten. If you find the time to use them it will make your lives much easier. You will not need rollback strategies any more and it is a very, very good safety net. Quality “assurance” at its best!

The mix for the win.

Of course we combine the two aspects of QA and testing. And this is the biggest challenge for us. Where to focus at what point of time. Where do we need more attention and what part of the application / system / team is running smooth? Ultimately, we try to pick the right tools and create the right mindset to build a high quality product.