You often hear within agile teams that creating a plan or strategy, to lay down the foundations for testing our products, is a waste of time? The focus of this article is to explore whether the Agile Manifesto principles are preventing us from creating test plans and strategies. Let’s explore some ideas related to this.

There are many opinions regarding this topic within the community.

Some experts affirm that based on acceptance criteria for each user story and a general understanding between the different team members; there is no need to create a test plan.  That testing and quality just…happen.

On the other hand, other experts affirm that some test strategy development and planning must be performed to specify the foundations and agreements regarding the product quality and the software testing approach.  Even in “Agile” projects.

My view

I am no “industry expert”, but I will try to share my experience surrounding this debate.

There are a myriad of different types of software projects including: maintenance, improvements, migrations, new developments, mixed projects, etc. But from my perspective, a consistent need across all of them is establishing a foundation for the basics of the quality assurance and testing approach. It certainly helps in meeting the Definition-of-Done, while also establishing the level of required product quality we are committing to deliver.

Having this approach defined not only helps establish a clear understanding on the ways the software will be tested, it also determines the customer’s expectations regarding quality. It also helps define how the different team members will contribute to product quality, for example, which are the key indicators / tools we will use to analyze our testing effort and product quality.

Test Strategy in the Agile World

It’s probably important to clarify my concept of test strategy in the agile world. I believe a test strategy is a living lightweight document, created in collaboration with the whole team and customer or stakeholders. It serves as a guide for ALL workflows (Defect, Epic, Feature, Release, and Sprint) through your teams and can be changed or adjusted as needed. It typically includes the following things; many of which are optional depending on the maturity level of the team, the customer needs, and the software and business domains:

• Scope of the testing effort
• Solution under test
• Product risks
• Test items
• Levels of testing
• Manual testing approach
• Automated testing approach
• Regression testing approach and scope. Existing testware
• Test Environments
• Code Drops, Pushes to different environments, build frequency agreement, etc.
• How should we test the set of features within scope
• How to define bug severity
• Metrics to be collected / defect
• Reporting
• Cause Analysis keys (for Retrospective meeting)

By reading through the topics in the list above, you might believe this to be a large document, with a lot of complexity, but I want to stress again that this must be a living, lightweight document used as a “guide” and not a “plan”. My intent here is to get you thinking about what you’re teams need in their testing plans for your contexts.

Who’s the audience of this document

It is NOT only for the testers and the reporting manager. It’s actually intended to drive quality thinking and testing activity across the entire agile team. The real intent is that it is written by and owned by the team and serves as a collaboration point for whole-team quality agreements.

How much should we chew

There’s another important focus for creating your test strategy. Do not plan too far in advance because things can dramatically change in a rather small period of time. Only focus on strategizing / planning those things clearly in front of you or those features that belong to your current release and that might span through a maximum of 3-4 future sprints.

How strict should it be

Another key aspect of the strategy is the adjustments that it drives. The entire team leverages it as their baseline understanding of testing and quality for a given set of sprints or the current release. If something changes, the team changes the strategy, without a great deal of formality or fanfare. In essence, it provides a framework for making changes and adjustments as the team executes towards their project goals. It serves as the focal point for these – change or adjustment discussions.

The software products that we build are, by nature, incremental and evolving. We need to have that in mind when creating the test strategy. We have to try to define how are we going to deal with functional increments, how to approach regression testing, how much unit testing we should do, and how detailed the test cases will be so they don’t become obsolete too soon. For all of the previous, we need to have a flexible approach that can adapt to frequent change without a lot of bureaucracy.

Are the Customer & Stakeholders involved in this process?

In a word – Yes!

Your customer participation and collaboration in the creation, maintenance and update of the test strategy is key for the success of the selected quality approaches.

Following a risk-based approach is one of the key success factors for testing in agile centric projects. Work with you customer to identify the product risks and prioritize them. Ask the customer for input / feedback on the different suggested testing approaches, make the customer aware of the implications of each approach and improve based on iterative and ongoing discussions.

Also, make sure you ask for input and feedback frequently during the project. Showing the results of the selected approaches, in whichever way you and your customer feel comfortable (test execution metrics, defect metrics, story related metrics, delivery metrics, etc.) is a good way to show how the selected approach is working. It will also be the baseline you and your customer can use to change, update, or improve your test strategy for future sprints & releases.

Let’s see how my recommendation align with the Agile Manifesto

Individual and Interactions over processes and tools

The testing strategy is created by the team and is the result of their interactions.. It just tries to lay down a set of foundational agreements for the whole team regarding quality and testing.

Working software over comprehensive documentation

As you see the testing strategy is not comprehensive documentation. It is just a guide, an outline of what is to be done and how it should be done. It will help you make sure the software is working and working as expected.

Customer collaboration over contract negotiation

We need to view all the agile teams plans, including the test strategy and plan, as living things. As we discover things daily, we make appropriate adjustments based on team and customer collaboration.

Responding to change over following a plan

We are not setting a fixed plan, a day-by-day plan with specific milestones and resource assignation and leveling. Instead this is a roadmap or guide for achieving the team and customers goals. It allows the team to be flexible and adapt on the fly, but also serves as a baseline for discussion.

Wrapping Up

I hope this article made you think about the importance of planning within your agile teams, in this particular case, test planning towards quality and testing strategies. Even though we’re “Agile”, I think planning is an incredibly important task within agile teams.

And remember, it’s not the plans that are important. It’s the act of PLANNING as a TEAM that makes all the difference.

Oscar.