“All we have to decide is what to do with the time that is given to us.” – J. R. R. Tolkien, The Fellowship of the Ring
We can’t test everything. It’s impossible. There are too many test cases, too many actions we could potentially perform, too many edge cases and things that our users will come up with that we can’t even imagine. At any given point of time, there are an infinite number of tasks we could perform. We need to choose our tasks to fit into the time we have available for testing, planning our tasks in advance to meet our deadlines.
It gets even worse for many testers – even once they’ve identified the tasks they feel are worth executing in the allotted time, the available time often gets shortened by schedule slips without final deadline movement. This leaves us unable to even perform all the testing that was initially planned! In short, every tester on the planet has many more things they COULD do than they’ll ever have time to actually perform, even if we only consider the tasks that tester might want to do.
Because of this massive number of tasks to do and this small slice of time to do it, it’s incredibly important that each of us carefully chooses the tasks we do so that we can be as effective as possible. We need to choose the actions that provide the most value in the time we have and explicitly avoid the actions that don’t have value.
What is the value of an action though? To know that, we need to look at the bigger picture – why are we doing ANY testing activities in the first place? There are many answers to this, but the one that I like the most is that we’re testing in order to provide information to the project team about the quality of the software system we’re testing. If we use this as our overarching goal, we can then define the value of an action based on the information we will obtain by performing that action (and subsequently processing any data it provides). There are several aspects of the information that matter – the amount of information provided, the uniqueness of that information, and the relevance of the information to our needs all play a part in how valuable we perceive the information as being. These 3 aspects intertwine – an action might provide a lot of information, but if much of it is information we already have, or information we don’t have but also can’t use, there is little value in the action. The most valuable action is then the one that gives us the most new information that we need to make decisions.
However, there is also a negative cost to performing an action. The time we have to perform our testing activities is finite, and even the best multitaskers among us can only do a few things simultaneously while maintaining focus. Every time we decide to perform an action, there are other actions that we’re simultaneously deciding to not perform in that same period of time. This may not be a conscious decision, but it’s inherent in our choice of what to do. For some of these rejected actions, we’ll be able to return to them later in the process as they become the most valuable remaining actions. Others, however, we may never get the chance to perform due to time constraints and the priorities of our testing efforts. Each of these other actions would also provide information of some value if we chose to perform it, but if we don’t have the capacity to perform the action (or simply choose not to perform it), we no longer get that value.
Thus we have two pieces of information to weigh as we decide which actions to perform – we want to maximize the value of the information we’ll get from the investment of time without incurring too much of a penalty due to not getting the information from the actions that we’ll no longer be able to perform. We want to pick the action that gives us the most valuable information without preventing actions that would give us more valuable information. This requires conscious thought as we evaluate what to spend our time on. Sometimes this may mean picking smaller tasks which give us high-value information rather than picking a large task which subsequently prevents many smaller tasks. Sometimes that large task may provide information sufficient to justify performing the task. As we’re planning our test efforts, we need to consider both the positive value of the information we receive and the negative cost of the actions we consider to determine the most effective course of action. By doing so, we can maximize the time we do get (even if it subsequently changes) and deliver the most useful information we can to our teams.
What is the value of your actions?