What is our focus? Delivering a whatever product in a fixed time slot? Or shouldn’t it rather be: deliver something really valuable to the customer? Long-term happy customers are key for the success of a company – not just accomplishing numbers or 5 year plans. That is why the quality of a product, as part of the attributes of a product is that important.
Our customers expect high quality all over the touch points in the customer live cycle. But what does quality mean? Do we have the same picture in mind? What is the impact of poor quality and how can we avoid it?
This article describes what quality can be and why it has to be in focus throughout the whole product development flow in order to satisfy our customers as well as reducing development cost and risks.
What is quality?
The word quality is derived from the Latin word qualitas, which means attribute or property. In the following quality is defined as the extent to which the attributes of a product or service comply with the expectations/requirements.
Quality has two meanings:
- Neutral: The sum of all attributes of an object, system or process (in the following text summarized as product)
- Valued: Fineness of all attributes of an object, system or process
In common speech quality is often used in the second way, as a synonym for fineness of a product. We often say that something has a poor or good quality. At least this stands for the matching level between expectations (requirements) and existing attributes of a product in reality (see an equal definition in ISO 9000:2005). You can transform the second, valued, meaning easily into the first, if you define, that even fineness is just an attribute composed of several other specific attributes and it is more or less a subjective valuation.
If we are talking about quality as specific attributes of a product, system or process, we have two different perspectives – the external and internal quality.
That means, that we have to pay attention to external quality as well as internal quality during development and have to deliver a product with a defined set of attributes.
All attributes which are perceptible by customers, influencing their experience and enjoyment of a product, contribute to the external quality. These attributes can include:
- robustness to error conditions, etc.
The external quality can be evaluated directly by the customer of a product. As such, attributes that are (not) as expected by the customer will quickly influence how the quality of a product is perceived.
All other attributes, which are not directly perceptible by the customer, determine a product’s internal quality. This means everything “behind the scenes” such as:
- modular design
- well-defined interfaces
- flexibility to accommodate new requirements
- adherence to agreed standards
- repeatability of build and test, etc.
We can’t afford to be complacent about internal quality issues. Although the customer can’t evaluate the internal quality directly, ignoring or delaying addressing them may impact the external quality, thereby affecting the customer experience and how a product is valued.
In addition, internal quality issues may not be experienced by external customers, but they may impact internal teams (e.g. development or operations) affecting their productivity as they are forced to work around issues.
Poor internal quality can arise from cutting corners for short-term benefit (e.g. a design hack to meet a deployment deadline) where “doing the right thing” would take longer/cost more. This is known as Technical Debt and, like all debt, these shortcuts attract “interest” until the corner-cutting is replaced with a quality development. For example it may be more difficult to implement new functionality leading to more Technical Debt).
Development teams should be encouraged to quantify and make visible their quality issues, including Technical Debt, to allow their correction to be planned alongside other work. Otherwise there is a danger of it becoming a vicious circle (pressure to deliver => take on technical debt => fail to pay back debt => decreased productivity => pressure to deliver => …).
Quality Issues or Bugs
From a technical view a bug is an error, failure or fault in a product, process or any kind of system. It causes incorrect or unexpected behaviour regarding to the specifications and acceptance criteria made before (external and internal). This sometimes differs from a customer perspective. The customer maybe expects a different behaviour or doesn’t understand a feature or even the whole product (external). In this context we often speak of „poor“ quality – a mismatch between expected and delivered attributes of a product. This mismatch can have impact on internal and external quality.
Aftereffects and cost of quality issues
Quality issues are at minimum something annoying for the customer (external quality) or the development team (internal quality). At maximum they cause non-maintainability or crashes of the whole system, or even personal injury. The cost for correcting those issues depends on one hand on the actual cost for correction and on the other hand on how big the impacts are and how expensive it would be to undo them.
There are several studies and papers where the cost growth, dependent on the development phase in which a bug is found, are described (see Error Cost Escalation Through the Project Life Cycle or Software Engineering Economics by Barry Boehm).
These studies show, that if you have an issue, the cost will increase exponentially the later you find it because of the increasing effort necessary for resolution. The more time passes by, the more time and money have to be invested in fixing it. That is because the risk is higher for other parts of your product to depend on the problematic part. Thus it is harder to identify the cause, analyse and fix it. Additionally the risk of destroying something during bug fixing, so-called side effects will increase the later you do it.
Typically these studies relate to a waterfall approach, but the conclusion holds true for Agile methodologies: the goal should be to prevent bugs, or identify and fix them as early as possible.
How to ensure a stable product quality
Quality isn’t something we can add to a product after building it. Instead it is an on-going process during the implementation. There are different activities needed to ensure that the matching level between requirements/expectations for a product and the delivered attributes of the product is as high as needed. The matching level is in a range between „no match“ and 100% match. According to the intended quality level, there are different possible actions with different effort. This includes a proper product development process as well as technical/implementation guidelines.
A suitable process
It is essential to accept, that not everything can be foreseen or anticipated when planning next steps, features or even whole products. To plan is to guess. That is why the underlying process itself has to fit.
The process needs to facilitate the actions needed to implement the required attributes – from understanding the problem space or vision over executing/developing and measuring results to continuous improvement of the process itself and the involved people.
It is crucial, that the customer gets involved as early as possible and on-going through the whole product development flow – for example in the product discovery or in early A/B tests.
Based on a fast feedback loop we are able to inspect and adapt and get closer to the customer and his needs and reduce the risk of producing a product with at least poor external quality.
Standards and guidelines
A good guidance could be a common „Level of Done“ with all required attributes that have to be met. For software development there is a range between no testing or quality assurance on one side and on the other side formal proving, which in theory covers at least the internal quality to 100%, like NICTA did it for a Microkernel.
It is essential that a company defines a minimum required Level of Done, that every team has to stick to. It is a trade-off between feasibility, cost and benefit. Over time this level should increase continuously.
Quality assurance is the sum of all activities, which ensure a stable product quality. As quality can be differentiated into external and internal, the quality assurance activities can be considered as external and internal, too.
There are methods and tools like mock-ups or prototyping that can be used from early stage over production phase up to live stage in the product development flow to validate ideas and plans. Other tools like A/B-testing or user testing will help to figure out if a product matches the expectations or requirements (external quality) in certain phases of the Flow.
Internal quality assurance includes the way a product is built as well as the methods used to prevent (like Test Driven Development or Behaviour Driven Development) or find (like Unit Tests in software development) deviations between requirements for a product and the implemented attributes. Common ways of working and standards are helpful to ensure that the product is build with the required attributes. For example in software development there are several guidelines for architecture, coding and testing as well as tools for automated testing.
Testing as part of quality assurance can be divided into two different approaches: manual and automated testing.
The number of tests, which can be done by a human being, is nearly the same per day. It depends on the complexity of the system and features – and of course there is also the human factor. The complexity of a system and therefore the effort for manual tests normally increase exponentially over time. The gap between increasing complexity and number of tests done per day will grow similar or even faster over time. This will slow down the productivity of a team very quickly, because bugs will be found late, features will be deployed late because of waiting for test etc.
Although the effort for automated testing is higher at the beginning, the use of automated tests will pay off very soon. The test coverage increases. The complexity of the system will move from the complex domain to complicated or even obvious domain as far as possible, because of the employees‘ better understanding of the problem space itself, the product, and how it is built (see Cynefin Complexity Model by Dave Snowden). The internal and of course the external quality will increase as well as the productivity of the team.
Quality as integral part of the product increment
Product development is an iterative process, which describes all activities needed to build a product – starting with an idea, „ending“ with the launch and restarting after measuring success and adapting the requirements.
At least every piece or increment has to be independent and has to have a value for the customer in a defined quality. That means, that quality is a part of every product increment and it is necessary to consider the actions and resources needed to implement it. As we want to make the production chain as efficient as possible, all actions/attributes needed for the intended quality are always in scope, because it is much more effort to add them later in the process (see above).
You can describe each iteration by transferring input resources over time to a feature/product increment (= output) through several functions (actions). At the beginning we have at least a clear vision of the output – a solution for a problem or a need to meet. We can describe this vision of the output with attributes we need to implement. Additionally we have constraints we need to keep, which can be described as attributes too. All these attributes, describing features and constraints, are the requirements for our product increment. At least these attributes describe the scope.
All the resources invested to achieve a goal – to implement the scope – can be declared as input. Input resources can be time, manpower, material or money etc. Theoretically you can convert money to every other resource. But in reality it isn’t that simple because of some constraints and special attributes of some resources like people and time.
Actions can be described as functions, which consume input resources and create an output. For example those functions as part of certain stages of the product development flow can be “understanding a domain”, “solving problems”, “planning”, “designing”, “coding”, “writing automatic tests”, “manual testing” etc. – in other words every action which is needed to create a product increment with defined attributes.
In short: The production chain consists of several actions, which consume input resources in every step and produces product increments as output with defined attributes. With every step the resources are consumed and there are less input resources for the next steps.
At every time the system/production chain is in one of these three states:
1. No input resources left, product increment ready (all attributes present)
2. Input resources left, product increment ready (all attributes present)
3. Needed input resources missing, product increment NOT ready (attributes missing)
It is important, that the feedback loop is part of the whole process. Maybe I could count it to the actions, but to make it more explicit I separated it in the picture above.
In conclusion there is a relation between input resources and output/scope. If fewer resources are available the scope has to be downscaled. If the scope increases, more input resources are needed.
In advance it is not possible to fix them both and just follow a plan. To make the intended deliveries manageable and to get more information about the needed input resources and the feasibility, it can help to split them in smaller pieces and make resources and scope/goal more transparent – divide and conquer and minimize risks. At least it is a trade-off between available input resources and the delivery of a given scope. If there are not enough resources available for a given scope, the production chain will end in state 3. Additionally there is always the factor of uncertainty and an on-going discovery of new information. That is why there is a need for continuous replanning. Always inspect and adapt.
As quality is the foundation for long-term success of a product, I suggest to never ever put quality into a discussion about target scope and available resources.
A predefined quality level has always to be achieved. That is why quality is separated from scope in the picture above. If there is a negotiation of scope or resources these both are in strong correlation but shouldn’t affect the defined quality level! Trying to increase the scope with fixed resources – quality will be a mess. The same happens, if you cut resources without slicing the scope.
The product has to meet the needs of the customer. That’s why the whole process must support this paradigm. We need fast feedback loops, to get closer to the customer and his needs in order to adjust our plans, hypothesis and at least our product – inspect and adapt.
To plan is to guess.
During the whole process costs for fixing quality issues will increase over time. That is why a production process, which ensures internal and external quality during the development, is required. Additionally high internal quality leads to a more efficient production and the foundation for future development. During production there is a set of actions including quality assurance necessary, which consume input resources as time, budget etc. and create an output – the product increment with specific attributes.
If input resources are limited it is logical that the output is limited, too. That is why it is important to always review the available resources and check, if they are sufficient for the expected output or scope.
During development, progress is uncertain until development is finished, because we don’t know what will happen in future. If unexpected or unplanned actions consume resources, this has a direct impact to the desired output.
It should never be an option to cancel quality assurance actions, just to hold the deadline.
In the long-term quality assurance during the whole production process saves a lot of money and it is the key for great products, happy customer and employees.