Software Products need their own unique approach to test adequately and correctly. Often times, teams treat them as any other software (i.e. internal applications built for a specific client or team; not accessible by the general public; non-revenue generating) and that is the starting point of trouble.
Software Product Testing needs a custom test style and strategy to add value. Software Product development and sustenance is in itself a complex ecosystem and to thrive testers need to adapt.
Software Product development challenges:
Here are some of the challenges that Software Product development teams face:
#1) Lack of control over user demographics, devices, environments, platforms, etc.: Software products unlike software built for specific stakeholders are not used in controlled and predictable situations. There are many just too many factors to take into account.
#2) Foggy product vision: Product behavior and features are forever changing and the journey to maturity isn’t clearly visible. Or the product is growing too rapidly that it spirals out of control that teams don’t know what is happening.
#3) Aggressive timelines: Due to heavy competition in the software product market, things have to move at a breakneck speed and teams must stay a step ahead of their peers. Otherwise, they are sure to lose out to the competition.
#4) Fear of failure: Software products are usually innovative. So, their success is not always a given. This is the reason companies can’t go all out in terms of budget, technologies, infrastructure, etc. They often have to hold back to gain a certain amount of immunity from failure or to even breakeven.
#5) Lack of actionable feedback: Since there are no stakeholders or business users or clients, so to say, it is difficult to understand what the end user may or may not like. Companies are constantly playing a guessing game and often have difficulty in bridging the gap between what they want for the software and what the customer wants.
These challenges affect all areas of product development, marketing, and sustenance- And they inherently impact product testing too.
To get ahead in the game, this type of testing has to take five key points into account:
- Speed of development and releases
- Short term and Long term product goals of the product
- Extent and nature of competition
- Target audiences and their environments
- Requirements – Functional, performance, security, usability, configuration, etc.
Before we go into more details, let’s understand product life cycle (This is a generic product lifecycle and not specific to software products but software follows a similar pattern):
A good Product Test Strategy/Approach should take into consideration the current stage of the product in its life cycle.
Example: A company XYZ’s product is a defect tracking software called ‘TrackFast’. It is a new product and the first version is set to be launched as a cloud and on-premise solution. TrackFast works like any other defect management system and is built for both Mobile and Web access. Currently, there are 2 to 4-week sprints at which the product is created in parts. You are on the testing team testing ‘TrackFast’ before it meets its customers. The testing involves checking functionality, performance, and security.
To summarize, these are the parameters you are working with. Or if you prefer, this is your context
Let’s see how to test at each stage. This is product test process, method, or life-cycle at each stage.
Stage #1) Product Introduction
Since this is the first time TrackFast would be going out into the market, the idea is to make a good first impression. So leave no stone unturned. Test everything and from every angle. In addition to that, lay the foundation for future testing.
A good test strategy at this point should include the following:
- Tests that validate the short term goals of TrackFast. “What does it need to be shipped correctly” should be at the forefront of the testing effort. Create End to end tests (front end, middleware, and backend) for thorough testing of every feature
- Tests that compare TrackFast with the competition (ideally this is the job of product owners but as a tester we can add our two cents. Also, this step is easier if the software has some peers already. For example: It is easy to compare TrackFast with Bugzilla or JIRA or other legacy systems. But let us say I am creating an app that does something unusual like being able to predict when a baby is hungry or cranky :), it might be hard to find an application that you can use as a baseline)
- Platform, browser and device compatibility tests
- Tests for ease of installation, set up and getting-up-to-speed
- Tests for performance, security, and usability
- Integration Tests if it interfaces with other systems. A simple integration example is that Defect tracking systems often interact with email clients to send notifications
- Plan for regression– It is a good idea to flag or mark critical tests that you think will be a part of future regression cycles and think about automating them for future releases
- Plan for known issues (are you going to be adding them to the backlog or handling them as CRs, etc.)
- Flexibility to change when the product progresses to the next life cycle stage.
It could sometimes be a long wait before the product goes out, so use all the time you have to do as thorough a job as possible.