This is part two of a two-part series. If you have not done so, read Part 1. Achieving excellence in continuous testing is not just about mastering all the new tools, programming languages, and frameworks. It involves developing a deep understanding of the product you are testing. What follows are some additional tips that can help.
Leverage Product Logs
At any given point in time, your product's current state can only be found in the logs and not from the front end. Logs mirror the behavior of your product. As a result, they are indispensable in troubleshooting and analyzing defects.
A good product log is as useful to a tester as the UI is to the customer. The product's testability suffers, as does its quality when the system isn’t logging sufficiently. So, make sure this gets addressed sooner, rather than later, in the development life cycle.
Use Configurations to Identify Defects
Have you used the exposed application configuration parameters across the different sets of values they support? Have you done any product integration? Are you able to configure your product in different modes?
A product can have different modes in different dimensions (e.g. SAML or SSO, single or multi-tenant, on-prem or saas, HTTP or HTTPS). Every product has several such configurations, and you gain key knowledge when you configure it in different modes. I once found several defects just when I switched from single-tenant to multi-tenant mode.
Read Between the APIs
APIs are the doors to the product's internal state, and you can get a good insight into it by putting them to use. APIs can tell much more than what the UI and logs tell you.
With APIs, you are talking straight to the product with no intermediary front end. As a software tester, I’m sure you must have done API testing. Does your API test framework cater to test individual APIs or as a complete system having end-to-end workflow? Are you able to unleash the power that comes with these APIs and use it to your advantage to test the product, both at the functional and non-functional level?
Correlate Database Values with the UI Data
Have you ever thought to correlate the values on the UI with the back end? Can you identify which cell of which table and which database is showing up on any particular UI?
You can quickly understand any database-related errors by getting to know those intricate details, e.g. when there is any change in schema or any error caused by long values. The same is true for any other data storage like non-relational database systems (e.g., Elasticsearch, Google Cloud Search, etc.). Get your hands dirty – know the low-level details about data storage and access, and I bet you could be a good data simulator. We will discuss this more later.
Understand Your App's Ecosystem
Do you know which system files are used for a particular configuration of your product? What about files that may be or act as a security certificate? Or files that are special utilities that aid you in your work (e.g., a traffic analyzer, an internal bus reader, etc.)?Can you use these utilities or configuration files to get the product to work the way you want?
The more you know this, the better you understand the product and the more confidence you will have. The objective here is to learn about the fundamental files and ancillary utilities that come with the product.
Make Dummy Data as Authentic as Possible
When testing a system, you always want the product to have data that is as close to reality as possible. However, this is not always possible, so generating and ingesting dummy data becomes the only option.
Dummy data should be as authentic as possible to ensure that your app's data consumption processes provide realistic results - even erroneous ones. Testing your product's behavior against realistic data (good and bad) not only ensures good customer experiences but helps confirm the safety of your product should it come in contact with malicious data attacks.
Know Your App’s “Hacks”
Some features are hidden from the customer for troubleshooting or other purposes, and you may have to befriend a developer or a support technician to learn these secret treasures. Educating yourself concerning these hidden features and capabilities will help you be a better-informed tester who can in turn give more enriched information to the developer to aid them in correcting a defect.
I once worked on a product in which, upon hitting a particular key combination, it would display the state of the communication bus and the packets and messages flowing through it. Similarly, there was a mobile application that I tested in my previous organization. On a long tap on a button on this app, it would display the diagnostics with statistics figures on the health of network connectivity, connection to the application server, versioning, etc.
Test with the Pareto Principle in Mind
If you thoroughly know your product's core and non-core functionality, your test strategy will follow the 80-20 rule. The Pareto principle, in this context, means understanding that your customers will likely only use 80% of a product's core features. Understanding this is extremely important and will help you focus and prioritize your efforts on the right set of components.
The law of constraints dictates that it’s impossible to test every core and non-core feature. We have to make decisions that are justified with reasonable customer usage predictions and align with business objectives.
Know Thyself, Know Thy “Enemy”
While it is important to know what our product does, it’s important to understand the industry, domain, and value your product delivers. This provides a richer context to your test strategy and aligns your efforts with a begin-with-the-end-in-mind mentality. You will have an edge if you become familiar with similar products (including competitors').
In Conclusion
After you fully understand the above points, apply them as appropriate to the underlying situation. You may then want to come back after three months and evaluate yourself.
Just ask yourself the questions below. If you answer “yes”, to most of these, then you are on the right track.
Am I able to:
- Handle any product query
- Configure a product in different modes
- Troubleshoot an issue - feature defect, infrastructure issue, system defect, wrong configuration, etc.
- Provide demos on product functionality
- Ingest dummy data by altering values in data sources or using APIs
- Find the right areas to test against changes of any functional area or component
- Sanity test the product on-demand within 2-4 hours (depending on the type and complexity of the product)
The path to learning a software product is a long and never-ever-ending process, and you should not limit it to a few weeks or months. As the product grows, keep yourself updated on the new features and functionalities it offers and that of similar and competitive products. Becoming a product expert will transform you into a strong advocate for your customers and the guardian of your product's business value.
Visit Broadcom’s Enterprise Software Academy to continue to build your competency in continuous testing.
