How to use test coverage?
The basic use of test coverage…
Most of the time, for functional test teams, test coverage is a control matrix that links risks, requirements and test scenarios.
As soon as scenarios match the requirements, the known risks and that they are successfully run, the coverage is validated and theoretically covers the risks…
Or strategic use
However, it is possible to approach test coverage from a new way. This way will give richer information that will be really useful for test improvement. It will allow testers to improve test efficiency.
Decrypting test coverage
What is that?
Test coverage is seeing the real coverage of tests in the application. Shortly, it’s analyzing what has been executed in the code by tests. We call it the “footprint” of tests in the software.
Global coverage
Globally on an application, it becomes possible to see accurately:
- What has been tested
- What hasn’t been tested
It will allow testers to make sure every single part of code is covered by a test scenario at least. But as testing all of the application is rarely possible regarding the short period of time allocated, we need more data…
Enrich the information
In order to exploit even better the test coverage knowledge, it is ideal to double check that tests are covering the changes that has been made in the software while validation stage. This will permit testers to better anticipate regression risks.
For doing this, unexpected changes must be identified first, and then the code has to be checked to reveal if weather or not it is covered by a scenario.
If some parts of the added code are seen as not covered, test teams can react very quickly and focus on these specific highly risked areas.
Of course, the closer is the deadline; the most important is the information. During the last test stages, it is absolutely impossible to re test all over again. It becomes a must to only select a few test cases and to make sure nothing has been missed.
And it’s the same scenario for minor versions or patches: allocated time and test load are making it impossible to re test all over. Identifying test scenarios impacted by changes allows focusing on relevant tests. This increases the team’s reactivity.
Case by case even more efficient
Individually looking the footprints gives an even finest details level.
With traditional test tools, test coverage is seen global: test execution paths in the application are not considered. Even test are not differentiated.
By capturing every scenario footprint individually, Kalistick brings a high value data to test teams.
When executing a test scenario, each and every executed action on the software is recorded. Thus, when a new version is received during a validation stage, if certain tests have already been executed, the system will detect which results can be compromised by the new version (when the footprint has changed).
If a choice has to be made between scenarios, or if you work with Risk Based Testing, Kalistick brings an ideal additional information. Based on each scenario’s footprint on code and on the application’s functional risks, the system shows which scenarios are to be run for each functional subset.
The “Test Learning System”
It is the name of the software that will help you improve your tests!
Here is how it works:
- When a test is played, it is recorded
- Every action executed on the application during the test is linked to the part(s) of used code
- The link between the test and the code is defined
- A new footprint is created
Thanks’ to its integration to HP Quality Center; the operation is very simple and automated. For each test execution, the process is automatically run and a new footprint is created. And in case the test has already been executed, the new footprint will be compared to the previous one; if changes happened, they are immediately identified.
Secured Go-live decision
Thanks to the Test Learning System, the Go decision is safe. Indeed, the real risks coverage is 100% known. If all tests are validated, then no unknown risk can impede the production.
