When are you done testing? It’s question that has both baffled and annoyed those in the software testing community for all time. Many teams rely on qualitative judgement, some on quantitative metrics and some a bit of both.
Code coverage is a quantitative metric that in most instances is pretty easy to understand. Line coverage answers what percentage of lines of code in the software-under-test has been exercised by testing. Branch coverage similarly answers whether Have you tested the software so that every decision point in the software-under-test has been exercised.
More advanced coverage concepts such as Change Risk Anti-Patterns get a little more complex but all of these metrics are intended to guide teams to understand their risk and where they should be concentrating their quality efforts.
There are many tools out there which work really well at assessing code coverage but they tend to be used only for unit- or integration-level tests. Those tests are often easy to automatically capture metrics and integrate with other tools. Testing at the UI-level though, either manually or using an automated tool such as CodeFuse is often left as a separate activity. But it doesn’t need to be!
Certain tools will allow you to start capturing code coverage statistics while you execute UI-level tests. Two of them are below. Once you have them installed and started, you just need to execute your manual or automated UI- tests and collect the reports.
.NET and nCover
nCover is a long established code coverage tool in the .NET marketplace. Assuming you are testing a web application using IIS, it is possible to set up nCover so that it records the coverage statistics for your UI testing sessions. This nCover blog post goes into detail. You can also see a video here of nCover being deployed to monitor an IIS server hosting a web application.
Java and SonarQube
If you’re using Java then nCover isn’t going to be much use to you. Thankfully, since SonarQube integrated JoCoCo you have a solution. This post covers off everything you need to set up SonarQube so it’s ready to capture your UI-testing coverage stats.