Code Coverage is important when testing software to investigate and determine how much of the programmers code is being coverage by the test cases. The code coverage amount is represented as a percentage. The Wikipedia definition is…
“In computer science, test coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs. A program with high test coverage, measured as a percentage, has had more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low test coverage.” Further information, click -> Wikipedia Code Coverage
Within the aviation spectrum it is important to develop software of the highest quality and to aid these there are many specifications, assurances, guidelines, etc that software developers are asked or must conform to; some of these include:
- EUROCAE ED-153 Guidelines for ANS Software Safety Assurance
- ICAO Doc 9906 Quality Assurance Manual for Flight Procedure Design: Volume 3 Flight Procedure Design Software Validation
- FAA 8260.58 Volume 1 General Guidance and Information – Attachment C Test cases
- EUROCONTROL Safety Assessment Methodology (SAM)
- EC Regulation No 482/2008 establishing a software safety assurance system to be implemented by air navigation service providers
Whilst developing the FPAssistant suite, it was key to develop the software with a comprehensive sets of tests cases that try to establish a very high code coverage percentage. Where possible test cases based on examples and data tables have been taken from the ICAO, FAA and EuroControl documents and added to the Unit Test suite. In Test Driven Development (TDD), test cases are added before the programming work starts to assist the developer in knowing that have the correct results. Below is a screen shot of part of the test hierarchy used in FPAssistant…
Once all the test cases have been run and hopefully they all pass (green circle with white tick – see above) then a software development tool can work out how much of the test cases will execute the the source code that has been written; in this examples shown here Microsoft Visual Studio has been used as this is the main development tool for FPAssistant Suite.
In the screen shot above the code results are 1.36% not covered and 98.64% of the source code is covered by the unit test cases.
Code Coverage is not the soul answer to developing bug free code but it is a good tool for reducing the potential bugs in the developed code and understanding which parts are covered by unit tests and those which are not. Even if 100% code coverage was achieved this does not mean that the program or App are bug free. What percentage should you aim for? Well, there is no defined percentage so aim to get as high as you can.
