How to write an effective Test summary report (STR)?
Updated: Jan 19, 2022
Any testing effort contains several documents prepared as part of the testing process, and each document is prepared to answer a specific request/demand during the software development life cycle, such as:
Software Requirements Specification (SRS)
Software High-Level Design (HLD).
Risk Management Plan (SRM).
Software Test Design (STD).
Software Test Plan (STP).
What is a test summary report?
As you probably know, the SDLC contains many phases, including testing, which is used to determine the quality of the application before releasing it to the company clients.
The testing phase contains many activities, such as test planning and execution, with specific documentation.
Once those phases are completed, and the testing team approves the quality of the application, they will need to summarize the details, activities, and any problems they encountered during the testing process and present it to the senior management, project owners, and sometimes even to the company clients.
What sections should be included in a Test Summary Report?
Before examining the content of this report, know that each company may have a different test report format and content. Still, in general, each report must include a list of mandatory sections and data that I will review in the following paragraphs (Please note that you can change the order of artifacts as you wish).
#1 –Title
The title should include an informative description of the project name.
#2 – Unique Identifier
Each test report should include a unique identifier associated with each particular testing cycle; using this identifier. We will increase the tractability once needed.
#3 - Objective
This section describes and explains the primary activities used during the testing process, besides a short description of what type of testing was conducted (Unit, System, Etc.).
#4 – Description of the Application
Under this section, we should add a concise description of the tested application; the main goal here is to let the readers a short review of the application's complexity, goals, and the business case behind it.
#5 – References
Add any essential links to the documentation used in the testing project and can be used to support this report; the mandatory references are:
A test plan is used during the testing process (STD / STP).
Software requirements specifications (SRS).
High-level Design (HLD) of the application.
#6 – Resource allocation
Add a short description of the resources that we're involved in the testing process; the description should include the following details:
Name and Title
Area of responsibility.
Time spent during the project (house per Day/week).
Planned vs. unplanned resource modification during the project.
#7 – The application Testing Scope
This section summarizes the Modules/features tested/untested by the testing teams. The general idea is to let the readers understand which areas are covered in the testing process and which are not.
When you create this section, think about the following bullets:
The integration among the application components.
The individual application components.
If a module is not tested, add the cause and reasons.
Main flows that were tested.
#8 – Testing Activities
Define the testing activities and methodologies used during the testing process (It will be more efficient if you include basic information about each activity).
#9 - Project Methodologies
Scrum.
Waterfall.
Extreme Programming (EP)
#10 - Testing Activities (In-Scope)
Functional testing.
Exploratory testing.
Usability testing.
Component and unit testing
Load and scale testing.
#11 - Testing Activities (Out of Scope)
Negative testing.
Security testing.
Integration testing.
Endurance testing.
Stress Testing.
#12 - Test Coverage
How many test cases are successfully executed?
How many test cases are were failed?
Describe the total number of tests planned at the beginning of the project against the actual tests executed.
#13 - Risks
Determine the modules that were excluded from testing.
Determine the modules with higher risk.
Determine the modules with low risk.
#14 – Defects Statistics and related information
This is the place to provide statistics about the number of defects found during the testing effort; the information should help the readers understand the test execution results.
Make sure that your report covers the following statistics:
The Number of defects is divided based on their severity (Blocker, Critical, Major, etc.).
A total number of defects found during the version and per each testing cycle.
Defects that were not fixed in the current version (Deferred to future version).
How many bugs are not resolved and deferred to future versions?
The total number of bugs found by automation/manual testing.
List of the most critical bugs found during the version.
Defects per application module (Defect patterns).
The Number of defects is divided based on their statuses (Invalid, Duplicate, Cannot Reproduce, Etc..).
#15 – Automation
Add a short description of the automation framework used during the tests; the information should include:
The number of defects found using the automation framework.
A general description of the automation framework.
The effort invested in creating automation tests.
The number of execution hours per automation.
The test coverage of the automation.
#16 – Testing Tools
Add a short description of the 3rd party / In-house tools used during the tests.
#17 – Test Environments
This area should include the test environments used by the testing team during the testing process; the information should include the following:
Details about the environment configurations.
The environment availability during the project.
Details about any unique configuration.
Details about the operating systems.
#18 – Definition and Approval of the Exit criteria
Description of the testing exit criteria defined at the beginning of the testing project; the main goal is to determine if all conditions are fulfilled. Example:
All bugs with Major and above severity are fixed and verified – Approved/Declined.
90% of all tests are moved to the automation framework – Approved/Declined.
All planned test cases are executed – Approved/Declined.
All necessary modules should be tested – Approved/Declined.
All significant Risks are removed – Approved/Declined.
There is a 90% code coverage – Approved/Declined.
All development activities are done, and no more testing is required – Approved/Declined.