Deciding when software is ready to release is a difficult. You have pressure from all sides to release perfect software, with added features. Below are some common facts:
– The engineers have said that the code was complete months ago, but are still making code changes.
– Sales people promised the software to major accounts months ago and are making commitments for the next release.
– The product manager wants a few more features added and you want to release a zero defect software.
Having no specific release criteria, many organizations wait for the manager to say “Release it”, not knowing when or why the decision was made at that time. In other cases, there is a general group consensus to just push the software out the door because the tired and stressed group wants to move out from the seemingly never ending cycles.
You can never take all of the stress out of releasing good software, but by planning ahead and setting up a good defect tracking repository, predetermining acceptable bug counts, properly testing the software and effectively triaging the defects you can always know the current state of the software, but most importantly know when the software is ready to ship.
Defect Tracking Repository
The most common as well as important tool to determine the state of the software is the defect tracking repository. The repository must be able to provide information required for the triage process. The repository must include the following info:
– The expected results of the testers test
– The steps to recreating the defect
– The version in which the defect was found
– The module in which the defect was found
– The severity of the defect
– A short descriptive name for the defect
– A description of the defect
– The actual results if the testers test
– Resolution notes
The project team should be able to query the database and gather information such as defect counts at various severity levels, and module levels, descriptions of the defects and steps to reproduce them. The repository becomes the archive for the project and it becomes important that correct and complete information be entered in order to use this information for later projects as a planning tool.
Predetermined Acceptable Bug Counts
The goal of releasing software with no defects cannot be achieved given the limited time and resources so, in the preliminary planning of the project exit criteria must be set up. The exit criteria should then be translated into the test plan outline, which should include acceptable bug count levels in order to ship the software.
The exact acceptable bug count level is not some magic number that will insure a successful product, but more a target goal that the group can use to see that they are moving forward toward a common goal. An acceptable bug count statement may look like this:
– Showstoppers: There may not be any
– High: There may not be any
– Medium: There may not be any that have a high probability of appearance to the customer
– Low: There may be a minimal amount in specified core areas and all other areas may have any amount.
In order for the entire group to know the state of the software at all timesit would be preferable to produce a weekly build grading process. The idea is that each week the build would be treated as if it were the software to be shipped and given a grade. The weekly grade keeps the team current on the software state and the team can see the software progressively improve.
This quantitative method takes a lot of the guesswork out of the equation.
Proper Testing
The key to shipping quality software is finding and fixing all defects. The testers responsibility becomes finding the defects and properly reporting them. In order for the testers to find the defects the lead tester must set up in the test plan outline, the specific stages of testing to ensure that the process is organized and covers all aspects of the software. The test plan may contain the following 7 phases:
– Test Case and Test Suite Planning
– Unit Testing
– Regression Testing
– System Testing
– Regression testing
– Performance Testing
– Release Candidate Testing
The goal is to turn up as many defects as early as possible in order to make the fixes easier on the engineers and to provide ample time for regression testing since every time the code is altered there is a greater risk of creating more defects.
As important as finding the defect, the tester must be able to correctly report it in the repository. This becomes extremely important in large projects where the defect counts may rise in the thousands.
Effective Triage
The Test Lead and Engineer must sit down regularly to evaluate the reported defects and drive the process forward in the most efficient manner for the engineers as well as the testers. Much of the decision making here is based on queries from the defect repository, which shows the importance of the accuracy and completeness of the repository.
The creation of a “Hot List” which is a list of important bugs is a great tool to create. This can be done in an plain Excel sheet, which identifies the most important defects, their module and a brief description. This list is great to use in triage to identify the defect action items for the upcoming week.
Defects that prevent further testing in a certain area must be given precedence and the engineers can assist in the risk analysis of future modules to assist the testers in their test case development. In the determination of what to fix next generally it is advantageous to group and fix defects relating to the same module at the same time even though there may be other important defects in other modules. Grouping of defects assists the engineers finish a module and move on.
Shipping quality software is not an easy task but by being organized from the start of the project the ship date and defect counts don’t have to be a “gut feeling”. The project must have a Defect Tracking Repository that must be maintained and updated continually with correct and complete information. The Repository is used as the main tool for an effective triage process and in conjunction with the Test Plan Outline that must be createdat the start of the project describing in detail the exit criteria to ship the software.
Please note the entire process rests on proper testing and the use of proper test planning which is what uncovers the defects. If you do not uncover the defects the users of the software will.
Thus, by following the above described process it is possible to know the state of the software at any time so that the entire team knows the goal, sees the progress and knows when the software is ready to release.
- WordPress Web Hosting for Small Businesses: Essential Tips - October 3, 2024
- Web Hosting for Online Startups: Scalability and Reliability - October 3, 2024
- 4 Best Upmetrics Alternatives for Your Business Planning [2024] - August 30, 2024
