Whether you're launching a new Website, or adding new features to an existing one, a traceability matrix can improve your chances of success. In simple terms a traceability matrix is a table that combines the product requirements document (PRD) with the test case document by creating a one to one relationship between features and test cases. For every feature the PRD lists there is a corresponding test case that the Website must pass before it goes live.
There are several easy ways to create a traceability matrix. One of the easiest ways is to a table in MS Word and save it as an HTML document. For slightly more effort a traceability matrix can be created and maintained as a table in an HTML editor like BBEdit. Many Wikis support tables and can be created and updated without having to bother with HTML code.
A critical aspect of getting a traceability matrix off to a good start is using the same numbering convention on both the PRD and test case documents. The industry standard is to use a numbering convention that uses periods to divide the numbers into segments (i.e. 1.1.1). In this approach, the first number represents a major feature, and the second number represents an element of that feature, and the third number identifies details of that element.
For instance, if you're adding a simple registration form for customers to register for a newsletter, a common approach would be to divide the page into its major components; the header, footer, left navigation bar, and the main body of the form. The assigned numbers would be: 1.0 for the header, 2.0 for the footer, 3.0 for the left navigation bar, and 4.0 for the main body. The beauty of this numeration format is that it makes it easy to add feature, and the numbers that go with them, after the initial document has been completed.
In this hypothetical case we assume that the header, footer and navigation elements are the same on all pages of the Website, so only the main body is adding new features. In this case, the body is to have four elements, the first name field, last name field, email address field and submit button.
The numbering convention used by the requirements document would give the numbers 4.1 for the first name field. 4.2 for the last name field, 4.3 for the email address field, and 4.4 for the submit button. Specific details of each element are handled by adding another number. For example, the first name would be required to accept letters only, and allow a maximum of 25 characters to be entered. This would assign the number 4.1.1 to the letters only requirement, and 4.1.2 for the maximum character entry requirement.
Adding a corresponding test case for each requirement is relatively easy and would result in something like this:
- 4.1. Verify the Web page has a first name field.
- 4.1.1 Verify the first name field will accept letters only.
- 4.1.2 Verify the first name field will accept a maximum of 25 characters only.
Sometimes the test cases form a one to many relationship, as when one requirement implies several test cases but doesn't specifically state them. An example might be negative tests for the first name field. The positive test case is 18.104.22.168 and it would verify that the first name field will accept letters. The negative test case, 22.214.171.124, will verify that when numbers are entered the form cannot be submitted and will return an error.
In its simplest form the traceability matrix would consist of three columns, one column for the number, one column for the requirements, and one column for the corresponding test case. A somewhat more advanced form would add a column for test results. Either format would create a matrix that would make any requirement lacking a corresponding test case glaringly obvious.
In its more complex forms, a traceability matrix can include the names of the QA engineers responsible for writing the test cases as well as the names of the QA people responsible for executing those test cases. On larger projects a column is frequently added to show the percentage of tests completed and another column to show the number of open defects, or bugs, filed against that part of the Website. When all the features show all tests area completed and there are zero open defects, you will know for certain it's safe for your Website to go live.
In a larger project, like a Website with a dozen sections or more, listing every feature and corresponding test case will result in a extremely long Web page. For larger projects it's better to condense the information by allocating just one line to each feature, with a link to the complete PRD and test case document for that feature.
While it would be easy to create a traceability matrix on paper, it's the usual practice to put it online as a Web page. The advantage of the Web approach is that it keeps everyone up to date with minimum effort. The marketing team can verify that all the requirements are present, and the QA team will know if it has created all the test cases needed to release a Website that works.
The traceability matrix also comes in handy when requirements are updated after the project is underway. Some of the automated traceability matrix tools, like Mercury Quality Center, will automatically email the appropriate QA team member responsible the test cases attached to requirements that have changed. These commercial matrix tools can cost over a thousand dollars, making them a dubious investment as a stand-alone purchase. However, if you're already committed to buying an automation tool like WinRunner or Rational, it may be worth it to add the traceability matrix that these companies offer as an option.
Adding a traceability matrix to your Website may sound like extra work, but the payoff will more than justify the effort. Given the relative simplicity of creating and using a traceability matrix, it's hard to justify not using one for every Website project
Glen Emerson Morris was recently a senior QA Consultant for SAP working on a new product to help automate compliance with the Sarbanes-Oxley law, an attempt to make large corporations at least somewhat accountable to stockholders and the law.
He has worked as a technology consultant for Yahoo!, Ariba, WebMD, Inktomi, Adobe, Apple and Radius.