Some Basic Testing Concepts

Before running any test, we should keep the answer of the following rough questions:

• What we are testing? Define the test subject, whether it is a thing, a process, a behavior, a threshold, etc. Define the scope of the test subject. For example, if you are testing a web site's links, will you test every link, or only links to static pages, or only links to other pages as opposed to internal links, etc?
• From what point-of-view are you testing? If your test is supposed to mimic the interaction of a specific agent or user, then you must have a strong understanding of that agent or user.
• What are you testing for? Be as specific as possible. If you are going to test one aspect of the test subject, make that limitation clear.
• How are you going to test? Define the test plan, the specific test(s), test methodologies, etc.
• What are the limits to the test? Set expectations carefully, because if the test can only measure a part of the test subject or its behavior, the results must be interpreted with this limitation in mind.

Testing and Quality Control

Testing is the process of creating, implementing and evaluating tests. If you are shopping for a new television, you can call that process "testing for the best TV for you"... it's kind of pretentious, but that is what you're doing as you compare prices and features to find what will work best for you. Testing usually has a limited scope and duration – you're just looking at TVs and only in your town, you're not going to spend a year shopping, are you?

Quality control is a refinement of testing, involving the formal and systematic use of testing and a precise definition of what quality means for the purposes of the test. You aren't just testing; you are testing and then doing something with the results.

Quality control is used for testing a product or output of a process, with the test measuring the subject's ability to meet a certain benchmark or threshold of quality. The tests usually take the form of "does this product meet requirement X?", and are often pass-fail.

Effective quality control testing requires some basic goals and understanding:

• You must understand what you are testing; if you're testing a specific functionality, you must know how it's supposed to work, how the protocols behave, etc.
• You should have a definition of what success and failure are. In other words, is close enough good enough?
• You should have a good idea of a methodology for the test, the more formal a plan the better; you should design test cases.
• You must understand the limits inherent in the tests themselves.
• You must have a consistent schedule for testing; performing a specific set of tests at appropriate points in the process is more important than running the tests at a specific time.

Any true attempt at quality control requires a great deal of planning before any tests are ever applied, and extensive documentation of quality standards, test plans, test scenarios, test cases, test results -- anything that goes into the testing must be carefully tracked and written down. In fact, for companies that manufacture products, as well as for software companies, a series of formal accreditation programs exist to measure and certify the company's adherence to some very strict standards, for example the ISO 9000 series of rules. No such certification systems exist for web sites, perhaps because sites are more experiences and resources than products to buy.