Introduction

There's much more to testing software-oriented solutions than simply evaluating functions, interfaces, and response-time characteristics. Additional tests should focus on other requirements attributes, such as:

• integrity (resistance to failure)
• ability to be installed and executed on different platforms
• ability to handle many requests simultaneously

To achieve this, testers develop different types of tests. Each test type has a specific objective and support technique. Each technique focuses on testing one or more characteristics or attributes of the solution.

Team members apply these types of tests at different scopes. While Testers are concerned with the end-to-end solution, Developers are usually focused on a specific unit or component of the Implementation.

Quality Dimension: Functionality

Function test: Tests focused on validating the solution functions as intended, providing the required services, methods, or use cases. This test is implemented and executed against different targets-of-test, including units, integrated units, applications, and systems.

Security test: Tests focused on ensuring the target-of-test data (or systems) are accessible only to those actors for which they are intended. This test is implemented and executed on various targets-of-test.

Volume test: Tests focused on verifying the target-of-test's ability to handle large amounts of data, either as input, output, or persistent. Volume testing includes test strategies such as creating queries that would return the entire contents of a database, or that would have so many restrictions that no data is returned, or where the data entry has the maximum amount of data for each field.

Quality Dimension: Usability

Integrity test: Tests that focus on assessing the target-of-test's robustness (resistance to failure), and technical compliance to language, syntax, and resource usage. This test is implemented and executed against different targets, typically implementation units and components.

Structure test: Tests that focus on assessing the target-of-test's adherence to its design and formation. For example, with web applications, theses tests might ensure that all links are connected, appropriate content is displayed, and no content is orphaned.

Quality Dimension: Reliability

Stress test: A type of reliability test that focuses on evaluating how the system responds under abnormal conditions. Stresses on the system could include extreme workloads, insufficient memory, unavailable services and hardware, or limited shared resources. These tests are often performed to gain a better understanding of how and in what areas the system will break, so that contingency plans and upgrade maintenance can be planned and budgeted for well in advance.

Benchmark test: A type of performance test that compares the performance of a new or unknown target-of-test to a known reference-workload.

Contention test: Tests focused on validating the target-of-test's ability to acceptably handle multiple actor demands on the same resource (data records, memory, and so on).

Quality Dimension: Performance

Load test: A type of performance test used to assess the operational limits of a system under varying workloads while the system-under-test remains constant. In some variants, the workload remains constant and the configuration of the system-under-test is varied. Measurements are usually taken based on the workload throughout and transaction response time. The variations in workload usually include emulation of average and peak workloads that occur within normal operational tolerances.

Performance profiling: A test in which the target-of-test's timing profile is monitored, including execution flow, data access, function and system calls to identify and address both performance bottlenecks and inefficient processes.

Configuration test: Tests focused on ensuring the target-of-test functions as intended on different hardware and software configurations. This test might also be implemented as a system performance test.

Quality Dimension: Supportability

Installation test: Tests focused on ensuring the target-of-test installs as intended on different hardware and software configurations, and under different conditions (such as insufficient disk space or power interruptions). This test is implemented and executed against applications and systems.