The purpose of performance testing is to reduce risks to system performance and its impact on the Business.
For example, some common question performance testing can answer are:
- Can the system process the expected number of transactions and load?
- At the expected levels of load (average & peak) are the response times acceptable?
- Is the system reliable?
- What is the capacity of the system and has the capacity been correctly specified?
- Will the system grind to a halt after the 10th user logs on and die 30 minutes after Go Live?
How is Performance different then Functional Testing?
Performance testing differs from functional testing in that it is focused on generating load on the back end of the system under test, and is not concerned with testing the functionality of the system beyond what is required to generate that load.
- Often no Pass/Fail criteria, involves repeated testing and tuning
- Requires a Production-like Test Environment
- Requires a Large and Stable Production-like Test Data Set
- Monitoring of Infrastructure and access to Various Domain Experts within an organization
- Requires Functional Stability of Application
Business Goals and Risk
- Performance test are designed and structured to mitigate a Business Risk
- Usually one Performance Test type mitigates one Business Risk
- Businesses have unique Business Risks that must be clearly defined so that Performance Test Scenarios can be constructed accordingly
- Example: “We losing business when the system grinds to a halt after the 10th user logs on”
Business Process Transaction Mix
- It is usually not cost effective to try and cover all of the functionality in a performance test
- Performance Testing is not about code coverage, rather the aim is about application and infrastructure readiness for Business use.
- To conduct an accurate load simulation, the number of transactions performed per hour must be determined for each Business Process
- These are either Production figures (best), or Estimates of the expected load (next best)
- These figures are used to select which transactions should be scripted, and to configure the resulting Performance Tests
- At a minimum both Average and Peak (busiest hour) values are required
Performance testing Tools and script development
When a GUI based client application (thick client) or Web base application is running on a PC, typically only a single user can use that PC at a time. However, a performance testing tool must simulate hundreds or thousands of end-users, a different approach must be taken.
Performance test scripts are usually “recorded” using the application under test. During recording, the communication protocol between the client and the server is captured. Therefore, during playback the client application is not used. The script simulates the client by communicating directly with the system under test. With no client application running, many “Virtual Users” can be executed on the same PC, thus generating load with minimal hardware.
- After each test, the results can be analysed to try and identify where problems are occurring, and to extract Transaction Response Times, and server statistics for inclusion in the Test Report.
- After the test it is common for some system remediation to take place to correct problems discovered.
- The test cycle is then repeated to verify that problem or bottleneck has been removed.