Load Testing Guide

The following table is a high level guide to help with delivering an effective and efficient load/performance testing project:-

Term	Description
Like Live	Always ensure that the test environment is a mirror of the live. If this is not possible then ensure that the servers, etc are scalable or have a quantifiable performance measure compared to the live boxes or components. Run one functional test machine at the same time the load is being generated.
Workload Model	Ensure that the workload model is derived from business users, transaction logs or similar ‘real life’ activities. Take account of peak times and transactions including end of year/period. Capture any background processes that the user experiences (whether knowingly or not)
Virtual Users	Ensure the Load/Performance test tool is able to simulate 125% of the anticipated user community
Load Test clients	Typical per virtual (NT box) memory requirement is 1mb. Try not to go much above 200 virtual users a box (however this depends on the Load Test product and test machine configuration). This would require 200mb to service the virtual users, at least 32mb for the operating system and more for any other applications that may reside on the box (this should be kept to a minimum). Ensure the test clients will not be a bottleneck by checking the CPU utilisation per test client. If the average is above 75% then the test client will be a bottle neck, reduce the amount of virtual users that client runs. Dual processors will typically give a 60 – 70% performance gain
Controller	Try to have a 19” monitor and a graphics card with 8mb+. This helps with monitoring several machines, reports, graphs, etc. 128mb of memory should be sufficient. Ensure that the PC has adequate hard disk space due to the amount of data that can be collected during tests. Don’t run virtual users on the controller PC.
Bandwidth	Ensure that there is sufficient bandwidth on the LAN to run all the virtual users. An example would be if the average amount of data being submitted (including images) by a virtual user was 25k and the test had to simulate a user community of 250 (concurrently), then the bandwidth requirement would be a minimum of in excess of 6mb/s. Note no network Topology will be 100% efficient in fact as clients increase on a LAN segment the available bandwidth can drop to below 50%. So in this case a 10mb hub would not be sufficient. It is preferable to use a switch as this does not share the available bandwidth between clients. In most cases a 100mb switch will be sufficient. Ensure that all test machines have 100mb Network Interface Cards (NIC).
Monitors	There are various resources that may be used to monitor system performance. All load test tools will usually have an array of performance monitors or have access to the servers under tests own monitors. However to reduce network traffic it is best to only use basic monitors as supplied by the load tools. These include response times (transactions), this has no network overhead as it measures the round-trip time of selected transactions. Note: Response times do not usually include the time it takes the browser to render the returned response (which can include several images). To measure this at least one test client should be included in the test to carry out functional tests while the system is under load. This client should be built to the same specification as those in the live environment. Also network/node-based monitors should be used to give a general indication of which segment of the system caused the most delay. Monitors should be set up on the local machine(s) as prescribed by system specialists. In some cases the tester may want to remotely monitor these on their machine, this can be achieved by using emulation software such as Exceed or VNU (VNU is free). Each monitor that you run will degrade the performance of the machine it is run on. This performance hit will be small but it should be known so that an account can be made of it in the test results. You may get an estimate of this by closing all non-critical processes on the server and then run the monitor and see what the CPU utilisation is with the monitor on and then what it is with it off. The hit is (approximately) the difference.
Load Test Scripts	When a client opens a connection to a web server this is known as a session. While the client continues to send and receive responses from the server, the session is maintained by a session ID. Sessions can be maintained by a variety of methods. It is crucial to performance testing web-based applications to ascertain how the web server/application identifies the session as this will need to be used in subsequent tests. Don’t use random number generators to produce unique id’s for log on, etc as inevitably (especially where extended tests are concerned), it will generate two numbers that are the same. In any HTTP traffic between client and server the GET and POST functions are the most important.
Test Results	Use Mean and Standard Deviation in preference to Min and Max values when analysing results. Min and Max values are normally known as outliners and can skew results. Users are not normally concerned if once in a while they get a response time that is less than expected. However what does concern users is what the performance is ‘normally’. Get as much information as is possible. Take your time to analyse the results with system specialists and rerun tests to confirm your findings.