Approaches to Performance Testing. Part 2

Approaches to Performance Testing. Part 1

18. Load/Volume Testing. Concept

• In the testing literature, the term “load testing” is usually defined as the process of exercising the system under test by feeding it the largest tasks it can operate with.
• Load testing is used to determine whether the system is capable of handling various anticipated activities performed concurrently by different users.
• Load testing ensures the level of confidence with which the customer uses the system efficiently under normal conditions.
• Usually load tests generate 80% of traffic (amount of load) a system can potentially handle.
• There is an extreme importance of having large datasets.

19. Load/Volume Testing. Goals

• Expose bugs that do not surface in cursory testing, such as memory management bugs, memory leaks, buffer overflows, etc.
• Ensure that the application meets the performance baseline established during performance testing. This is done by running regression tests against the application at a specified maximum load.

20. Load/Volume Testing. While Executing

• During the execution of the load test, the goal is to check whether the system is performing well for the specified load or not.
• To achieve this, system performance should be captured at periodic intervals of the load test.
• Performance parameters like response time, throughput, memory usage, and so forth should be measured and recorded.
• This will give a clear picture of the health of the system.

21. Stress Testing. Concept

• Stress testing goes one step beyond the load testing and identifies the system’s capability to handle the peak load.
• In stress testing, think time is not important as the system is stressed with more concurrent users beyond the expected load.
• Stress testing tries to break the system under test by overwhelming its resources or by taking resources away from it (in which case it is sometimes called negative testing).
• The main purpose behind this madness is to make sure that the system fails and recovers gracefully this quality is known as recoverability.

22. Stress Testing. Examples

• Double the baseline number for concurrent users/HTTP connections.
• Randomly shut down and restart ports on the network switches/routers that connect the servers (via SNMP commands for example).
• Take the database offline, then restart it.
• Rebuild a RAID array while the system is running.
• Run processes that consume resources (CPU, memory, disk, network) on the Web and database servers.
• This list can be enhanced with your favorite ways of breaking systems.

23. Stress Testing. Goals

• However, stress testing does not break the system purely for the pleasure of breaking it, but instead it allows testers to observe how the system reacts to failure.
• Does it save its state or does it crash suddenly?
• Does it just hang and freeze or does it fail gracefully?
• On restart, is it able to recover from the last good state?
• Does it print out meaningful error messages to the user, or does it merely display incomprehensible hex codes?
• Is the security of the system compromised because of unexpected failures?
• And the list goes on.

24. Configuration Testing. Concept

• Integrated with performance testing to identify how the response time and throughput vary as the configuration of infrastructure varies and to determine the reliability and failure rates.
• Configuration tests are conducted to determine the impact of adding or modifying resources.
• Verifies whether a system works the same, or at least in a similar manner, across different platforms, Database Management Systems, Network Operating Systems, network cards, disk drives, memory and central processing unit settings, and execution or running of other applications concurrently.
• Compatibility testing is a term which is used synonymously with configuration testing since compatibility issues are the matter of interest here.

25. Scalability Testing. Concept

• Scalability Testing, part of the battery of non-functional tests, is the testing of a software application for measuring its capability to scale up or scale out – in terms of any of its non-functional capability – be it the user load supported, the number of transactions, the data volume etc.
• The purpose of scalability testing is to determine whether the application automatically scales to meet the growing user load.

26. Scalability Testing. Scale Up / Scale Vertically

• To scale vertically (or scale up) means to add resources to a single node in a system, typically involving the addition of CPUs or memory to a single computer.
• A server twice as fast is more than twice as expensive.
• Taking advantage of such resources can also be called “scaling up”, such as expanding the number of Apache daemon processes currently running.

27. Scalability Testing. Scale Out / Scale Horizontally

• To scale horizontally (or scale out) means to add more nodes to a system, such as adding a new computer to a distributed software application.
• An example might be scaling out from one web server system to three (organizing a cluster system).

28. Scalability Testing. Ideal Scalability

29. Scalability Testing. Tradeoffs between Scale Up and Scale Out models

• Larger numbers of computers means increased management complexity, as well as a more complex programming model and issues such as throughput and latency between nodes; also, some applications do not lend themselves to a distributed computing model.
• However, the price differential between the two models is increasingly in favor of “scale out” computing for those applications that fit its paradigm.
• Super computers are scaled out!

30. Testing which is driven by what we want to measure.

• Response time testing
• Throughput testing
• Availability testing
• Measurement of resource utilization
• Capacity testing
• Measurement of delays (latency)
• Measurement of losses in networks
• Error rate measurement

31. Testing which is based on source or type of load.

• Usage-based testing
• Standard benchmark testing
• Load variation testing
• Ramp-up testing
• Component-specific testing
• Calibration testing

32. Testing which focuses on the impact of changes.

• System change impact assessment
• Infrastructure impact assessment
• Baseline testing
• Volume testing
• Parallel testing
• Live patch and change testing
• Extreme configuration testing

33. Testing which seeks to stress the system or find its limits.

• Scalability testing
• Bottleneck identification and problem isolation testing
• Duration or endurance testing
• Hot spot testing
• Spike and bounce testing
• Breakpoint testing
• Rendezvous testing
• Feature interaction / interference testing
• Deadlock testing
• Synchronization testing
• User scenario, bad day or soap opera testing
• Disaster recovery testing
• Risk-bases testing
• Hazard and threat identification
• Environmental testing
• Compatibility and configuration testing

Sources:

• “Integrated Approach to Web Performance Testing: A Practitioner’s Guide“, B. M. Subraya, Infosys Technologies Limited, Mysore, India
• “System Performance Testing. A Case Study. Part 1: Developing the Test Strategy”, Ross Collard, Collard & Company, New York, NY
• http://en.wikipedia.org/wiki/Performance_Testing
• http://en.wikipedia.org/wiki/Performance_Engineering
• Approaches to Performance Testing. A best-practices approach to maximize your performance test effort
• Performance vs. load vs. stress testing

Explore posts in the same categories: Performance testing

Tags: Configuration Testing, Ideal Scalability, Load, Scalability Testing, Scale Out / Scale Horizontally, Scale Up / Scale Vertically, Stress testing, Volume Testing

You can comment below, or link to this permanent URL from your own site.