Performance testing: considerations and best practices

Your test environment should be capable of simulating production environment conditions. To do this, keep the following considerations in mind during the test cycle:

• Do not place too much stress on the client.

• Create baselines for your test setup.

• Allow for think time in your test script.

• Consider test duration.

• Minimize redundant requests during testing.

• Consider simultaneous versus concurrent users.

• Set an appropriate warm up time.

The next sections describe each of these considerations.

How to simulate HTTP Basic Authentication in Borland SilkPerformer

How Basic Authentication works: http://tools.ietf.org/html/rfc2617#page-5

Code:

WebBase64Encode(sUserPass, 200, nUserPassLen, “username:password”);
sUserPass := “Basic ” + sUserPass;
WebHeaderAdd(“Authorization”, sUserPass);

Variables sUserPass and nUserPassLen should be declared.

Approaches to Performance Testing. Part 1

1. Diversity of Approaches to Performance Testing

• There are many variations within the broad framework of performance testing.
• There is no universal or consistent set of terminology, and many organizations have their own terms such as “work load testing” and “sweet spot testing”

2. Amount of Load that is put onto the server

• It can come from two different areas:
  • the number of connections (or virtual users) that are hitting the server simultaneously
  • the amount of think-time each virtual user has between requests to the server
• The more users hitting the server, the more load will be generated.
• The shorter the think-time between requests from each user, the greater the load will be on the server.
• Keep in mind that as you put more load on the server, the throughput will climb, to a point.

3. Baseline/Performance Testing. Concept

• Baseline — a range of measurements that represent acceptable performance under typical operating conditions.
• Testers have a baseline for how the system behaves under normal conditions.
• Baseline can then be used in regression tests to gauge how well a new version of the software performs.
• Baseline provides a reference point that makes it easier to spot problems when they occur.

4. Benchmark Testing. Concept

• The key to benchmark testing is to have consistently reproducible results.
• Benchmark tests should be used to determine if any performance regressions are in the application.
• Benchmark tests are great for gathering repeatable results in a relatively short period of time.
• The best way to benchmark is to change one and only one parameter between tests.

5. Benchmark Testing. “Flat” and “Ramp-Up”. Run Modes

• In case of “Flat” run mode, all of the users are loaded at once, and then run them for a predetermined amount of time.
• In case of “Rump-Up” run mode, users are loaded step by step.

Technical Skills For Performance Testers

Performance Concepts:

• Methodology
• Load Generation Tools
• User/Workload Modeling
• Results Analysis (Latency, Throughput, Metrics)
• Bottleneck Detection
• Code Profiling
• Scalability
• Concurrency
• Charting/Graphing
• Statistics

Operating Systems and Servers:

• Monitoring (CPU/Network/Mem/Disk/etc)
• System Tuning
• Web/Application/Middleware Server Tuning
• System Administration
• Virtualization
• OS Concepts (CPU Scheduling, Memory Management, etc)

Database:

• SQL
• Stored Procedures
• Monitoring
• Tuning

Network:

• Topology
• Monitoring
• Load Balancing
• TCP/IP
• HTTP
• Packet Sniffing and Protocol Analysis
• Caching
• OSI Model

Programming:

• Proficiency in at least on general programming language. Preferably a dynamic scripting language (Python/Perl/Ruby/etc)
• Code/Algorithm Analysis

Source

Resources about Performance testing

In this post I’m going to share a set of URLs about performance testing.

There is a wonderful monthly on-line journal “Software Test and Performance“. It’s free.

There are interesting articles on StikyMinds.com:

“Web Load Test Planning”
Summary:
Predicting how a particular Web site will respond to a specific load is a real challenge. Here are three basic steps necessary to design highly realistic and accurate Web site load tests.

” The Science and Art of Web Site Load Testing”
Summary:
Web site load testing is very different from traditional load testing and requires new tools and new approaches. Most Web site load tests are wildly inaccurate and unrealistic and consequently useless or dangerously misleading. Useful load testing requires loading a Web site with different scenarios, at different load points, and monitoring all the key components. Approach Web site load testing as 80% science and 20% art.

“Web Page Response Time 101”
Summary:
Online sites lose billions of dollars every year due to sluggish performance speed and user bailouts. Here are four laws of Web site performance that will help you understand response rates and deal with your “abandonment issues.”

Articles, Columns and Papers on PerfTestPlus, Inc.

Also I would recommend reading “Performance Testing Guidance for Web Applications” .

J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber, and Dennis Rea
Microsoft Corporation

September 2007

This guide shows you an end-to-end approach for implementing performance testing. Whether you are new to performance testing, or looking for ways to improve your current performance testing approach, you will find insights that you can tailor for your specific scenarios.

Download the guide:
The Final Release is Available! Start using the guide today, while we continue to make improvements.

• Download the Performance Testing Guidance for Web Applications Guide

MySQL: Tracking slow queries

While running performance testing against some web application we should measure not only such metrics as page download time, hits per second and so on, but also those ones that relate to a database. It’s obvious that the time of queries execution has a great impact on the first metrics like page download time.

For such cases MySQL has a very handy ability to track slow queries. For more info please visit http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html

Also there is a simple program called MySQL Proxy that can monitor, analyze or transform communication between application and server. Please go here to get more info.

And it’s useful to know about MySQL Profiles. Here is an article “Using the new MySQL Query Profiler“