14.3. Benchmarking

What matters in the end is how your site performs overall. An effective way of testing designs and detecting bottlenecks is to benchmark your site by simulating production traffic.

This section briefly introduces two tools for site benchmarking: ApacheBench and Siege.

14.3.1. Using ApacheBench

One benchmarking tool is ab (which stands for Apache Benchmarking tool) which is bundled with the Apache web server and is most likely installed on your system already if you are running Apache. ab works by simulating a number of clients sending requests to your web server with a specified delay, hammering away on the same URL.

Here's an example:

$ ab -n 10000 -c 10 http://localhost/test.php

The n option specifies the number of requests, and the c option specifies the number of concurrent clients. This code will fire off 10,000 queries requesting /test.php from localhost, 10 at a time. When all requests have finished, ab prints a summary:

[...skipping first part of output...]

Document Path:          /test.php
Document Length:        3037 bytes

Concurrency Level:      10
Time taken for tests:   15.875129 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      32080000 bytes
HTML transferred:       30370000 bytes
Requests per second:    629.92 [#/sec] (mean)
Time per request:       15.875 [ms] (mean)
Time per request:       1.588 [ms] (mean, across all concurrent requests)
Transfer rate:          1973.40 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0      11
Processing:     1   14  19.2     13     404
Waiting:        0   10  14.8     10     403
Total:          1   14  19.2     13     405

Percentage of the requests served within a certain time (ms)
  50%     13
  66%     14
  75%     15
  80%     15
  90%     17
  95%     26
  98%     62
  99%    110
 100%    405 (longest request)

The interesting numbers here are the throughput (requests per second and time per request), and the percentiles at the end. In this case, 80 percent of the requests finished in 17ms or less, and 99 percent finished in less than 110ms.

For more information, run just ab to get a full list of options

14.3.2. Using Siege

The major weakness of ab is that it does not let you simulate a more realistic request distributionfor example, by letting you specify a list of request URLs to rotate between.

One benchmarking tool that provides this feature is Siege. You can find more information about Siege at http://www.joedog.org/siege/.

Siege lets you specify a file with full URLs, and picks a random URL for each request. Here's an example:

$ siege -i -t 10S -f urls.txt
** Siege 2.59
** Preparing 15 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200   0.02 secs:     131 bytes ==> /test.php
[...skipping...]

Lifting the server siege...\done.
Transactions:                     29 hits
Availability:                 100.00 %
Elapsed time:                   1.98 secs
Data transferred:              64825 bytes
Response time:                  0.01 secs
Transaction rate:              14.65 trans/sec
Throughput:                 32739.90 bytes/sec
Concurrency:                    0.19
Successful transactions:          29
Failed transactions:               0

Although Siege does not print a percentile summary, you can create one yourself by processing the requests printed on standard output. Again, run siege without parameters or man siege for more details.

14.3.3. Testing Versus Real Traffic

The danger of running a test like this is that it does not really simulate real-world traffic. Real traffic includes web browsers behind slow modems that cause requests to take a long time, as well as search engine crawlers and other weird things that can affect your site's performance and are difficult to simulate with a benchmarking tool.

You can approach this by carefully creating your benchmarking requests file, preferably basing it on real traffic logs, or at least by making a realistic estimate.