Benchmarking Perl scripts with Benchmark.pm

Kids argue over whose daddy is strongest, or coolest, or smartest. When those kids grow up and become Web developers they argue over which programming language is better, or faster, or has more features. But while little kids may have little concrete data to base their arguments on, the big kids have some very sophisticated toys to play with...

Everyone knows that Perl is extremely fast when it comes to handling regular expressions and text processing... but have you ever wondered how fast is extremely fast? Well, one of the toys us big kids have at our disposal is the Perl Benchmark module, which lets you test the speed of a Perl script.

Calculating differences in script execution time
The traditional way of measuring script execution time is also the common-sense one: check the time when the script starts, check the time when it ends, and the difference between the two values is the script execution time. In Perl these time values are obtained with the built-in time() function:

#!/usr/bin/perl

# declare array
my @data;

# start timer
$start = time();

# perform a math operation 200000 times
for ($x=0; $x<=200000; $x++)
{
  $data[$x] = $x/($x+2);
}

# end timer
$end = time();

# report
print "Time taken was ", ($end - $start), " seconds";

While this is fine for basic use, it becomes complicated if what you really want is to compare the times of different scripts, or run arbitrary pieces of code for fixed time intervals. For these uses, the Benchmark module is more appropriate. This module comes bundled with Perl, and can be imported into your Perl script through the "use" command. Take a look at the next example, which rewrites the previous one to use Benchmark instead of time().

#!/usr/bin/perl

use Benchmark;

# declare array
my @data;

# start timer
$start = new Benchmark;

# perform a math operation 200000 times
for ($x=0; $x<=200000; $x++)
{
$data[$x] = $x/($x+2);
}

# end timer
$end = new Benchmark;

# calculate difference
$diff = timediff($end, $start)

# report
print "Time taken was ", timestr($diff, 'all'), " seconds"; [/code]

Every time you create a new Benchmark object with new(), the current time is returned. The difference between the start and end times is calculated with the Benchmark module's timediff() function, and the result is formatted for display with the timestr() function. Here's the sample output of the script above:

Time taken was 2 wallclock secs ( 2.14 usr 0.00 sys + 0.00 cusr
0.00 csys = 2.14 CPU) seconds

As you can see, Benchmark returns a little more detail than the time() function.

Timing multiple runs of a script
Of course, a sample size of one is not necessarily representative of how fast your script is, especially on Web servers that are subject to varying loads. Therefore, what you really need is a way to run this script many times, and calculate the average time taken after compiling the data from each run. Luckily, Benchmark comes with a function to do this too. It's called timethis(), and it's demonstrated in the following example:

#!/usr/bin/perl

use Benchmark;

# run code 100000 times and display result
timethis(100000, '
      for ($x=0; $x<=200; $x++)
      {
            sin($x/($x+2));
      }
');

The timethis() function accepts two arguments: the number of times to run the code block, and the code block itself. This code block must be provided to timethis() in a format suitable to the eval() function.

Once the benchmark is complete, timethis() displays a report like this:

timethis 100000: 210 wallclock secs (209.37 usr + 0.00 sys = 209.37
CPU) @ 477.62/s (n=100000)

There are two pieces of useful data here: the number of CPU seconds, which tells you how long Perl takes to run the code N times, and the per-second data, which tells you how many runs take place per second. Obviously, the higher the second value, the faster your code is.

Instead of a fixed number of iterations, now let's see how to have timethis() run the code for a fixed period of time.