Benchmark 3 - The Alternative PHP Cache (APC)

Posted on 16.12.2010 by Kim N. Lesmer.
This is a benchmark test of the performance of PHP with and without APC (Alternative PHP Cache). The test was performed on an Intel P4 1.7 Ghz with 1036028 kB of memory over a private LAN. The server runs PHP 5.3.3 on a standard Debian installation. Between each change of settings all related processes was restarted.

The APC application is an Open Source cache for PHP written by Daniel Cowgill and George Schlossnagle. APC optimizes PHP intermediate code and caches data and compiled code from the PHP bytecode compiler in shared memory. APC is quickly becoming the de-facto standard PHP caching mechanism as it will be included (built-in) to the core of PHP starting with PHP 6.

On Debian Lenny and above APC can be installed using the command:

# apt-get install php-apc

APC will then be enabled by default.

Because APC is a compiler cache it is only really effective (which the benchmark will prove) when it is used on scripts that have a large code base.

Test 1

The first benchmark shows no increase in performance because it only deals with a simple class and method.

The small script I used to benchmark looks like this:

class c {
    function hw()
    {
        echo "Hello World!\n";
    }

    function do_it()
    {
        $i = 0;
        while ($i < 1000000) {
            $i++; $this->hw();
        }
    }
}

$inst = new c();

$inst->do_it();

I am using the Apache HTTP server benchmarking tool for the benchmarking with the following command: ab -c5 -n10 This will create 5 concurrent connections that each will perform 10 requests.

Test 1 Without APC

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            83
Document Path:          /hello-oo.php
Document Length:        13000000 bytes
Concurrency Level:      5
Time taken for tests:   15.081 seconds
Complete requests:      10
Failed requests:        0
Write errors:           0
Total transferred:      130002330 bytes
HTML transferred:       130000000 bytes
Requests per second:    0.66 [#/sec] (mean)
Time per request:       7540.598 [ms] (mean)
Time per request:       1508.120 [ms] (mean, across all concurrent requests)
Transfer rate:          8418.13 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       0
Processing:  6335 7410 651.2   7615    8746
Waiting:        3   14  22.2      3      74
Total:       6335 7410 651.2   7615    8746
Percentage of the requests served within a certain time (ms)
  50%   7615
  66%   7674
  75%   7729
  80%   7791
  90%   8746
  95%   8746
  98%   8746
  99%   8746
 100%   8746 (longest request)`

APC is then enabled.

Note: apc.enabled can be set to 0 to disable APC. This is primarily useful when APC is statically compiled into PHP since there is no other way to disable it (when compiled as a DSO, the extension line in php.ini can just be commented-out).

Test 1 With APC

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            83
Document Path:          /hello-oo.php
Document Length:        13000000 bytes
Concurrency Level:      5
Time taken for tests:   14.988 seconds
Complete requests:      10
Failed requests:        0
Write errors:           0
Total transferred:      130002330 bytes
HTML transferred:       130000000 bytes
Requests per second:    0.67 [#/sec] (mean)
Time per request:       7493.833 [ms] (mean)
Time per request:       1498.767 [ms] (mean, across all concurrent requests)
Transfer rate:          8470.66 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       1
Processing:  7317 7475 107.9   7476    7669
Waiting:        5   20  26.2      7      86
Total:       7318 7475 107.7   7476    7669
Percentage of the requests served within a certain time (ms)
  50%   7476
  66%   7489
  75%   7550
  80%   7600
  90%   7669
  95%   7669
  98%   7669
  99%   7669
 100%   7669 (longest request)`

From the above there is no difference at all and I have checked that APC is enabled and caching files.

Test 2

In this test I decided to try using the PHP Benchmark Script with Firefox running on the client.

Test 1 Without APC - Avg Result

Strings Encryption Dates Images Arrays Filesystem Objects Overall
220 ms  145 ms     36 ms 361 ms 52 ms  200 ms     80 ms   1094 ms

Test 2 Without APC - Avg Result

Strings Encryption Dates Images Arrays Filesystem Objects Overall
219 ms  145 ms     36 ms 359 ms 53 ms  199 ms     82 ms   1092 ms

APC is then enabled.

Test 1 With APC - Avg Result

Strings Encryption Dates Images Arrays Filesystem Objects Overall
218 ms  144 ms     36 ms 367 ms 53 ms  198 ms     78 ms   1095 ms

Test 2 With APC - Avg Result

Strings Encryption Dates Images Arrays Filesystem Objects Overall
221 ms  144 ms     36 ms 359 ms 53 ms  197 ms     79 ms 1089 ms

Again, as exspected, there is no noticable performance boost.

Test 3

As mentioned previously, APC has to work on a lot of code in order to reveal any performance improvements, so I am now going to throw some "heavy" stuff on the webserver to see if that will reveal some better results. I have installed Wordpress, Drupal, and Phorum and populated them with some data.

I run each test with: ab -c5 -n50 which creates 5 concurrent connections with 50 requests on each.

Wordpress Without APC.

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            90
Document Path:          /
Document Length:        9548 bytes
Concurrency Level:      5
Time taken for tests:   28.276 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      501000 bytes
HTML transferred:       477400 bytes
Requests per second:    1.77 [#/sec] (mean)
Time per request:       2827.558 [ms] (mean)
Time per request:       565.512 [ms] (mean, across all concurrent requests)
Transfer rate:          17.30 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  2256 2816 100.4   2823    3072
Waiting:     2203 2517  77.9   2514    2800
Total:       2257 2816 100.4   2823    3072
Percentage of the requests served within a certain time (ms)
  50%   2823
  66%   2832
  75%   2844
  80%   2850
  90%   2880
  95%   2922
  98%   3072
  99%   3072
 100%   3072 (longest request)`

Wordpress With APC.

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            90
Document Path:          /
Document Length:        9548 bytes
Concurrency Level:      5
Time taken for tests:   12.734 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      449878 bytes
HTML transferred:       426278 bytes
Requests per second:    3.93 [#/sec] (mean)
Time per request:       1273.351 [ms] (mean)
Time per request:       254.670 [ms] (mean, across all concurrent requests)
Transfer rate:          34.50 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   783 1268 651.8    999    3147
Waiting:      758 1155 587.0    921    2882
Total:        783 1268 651.8    999    3148
Percentage of the requests served within a certain time (ms)
  50%    999
  66%   1037
  75%   1202
  80%   1482
  90%   3064
  95%   3135
  98%   3148
  99%   3148
 100%   3148 (longest request)
`

Wordpress is no longer the simple and performance friendly CMS it used to be and this time there is quite a performance boost. Without APC Wordpress could deliver 1.77 requests per second whereas it could deliver 3.93 requests per second when APC was running. That's about twice as many requests. The entire process was performed at less than half the time. With APC off it took 28.276 seconds to complete the 50 requests. With APC on it took only 12.734 seconds to complete the same amount of requests.

Now, what does this tell us besides the fact that Wordpress has become bloated, slow, and heavy to dance with?

APC can have especially beneficial effects in systems with a large number of includes. If you are working with scripts that has many recursive includes (say 30 or more) APC can provide over 100% in application performance, but if you are working on simple scripts with few includes you are not going to see any performance boosts using APC.

Now, I am not going into details about the specific setup of Wordpress or what version is actually running on the server because the test is not about optimizing Wordpress. The test is about how much improvement of performance we can achieve on a given installation of the same product with APC enabled versus not having APC enabled.

Lets run the same test on a Drupal installation.

Again I run each test with: ab -c5 -n50 which creates 5 concurrent connections with 50 requests on each.

Drupal Without APC.

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            91
Document Path:          /
Document Length:        4957 bytes
Concurrency Level:      5
Time taken for tests:   18.988 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      275650 bytes
HTML transferred:       247850 bytes
Requests per second:    2.63 [#/sec] (mean)
Time per request:       1898.817 [ms] (mean)
Time per request:       379.763 [ms] (mean, across all concurrent requests)
Transfer rate:          14.18 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  1588 1893 146.3   1843    2202
Waiting:     1588 1892 145.5   1843    2201
Total:       1588 1893 146.3   1844    2202
Percentage of the requests served within a certain time (ms)
  50%   1844
  66%   1854
  75%   1870
  80%   2098
  90%   2157
  95%   2160
  98%   2202
  99%   2202
 100%   2202 (longest request)`

Drupal With APC

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            91
Document Path:          /
Document Length:        4957 bytes
Concurrency Level:      5
Time taken for tests:   9.215 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      275650 bytes
HTML transferred:       247850 bytes
Requests per second:    5.43 [#/sec] (mean)
Time per request:       921.476 [ms] (mean)
Time per request:       184.295 [ms] (mean, across all concurrent requests)
Transfer rate:          29.21 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   542  919 428.4    754    2266
Waiting:      541  918 424.8    754    2227
Total:        542  920 428.5    754    2266
Percentage of the requests served within a certain time (ms)
  50%    754
  66%    759
  75%    830
  80%    974
  90%   2065
  95%   2127
  98%   2266
  99%   2266
 100%   2266 (longest request)`

Again APC provides a great performance boost. With APC off Drupal could deliver 50 requests in 18.988 seconds with an average of 2.63 requests per second. With APC on Drupal could deliver 50 requests in only 9.215 seconds with an average of 5.43 requests per second.

Now, it's time to run the same test on Phorum. Again I'm running with ab -c5 -n50 which creates 5 concurrent connections with 50 requests on each.

Phorum Without APC.

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            98
Document Path:          /forum/index.php
Document Length:        26322 bytes
Concurrency Level:      5
Time taken for tests:   9.712 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      1336350 bytes
HTML transferred:       1316100 bytes
Requests per second:    5.15 [#/sec] (mean)
Time per request:       971.164 [ms] (mean)
Time per request:       194.233 [ms] (mean, across all concurrent requests)
Transfer rate:          134.38 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   728  966  90.0    943    1148
Waiting:      691  836  82.3    808     997
Total:        728  966  90.0    943    1148
Percentage of the requests served within a certain time (ms)
  50%    943
  66%    966
  75%   1012
  80%   1087
  90%   1111
  95%   1117
  98%   1148
  99%   1148
 100%   1148 (longest request)`

Phorum With APC.

Server Software:        Apache/2.2.9
Server Hostname:        webserver
Server Port:            98
Document Path:          /forum/index.php
Document Length:        26322 bytes
Concurrency Level:      5
Time taken for tests:   5.090 seconds
Complete requests:      50
Failed requests:        0
Write errors:           0
Total transferred:      1336350 bytes
HTML transferred:       1316100 bytes
Requests per second:    9.82 [#/sec] (mean)
Time per request:       508.971 [ms] (mean)
Time per request:       101.794 [ms] (mean, across all concurrent requests)
Transfer rate:          256.41 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   190  504 211.7    425    1095
Waiting:      181  414 200.7    330     977
Total:        191  504 211.7    425    1095
Percentage of the requests served within a certain time (ms)
  50%    425
  66%    458
  75%    517
  80%    595
  90%   1057
  95%   1083
  98%   1095
  99%   1095
 100%   1095 (longest request)`

With APC off Phorum delivered the 50 requests at 9.712 seconds with an average of 5.15 requests per second. With APC on Phorum delivered the 50 requests at 5.090 seconds with an average of 9.82 requests per second.

Conclusion

In all three cases APC provides a significant performance boost of about twice the performance. The only situation where APC has little or no noticable effect is when the application is rather small and this makes sense because APC is a compiler cache.

If you have any comments or corrections feel free to email me.