HTTP/2 and PHP with Apache on FreeBSD

Steps Toward The Goal

On the first page I showed how to set up a Google Compute Engine virtual machine, and on the second page I finished setting up FreeBSD. That including installing the Apache and PHP software packages. On the previous page I started setting up Apache, customizing the logging and telling the client to cache large and infrequently changing data. Now let's finish setting up the web server! Some of this will improve performance, the rest will increase capability.

The goals of this page are:

1. Support HTTP/2, preferring it over HTTP/1.1.
   
2. Use a more efficient Apache Multiprocessing Module.
   
3. Use Index.html as the directory index file.
   
4. Treat all HTML files as PHP.

On following pages I will install dual free Let's Encrypt TLS certificates for RSA and ECC, and then adjust the Apache configuration for a good score from the authoritative Qualys server analysis.

Why HTTP/2?

HTTP/2 provides better performance than HTTP/1.1. The HTTP headers are compressed, and multiple HTTP requests are sent through a single TCP connection (called HTTP pipelining or multiplexing).

Recall that each TCP connection requires a 3-way handshake to set it up, and another 3-way handshake to shut it down. Moving everything through one TCP connection saves a lot of back-and-forth exchanges to set up and tear down multiple connections. Mobile clients have greater latency, and all those additional TCP connections can really slow things down.

Concurrency, using one TCP pipeline for several sets of data, can be especially helpful with mobile clients. Some hosting providers claim that HTTP/2 can cut page load times in half.

Other HTTP/2 features can provide further performance advantages if the server and client can take advantage of them. If the server can anticipate which additional resources are needed for a given page, server push is the technique of "pushing" those resources to the client before the client analyzes the initial content and realizes that it needs to request them. See the Apache documentation for details on server push.

From the opposite direction, stream prioritization is a technique by which the client can prioritize certain data streams over others. It can request components like images and CSS and JavaScript files in an order that speeds page rendering.

Site speed is a Google search ranking factor. So, while they don't specifically rank on support for HTTP/2, its speed improvements will help your ranking.

How Did We Get HTTP/2?

HTTP appeared, got an initial tweak, and then stayed the same for almost two decades. Its design was about as stable as boring old UDP. Meanwhile, HTTP became the Internet's most-used application protocol. Then Google started stirring things up in their quest for web performance.

HTTP was initially developed in the early 1990s. Remember Netscape Navigator, and NCSA's Mosaic before that? You very likely don't, but that's what they were designed for. Version 1.0 was formally defined in 1996. An update, version 1.1, came out in early 1997. Then things were quiet for a decade, and it was almost 20 years before HTTP/2 arrived.

Google designed the SPDY protocol in the early 2010s. It aimed to increase page load speed, and was supported by several browsers.

SPDY evolved into HTTP/2, which was published as a standard in 2015. Most browsers have supported HTTP/2 since 2015.

Your server will need support from its shared libraries. You need a TLS library that supports the ALPN protocol. That means at least OpenSSL 1.0.2 (Jan 2015), LibreSSL 2.1.3 (Jan 2015), GnuTLS 3.2.0 (May 2013), or others. That was no problem on FreeBSD, but someone locked into older Red Hat or CentOS Linux distributions may be unable to run HTTP/2. On FreeBSD, try the following commands. The first one checks the standard command included in the base operating system. The second checks the version installed when you add the libressl or openssl package. The openssl package provides a slightly newer version of the binary and shared libraries, plus a large collection of manual pages.

$ openssl version
$ gnutls-cli --version
$ /usr/local/bin/openssl version   # for libressl
$ pkg info | egrep 'openssl|gnutls|libressl' 

HTTP/2 is an alternative to HTTP/1.1 and HTTP/1.0, not a replacement. It includes a negotiation mechanism, so the client and server can use HTTP/1.0, HTTP/1.1, or HTTP/2.

You will find entries for HTTP/1.0 clients in your log, but you will probably find that most are automated indexing bots. I was surprised to learn that many indexing bots run the original protocol. However, I suppose that HTTP/2 doesn't really have a big advantage in that specific situation.

Getting Started, and Looking Ahead

I initially did everything described in this series of pages other than the parts specific to HTTP/2. I started with the default Apache configuration file, modifying it to treat all HTML files as PHP. I use PHP to insert a standard footer on every page on the site, and to insert Javascript blocks for Google AdSense. The server worked just fine, as far as serving web pages with functioning PHP.

I didn't notice until a month or two later that it wasn't supporting HTTP/2. I read an article that listed estimates of the percentage of clients using HTTP/2. That made me curious. I used some simple grep -c commands to count how many lines in the Apache log file contain the strings "HTTP/1.0", "HTTP/1.1", and "HTTP/2". I was surprised to find no entries for HTTP/2!

After tracking down what seemed to be missing, I made some changes. But now Apache would not start! I fixed that and broke something else. After some investigation, I got everything working. Here's what I discovered:

The sample httpd.conf file supplied in the FreeBSD Apache24 package does not support HTTP/2. It fails to do so because of an error that is not logged using that configuration file. Then, what seems to be the obvious solution prevents Apache from starting. Apache configuration debugging doesn't make for a thrilling tale, but there were mysteries to solve.

Follow along with my experiments and debugging. By the time we reach the bottom of the page, I will have it all working!

Enabling HTTP/2 and PHP with Apache Modules

I had first set out to enable HTTP/2 and PHP. I had added the following to my other changes near the end of the httpd.conf file:

[... many lines not shown ...]

# Enable HTTP/2
LoadModule http2_module libexec/apache24/mod_http2.so
# Prefer HTTP/2 over TLS, then HTTP/2 without TLS, then HTTP/1.1, finally HTTP/1.0
Protocols h2 h2c http/1.1

# Set up PHP
# NOTE: This line will be replaced by the time we're done,
# so keep on reading to see what really goes here...
LoadModule php7_module        libexec/apache24/libphp7.so

[... many lines not shown ...] 

Mysterious Failure to Support HTTP/2

I restarted Apache, and used curl to request just the header (with -I), ignoring the fact that the certificate is not good for host name localhost (with -k), over HTTP/2 (with --http2). But I did not get what I expected:

# curl -I -k --http2 https://localhost/
HTTP/1.1 200 OK
Date: Sat, 27 Apr 2024 06:52:52 +0000
Server: Apache/2.4.29 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.1.14
Upgrade: h2,h2c
Connection: Upgrade
Last-Modified: Sat, 27 Apr 2024 06:52:52 +0000
ETag: "fc2-55dbe74d77540"
Accept-Ranges: bytes
Content-Length: 4034
Content-Type: text/html charset=UTF-8 

Note: the libcurl package on some Linux distributions (e.g., Mint) may not support HTTP/2, giving you the error message "curl: (1) Unsupported protocol". See which features yours includes with:

# curl -V 

I verified that I had loaded the mod_http2.so module, and that the server reported no error or warning when restarting. There was an error but the server did not log it by default. Here is all I got:

# grep mod_http2.so /usr/local/etc/apache24/httpd.conf
LoadModule http2_module libexec/apache24/mod_http2.so
# tail /var/www/logs/httpd-error.log
[...]
[Sat Apr 27 06:52:52.817137 2024] [mpm_prefork:notice] [pid 6201] AH00163: Apache/2.4.29 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.1.14 configured -- resuming normal operations
[Sat Apr 27 06:52:52.817137 2024] [core:notice] [pid 6201] AH00094: Command line: '/usr/local/sbin/httpd -D NOHTTPACCEPT'

This took a while to track down. Eventually I added a directive to httpd.conf for some logging from the http2 module:

[... many lines not shown ...]

# Enable HTTP/2
LoadModule http2_module libexec/apache24/mod_http2.so
Protocols h2 h2c http/1.1
<IfModule http2_module>
        LogLevel http2:info
</IfModule>

# Set up PHP
LoadModule php7_module        libexec/apache24/libphp7.so

[... many lines not shown ...] 

Then I restarted the server and looked at the end of the error log. There had been a problem, but without the elevated logging Apache did not report it. See the error code AH10034 below.

# /usr/local/etc/rc.d/apache24 restart
Performing sanity check on apache24 configuration:
Syntax OK
Stopping apache24.
Waiting for PIDS: 7138.
Performing sanity check on apache24 configuration:
Syntax OK
Starting apache24.
# tail /var/www/logs/httpd-error.log
[...]
[Sat Apr 27 06:52:52.000130 2024] [mpm_prefork:notice] [pid 6956] AH00169: caught SIGTERM, shutting down
[Sat Apr 27 06:52:52.002498 2024] [http2:info] [pid 7138] AH03090: mod_http2 (v1.10.12, feats=CHPRIO+SHA256+INVHD+DWINS, nghttp2 1.29.0), initializing...
[Sat Apr 27 06:52:52.011375 2024] [http2:warn] [pid 7138] AH10034: The mpm module (prefork.c) is not supported by mod_http2. The mpm determines how things are processed in your server. HTTP/2 has more demands in this regard and the currently selected mpm will just not do. This is an advisory warning. Your server will continue to work, but the HTTP/2 protocol will be inactive.
[Sat Apr 27 06:52:52.217357 2024] [mpm_prefork:notice] [pid 7138] AH00163: Apache/2.4.29 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.1.14 configured -- resuming normal operations
[Sat Apr 27 06:52:52.293589 2024] [core:notice] [pid 7138] AH00094: Command line: '/usr/local/sbin/httpd -D NOHTTPACCEPT' 

Apache MultiProcessing Modules

Apache provides a variety of MPMs or MultiProcessing Modules. In general, one master or mother process running as root is started by the apache24 script. The privileged mother process opens the privileged TCP ports 80 and 443. It then starts a number of child processes. The child processes inherit the open ports and then call setuid() to change their user identity to the relatively unprivileged www user. Then they do the actual work of serving out data.

There are multiple MPMs to choose from. They work in slightly different ways and offer different advantages.

I had started with a copy of the default httpd.conf.sample and so the server was using the mod_mpm_prefork.so module. But that is not compatible with the mod_http2.so module. The server starts and provides all functionality, except it will not use HTTP/2.

The prefork module implements a non-threaded pre-forking server. A single control process launches child processes which listen for connections and serve them. It tries to always maintain a few idle server processes so clients do not need to wait for a new child server process to be forked.

The worker module implements a hybrid multi-process multi-threaded server. It can serve a large number of requests while using fewer system resources than the prefork module would require. A single control process launches child processes. Each child process creates a fixed number of server threads, plus a listener thread that listens for connections and passes each one to a server thread. Again, it tries to always maintain a pool of idle server threads, so clients do not need to wait for new threads or processes to be created.

The event module was based on the worker module, and it was created for Apache 2.4 because Apache 2.2 was significantly slower than Nginx. It uses several processes and several threads per process in an asynchronous event-based loop. This gives performance equal to or slightly better than event-based web servers.

I enabled the mod_mpm_event.so module, as that should provide the best performance. Here's what I put in httpd.conf:

[... many lines not shown ...]
LoadModule mpm_event_module libexec/apache24/mod_mpm_event.so
#LoadModule mpm_prefork_module libexec/apache24/mod_mpm_prefork.so
#LoadModule mpm_worker_module libexec/apache24/mod_mpm_worker.so
[... many lines not shown ...] 

Now Apache would not start, because the mod_php module is not compiled to be thread-safe:

# /usr/local/etc/rc.d/apache24 restart
Performing sanity check on apache24 configuration:
[Sat Apr 27 06:52:52.293589 2024] [php7:crit] [pid 7604:tid 34397577216] Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe.  You need to recompile PHP.
AH00013: Pre-configuration failed 

I wanted the higher-performance of HTTP/2 and multi-threaded web server processes, but I did not want the added maintenance work of compiling and installing my own PHP module. I wanted to use the OS packages so that I would get PHP module updates automatically when they became available.

The Solution

I want to use the event multiprocessing module, so I commented out the line loading libphp7.so.

I added package php73. It includes php-fpm, the PHP FastCGI Process Manager. It runs as a daemon, listening on a TCP socket for CGI requests. By default it listens on TCP port 9000 on localhost only.

Next, I added a line to /etc/rc.conf

php_fpm_enable=YES

I started php-fpm and verified that it is listening. It uses the same model of starting as root and spawning multiple unprivileged workers.

# /usr/local/etc/rc.d/php-fpm start
Performing sanity check on php-fpm configuration:
[14-Feb-2018 15:59:00] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful

Starting php_fpm.
# lsof  -i | egrep 'PID|php'
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
php-fpm 40778  www    0u  IPv4 0xfffff800100f9410      0t0  TCP localhost:9000 (LISTEN)
php-fpm 63636  www    0u  IPv4 0xfffff800100f9410      0t0  TCP localhost:9000 (LISTEN)
php-fpm 64754  www    0u  IPv4 0xfffff800100f9410      0t0  TCP localhost:9000 (LISTEN)
php-fpm 91117 root    7u  IPv4 0xfffff800100f9410      0t0  TCP localhost:9000 (LISTEN)

I need to make a few more changes, telling the server to use Index.html as the default file for a directory, and to handle all HTML files with PHP.

Use Index.html (and not index.* or *.htm)

An index file is used when the client requests a directory. For example, http://cromwell-intl.com/ will be treated as a request for the index file in the root directory of the web site. This requires the mod_dir module. Apache should load that module by default, but let's check.

Index.html is my site's standard index file name, versus Apache's default lower-case index.html. Here's what I changed in httpd.conf.

[... many lines not shown ...]
LoadModule dir_module libexec/apache24/mod_dir.so
[... many lines not shown ...]

# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
    ## DirectoryIndex index.html
    DirectoryIndex Index.html
</IfModule>

[... many lines not shown ...]

Treat All HTML Files As PHP

All of my pages use PHP to include standard headers and footers, to create microdata for search engine and social media indexing, and to load Google AdSense code blocks. Every page needs PHP. However, I have always named the files "*.html". So, all HTML files must be treated as PHP.

This is a UNIX-family operating system, where file name extensions don't matter. Until, of course, they do... The php-fpm daemon cares about file name extensions!

Edit /usr/local/etc/php-fpm.d/www.conf to tell it to accept files named both *.php and *.html. Without that, passing it a file named *.html results in a very simple page saying "Access denied".

[... many lines not shown ...]

; Limits the extensions of the main script FPM will allow to parse. This can
; prevent configuration mistakes on the web server side. You should only limit
; FPM to .php extensions to prevent malicious users to use other extensions to
; execute php code.
; Note: set an empty value to allow all extensions.
; Default Value: .php
;security.limit_extensions = .php .php3 .php4 .php5 .php7
security.limit_extensions = .php .html

[... many lines not shown ...]

Now I can tell Apache to pass all files named *.html to the PHP proxy. Putting all of this together, here are the changes and additions in httpd.conf:

[... many lines not shown ...]
LoadModule dir_module libexec/apache24/mod_dir.so
[... many lines not shown ...]
LoadModule mpm_event_module libexec/apache24/mod_mpm_event.so
#LoadModule mpm_prefork_module libexec/apache24/mod_mpm_prefork.so
#LoadModule mpm_worker_module libexec/apache24/mod_mpm_worker.so
[... many lines not shown ...]

# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
    ## DirectoryIndex index.html
    DirectoryIndex Index.html
</IfModule>

[... many lines not shown ...]

# Enable HTTP/2
LoadModule http2_module libexec/apache24/mod_http2.so
# Prefer HTTP/2 over TLS, then HTTP/2 without TLS, then HTTP/1.1, and finally HTTP/1.0
Protocols h2 h2c http/1.1
<IfModule http2_module>
        LogLevel http2:info
</IfModule>

# Set up PHP
LoadModule proxy_module libexec/apache24/mod_proxy.so
LoadModule proxy_http2_module libexec/apache24/mod_proxy_http2.so
LoadModule proxy_fcgi_module libexec/apache24/mod_proxy_fcgi.so
<FilesMatch \.html$>
	SetHandler "proxy:fcgi://127.0.0.1:9000"
</FilesMatch>

[... many lines not shown ...] 

The above will work as long as the clients don't ask for files that don't exist. In that case the client will get a simple "File not found" page, and a "Primary script unknown" entry appears in the error log. Add the following to the end of your .htaccess file to solve that problem. Don't duplicate the RewriteEngine line if it's already in the file:

# Do NOT duplicate the following line if it
# already exists earlier in the file.
RewriteEngine on

# If the client asks for a specific non-existent *.html file, rewrite it
# so it isn't passed to the PHP engine causing a "Primary script unknown"
# log entry and a plain "File not found" page.
RewriteCond %{REQUEST_FILENAME} \.html$
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-f
RewriteRule (.*) - [H=text/html] 

Restart apache24 and test it. This needs to include testing with requests for nonexistent directories and files.

You will find suggestions to use ProxyPassMatch instead of SetHandler in the above use of the PHP proxy. That does not work with .htaccess redirection, you get a simple "File not found" error for cases where it should have been redirected.

Now my PHP inclusion works, so my pages all get the standard header with automatically generated social media microdata, the standard footer, and the Google AdSense ads. The pages now look as they should! And, they should be served more efficiently with HTTP/2.

Testing HTTP/2

Let's make sure that HTTP/2 is working:

$ curl -I --http2 https://cromwell-intl.com/
HTTP/2 200 
date: Sat, 27 Apr 2024 06:52:52 GMT
server: Apache/2.4.29 (FreeBSD) OpenSSL/1.0.2k-freebsd
x-powered-by: PHP/7.3.2
content-type: text/html; charset=UTF-8 

That looks good!

Final Cleanup

Be careful! If you have installed an earlier mod_php7* package, then a package update will run a post-installation script that automatically re-inserts the troublesome libphp7.so line in your configuration file!

Remove the earlier mod_php7* package to be safe. It only contains the module and some license files:

$ pkg info -l mod_php71
mod_php71-7.1.17:
        /usr/local/libexec/apache24/libphp7.so
        /usr/local/share/licenses/mod_php71-7.1.17/LICENSE
        /usr/local/share/licenses/mod_php71-7.1.17/PHP301
        /usr/local/share/licenses/mod_php71-7.1.17/catalog.mk