In this article:
Published March 10 2018
If you're serving websites (or APIs) with HAProxy in front, and you're looking for how to get those sites set up with https, for free, then you've come to the right place.
We'll start with a primer on using certbot to mostly automate issuing fully valid and free SSL/TLS certificates, and then configure HAProxy to use them.
I am using Docker with a Docker network to run my apps. This way I don't have to expose any ports, and services can talk via their container names (which double as host names). To follow along, create a Docker network like so:
docker network create myworld
certbot is a tool from the EFF that automates most of the process of acquiring a free SSL/TLS certificate from Let's Encrypt. You can probably install it with your package manager, but I had some poor luck using the one from EPEL on Centos 7. If your server has Docker, I recommend using the official certbot image for a trouble-free experience.
To request a new certificate for a domain, you must prove that you are its owner. This done with an ACME challenge, which consists of serving some files over http://yourdomain.com/.well-known/acme-challenge/.
Certbot comes with a bunch of plugins that can automate this completely for you using Apache httpd, nginx, and others. If you have HAProxy in front you'll need to do some work though, which suits me fine, as I like understanding roughly what's going on.
Using the webroot plugin
allows you to fully control the web server - just point certbot at a directory
that is served by your web server, and it will complete the ACME challenge by
files in WEBROOT/.well-known/acme-challenge/. I will use nginx to serve these
files:
docker run \
--name static-http \
-v /home/deploy/letsencrypt:/usr/share/nginx/html \
--restart always \
--network myworld \
nginx
To expose the files we will configure the nginx server as a backend for HAProxy.
NB! This only makes sense if you intend to use HAProxy for other things. If you only have one static site, you might as well use nginx directly and forego the additional setup.
Here's the frontend and backend for haproxy.cfg (defaults, globals, and other
sections omitted):
frontend http-in
bind *:80
compression algo gzip
compression type text/html text/plain text/javascript application/javascript application/xml text/css
option accept-invalid-http-request
acl is_well_known path_beg -i /.well-known
use_backend letsencrypt if is_well_known
backend letsencrypt
mode http
balance roundrobin
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
server static-http static-http
With this file in /home/deploy/haproxy/haproxy.cfg, run the HAProxy Docker
container like so:
docker run \
--name load-balancer \
-p 80:80 \
--restart always \
-v /home/deploy/haproxy:/config \
--network myworld \
haproxy:1.7 \
haproxy -f /config/haproxy.cfg
Because the container is in the same Docker network as the nginx container, it
can reach it over the hostname static-http. Exposing HAProxy's port 80 on the
host's port 80 creates a link to the outer world. If your site's DNS is
configured correctly, you should now be able to reach files in
/home/deploy/letsencrypt/.well-known from http://yoursite.com/.well-known.
Now we have enough infrastructure to let certbot conduct the ACME challenge on our behalf:
mkdir /etc/letsencrypt
docker run -i --rm --name certbot \
-v /etc/letsencrypt:/etc/letsencrypt \
-v /home/deploy/letsencrypt:/webroot \
certbot/certbot certonly \
--webroot \
-w /webroot \
-d mysite.com \
--email christian@cjohansen.no \
--non-interactive \
--agree-tos
If all goes well, you should now have a freshly issued SSL/TLS certificate for
your site in /etc/letsencrypt/live/mysite.com.
To use your newly acquired SSL certificates with HAProxy, you must combine their private keys and certificate:
mkdir /etc/letsencrypt/haproxy
cat /etc/letsencrypt/live/site-a.com/privkey.pem \
/etc/letsencrypt/live/site-a.com/fullchain.pem \
> /etc/letsencrypt/haproxy/site-a.com.pem
HAProxy supports Server Name Indication (SNI), which allows you to serve multiple HTTPS websites from the same IP address by including the hostname in the TLS handshake. Just tell HAProxy about all your certificates, and it'll figure out the rest.
If you have more than one certificate, you can concatenate them all in one go like this:
function cat-cert() {
dir="/etc/letsencrypt/live/$1"
cat "$dir/privkey.pem" "$dir/fullchain.pem" > "/etc/letsencrypt/haproxy/$1.pem"
}
for dir in /etc/letsencrypt/live/*; do
cat-cert $(basename "$dir")
done
Let's say you provisioned certificates for two sites, site-a.com and
site-b.com, and concatenated them into /etc/letsencrypt/haproxy as suggested
above. Assuming the certificate directory is exposed as the volume /ssl-certs
in the HAProxy container, you can create an HTTPS frontend as such:
frontend https-in
bind *:443 ssl crt /ssl-certs/site-a.com.pem crt /ssl-certs/site-b.com.pem
compression algo gzip
compression type text/html text/plain text/javascript application/javascript application/xml text/css
option accept-invalid-http-request
use_backend site-a if { hdr_end(host) -i site-a.com }
use_backend site-b if { hdr_end(host) -i site-b.com }
With this configuration in place, restart HAProxy with Docker the following way:
docker run \
--name load-balancer \
-p 80:80 \
--restart always \
-v /etc/letsencrypt/haproxy:/ssl-certs \
-v /home/deploy/haproxy:/config \
--network myworld \
haproxy:1.7 \
haproxy -f /config/haproxy.cfg
The only difference from before is the added /ssl-certs volume.
Now that our sites have SSL certificates, we want to serve all traffic over HTTPS. One way to do this is to redirect all attempts at HTTP to HTTPS. As we do this, keep in mind that the ACME challenge needs to be performed over HTTP, so there should be an exception for those URLs:
frontend http-in
bind *:80
compression algo gzip
compression type text/html text/plain text/javascript application/javascript application/xml text/css
option accept-invalid-http-request
acl is_well_known path_beg -i /.well-known
# Add this line
redirect scheme https code 301 if !is_well_known !{ ssl_fc }
use_backend letsencrypt if is_well_known
HAProxy processes redirects before backend assignment, and will issue a warning in the logs if you place them out of order. It's not technically an error, but it is potentially confusing.
HSTS, or HTTP Strict Transport Security, is security mechanism that avoids some phishing scenarios by informing the browser to never access the site over plain HTTP. Add this header to all responses going out of HAProxy like so (include it in either your frontend or backend configuration):
http-response set-header Strict-Transport-Security "max-age=16000000; includeSubDomains; preload;"
Let's Encrypt certificates are valid for 90 days. To ensure your site stays well-configured, you should renew certificates in a cronjob. The renewal process goes like this:
certbot renewSIGHUPHere's a script you can run from a cronjob that does just that, assuming the same directories as used above:
#!/bin/bash
set -e
echo "$(date) About to renew certificates" >> /var/log/letsencrypt-renew.log
/usr/bin/docker run \
-i \
--rm \
--name certbot \
-v /etc/letsencrypt:/etc/letsencrypt \
-v /home/deploy/letsencrypt:/webroot \
certbot/certbot \
renew -w /webroot
echo "$(date) Cat certificates" >> /var/log/letsencrypt-renew.log
function cat-cert() {
dir="/etc/letsencrypt/live/$1"
cat "$dir/privkey.pem" "$dir/fullchain.pem" > "/etc/letsencrypt/haproxy/$1.pem"
}
for dir in /etc/letsencrypt/live/*; do
cat-cert $(basename "$dir")
done
echo "$(date) Reload haproxy" >> /var/log/letsencrypt-renew.log
/usr/bin/docker kill -s HUP load-balancer
echo "$(date) Done" >> /var/log/letsencrypt-renew.log
