At Kontena, we're pretty proud of the loadbalancing solution we've baked into the platform almost from the inception of the project. Loadbalancing is something that pretty much everyone needs in their containerized environments, thus in Kontena it's really something baked in to the platform itself.

Kontena loadbalancer is going through some changes, in this blog I'll dive into what's happening with it.

No, don't worry, we're still keeping it super simple to use. And fully backwards-compatible with existing deployments.

New code base

When we initially developed the loadbalancer automation, we implemented the dynamic configuration management using confd and some scripting. Confd is a fantastic tool for generating different types of config files but as with many other templating "languages" it also has some limitations. In our case, some of the conditionals used when generating the HAProxy config file ended up being almost unreadable. So we're moving the config management to a new code-base.

No big surprise that we're using Ruby also for this bit. This allows us to have more logic baked in the configuration generation process and more importantly, it allows us to write all the code so that we can actually also add some tests to it. We already had pretty decent "integration" tests written with bats that actually made it possible to even change the code-base. With these bats tests we were able to make sure the change does not actually break anything.

We're still keeping the "old" version around with kontena/lb:1.0 just in case someone needs to rollback. But there's no planned maintenance for that.

Newer HAProxy

We're not getting rid of the trusted work-horse HAProxy, it's well proven technology and has served us and our users really well. We're however upgrading HAProxy to version 1.8 which offers some new features and more control over the reloads for example.

HTTP/2

With HAproxy 1.8 it finally starts to support HTTP/2 protocol. So quite naturally, we're also adding support for it in the default configuration we bake into it. What this means is that your apps can now support HTTP/2 clients out-of-box when they are deployed using the Kontena Loadbalancer.

HTTP/2 is already in action at our main website:

In general HTTP/2 should make client apps see less latency thanks to things like request pipelining, multiplexing.

Zero downtime reloads

In the past, we got pretty close to zero downtime during deploys. This was achieved using the HAProxy -sf option. It told HAProxy to send a "finish" signal to old worker processes to let them gracefully finish up what they were doing before going away. This however still might cause a few requests during a deployment, during which we have to reload HAProxy configs, to be broken. Newer HAProxy comes with a feature where the reloaded HAProxy can actually "transfer" the connection from old processes to new ones without dropping any new connections.

Getting to true zero-downtime naturally needs some help from your application. Firstly, it must be running on more than a single instance as Kontena will first remove the instance that needs to be re-created. Secondly, your application should also support graceful shutdown. Graceful shutdown here means that upon receiving SIGTERM, or any other signal configured with stop_signal, it should not accept any new connections. That also means that the Kontena LB will drop it from the healthy backends and not forward any new connections for it. By default Kontena gives 10 seconds for the app to shutdown gracefully, but it can be controlled with stop_grace_period option.

Multiple virtual paths

Sometimes it would be nice to have one single app to respond to multiple different paths in the loadbalancer. For this we've added support for adding multiple paths as a comma-separated list. So you could do something like:

environment:
  KONTENA_LB_VIRTUAL_PATH:/foo,/bar

Now both http://.../foo and http://.../bar requests will be forwarded to your single service.

Multiple SSL envs

In the past the only way to "inject" SSL certs was through a single SSL_CERT environment variable. In cases where many certificates were configured this actually caused some problems as there are some limits on how big the environment variables can be. See this issue for more details. The new LB supports reading certificates from any SSL_CERT_* variables. Basically this means that you can now easily put each certificate in its own env variable to mitigate the env size limit.

services:
  lb:
    image: kontena/lb:latest
    ports:
      - 443:443
    certificates:
      - subject: www.example.com
        type: env
        name: SSL_CERT_www.example.com
      - subject: test.example.com
        type: env
        name: SSL_CERT_test.example.com

Summary

The new codebase allow us to ship new features faster and safer on the Kontena Loadbalancer. With the new HAProxy 1.8 version we are finally able to support HTTP/2 out-of-the-box which our users have been asking already for a long time. And finally we also get true zero-downtime deployments with the connection transfer mechanism. Even with all these additions, configuring loadbalancing for your apps is as simple as it was before. :)


Image credits: Rock, stone, balance and beach, by Jeremy Thomas

About Kontena

Kontena provides the most easy-to-use, fully integrated solution for DevOps and software development teams to deploy, run, monitor and operate containers on the cloud. The underlying Kontena Platform technology is open source and available under Apache 2.0 license. It is used by hundreds of startups and software development teams working for some of the biggest enterprises in the world. www.kontena.io