Skip to content

Installing Traefik Enterprise on Docker Swarm

This page guides you through the installation of Traefik Enterprise on Docker CE Swarm Mode.

Docker Knowledge

Assistance with configuring or setting up a Docker Swarm Mode cluster is not included in this guide. If you need more information about Docker and Swarm, start with the following resources:

Requirements

  • A Docker Swarm cluster
    • Supported versions: 17.06 to 19.03
    • At least 1 manager swarm node and 1 worker swarm node
    • The docker CLI tool properly configured to communicate with the swarm cluster. teectl relies on the DOCKER_HOST, DOCKER_CERT_PATH and DOCKER_TLS_VERIFY environment variables to communicate with the cluster.
  • Controller containers can reach https://v4.license.containous.cloud.
  • The teectl binary is installed, for cluster management

Installing on DockerCE

This guide provides a detailed explanation of the installation process. For a lighter approach, please refer to the getting started guide.

Generating the teectl Configuration

To start the installation, teectl must first be configured:

teectl setup --swarm --swarm.hosts="node1,node2,node3"

This command creates a new teectl configuration under ~/.config/traefikee/default.yaml and generates a bundle.zip which carries a set of needed assets for the installation.

To further configure the teectl setup, please refer to the reference documentation.

About the --swarm.hosts option

teectl communicates with the cluster through the network. To do so it needs to know where the cluster API is served. This option accepts a comma separated list of ip addresses or domain names that serve the API.

By default, the API is exposed by the controllers on the swarm routing mesh with the port 55055, so the host to use is potentially one or more node addresses of the swarm cluster.

Sensitive information

The generated bundle.zip carries sensitive information and is required to generate installation manifests. It should be stored in a secure place.

Managing Multiple Clusters from a Single Workstation

To manage multiple clusters from a single workstation, please refer to the Managing Multiple Clusters guide

Writing the bundle at a different path

teectl setup comes with an --output option that specifies the output path in which to create the bundle.

Generating the Docker Compose Files

Traefik Enterprise is typically installed in its own stack. To create the stack resources, teectl can generate docker compose v3 files for the controllers and proxies.

Controllers

# Generates a docker-compose file to deploy 3 controllers and store it in a file called controllers.yaml
teectl setup gen --license="${TRAEFIKEE_LICENSE}" --controllers=3 > controllers.yaml

To further configure the setup gen command, please see the reference documentation.

Filesystem assets

The generated compose file refers to the bundle generated by teectl setup. By default it is located at ./bundle.zip but it can be configured using the --bundle option.

Stateful instances

Controllers persist their state to a volume to ensure tolerance to container restart.

To be compatible with the greatest amount of environments, Traefik Enterprise relies by default on a local volume and assigns a replica to a specific node using a placement constraint.

Using global volumes is possible. To do so please change the volume driver on the generated compose file before deploying.

One service per controller

The generated docker compose file defines one controller per service. As it might seem counterintuitive, it is actually done for multiple reasons:

  • To operate properly, controllers need to maintain their network identifiers (network names in this case) across container restarts. This is the only way to do so in swarm at this moment.
  • As explained above, controller instances are assigned to a node using a placement constraint. To do so it is required to have one service per instance.

Never scale a controller service

As explained above, controllers rely on one and only one instance per service.

Scaling a controller service would have an undefined behaviour and possibly severe consequences.

What is the api_proxy service?

As mentioned above, teectl communicates with the controllers using the port 55055 of the ingress network. However, one service is created per replica of the controllers so it is impossible to expose the port 55055 of all the controllers to the ingress network.

To achieve this, a small TCP proxy service is started alongside the controllers and exposes the port 55055 to the ingress network and load-balances all received connections between the controllers.

On DockerEE

To enable UCP, please refer to the DockerEE guide below.

Proxies

# Generates a docker-compose file to deploy 2 proxies.
teectl setup gen --license=${TRAEFIKEE_LICENSE} --proxies=2 > proxies.yaml

To further configure the setup gen command, please see the reference documentation.

About networking

Two networks are declared in the generated compose files to guarantee network isolation:

  • The control network is dedicated to the communication between the controllers and the proxies.
  • The ingress network is dedicated to the communication between proxies and application services.

All the services that need to be exposed have to be connected to the ingress network and declared as external. An example is shown below:

# If the cluster is deployed under the stack `traefikee`
networks:
  traefikee_ingress:
    external: true

The swarm network discovery is also available to automatically connect proxies to networks where there are applications to expose.

Customizing the Docker Compose Files (Optional)

There is a section dedicated to customizing the compose file, with some common scenarios that requires it.

The following aspects should be taken in account:

  • Memory/CPU Resevations and Limits: Both controllers and proxies come with enforced resource limits. The defaults are voluntarily kept large to fit most of use cases. They can be adjusted to better fit the targeted installation.
  • Network Configuration and Exposed Ports: By default, the proxies service is configured to expose the ports 80 and 443 to the swarm ingress network. This setting has to be adjusted according to the target installation. If network performance is needed, please consider bypassing the routing mesh.
  • Volumes: Local volumes are being used by default in order to fit most installations. If a global volume driver is available on the targeted swarm cluster, it can be used instead. This would remove the placement constraints on the controllers, providing more flexibility. However, be careful with the I/O performance of these volumes as it is critical for the stability of the cluster.
  • Placement constraints: By default, proxy services are configured to be deployed on workers only. If the workload is allowed on manager nodes or only certain swarm nodes are exposed to the internet then the placement constraint can be adjusted. However, unless a global volume is being used for the controller volumes, please do not remove the placement constraints on the controller services.
  • Labels: To expose the dashboard, labels on the proxies service is required. However, please make sure not to remove the com.containous.traefikee.component label.
  • Environment variables: To use Datadog or ACME DNS Challenge for instance, environment variables should be configured either on proxies or controllers.

Rootless Image

A rootless image is available. For more information, please refer to this section.

Healthcheck

We don't provide a swarm compliant health check at this moment.

Deploying the Cluster

Deploy the controllers before the proxies

The Proxies docker compose files depends on resources that are created by the controllers once they are running. To be able to deploy proxies, please first deploy the controllers and make sure they are running.

Deploy the controllers and the proxies under the same stack

Docker Stack provides scoped resources on a stack. If the controllers and proxies services are deployed under different stacks, they won't be able to reach each other.

Once the compose files are reviewed and ready, the next step is to deploy the cluster:

First, deploy the controllers:

docker stack deploy -c ./controllers.yaml traefikee
Creating secret traefikee_bundle
Creating network traefikee_control
Creating service traefikee_api-proxies
Creating service traefikee_controller-0
Creating service traefikee_controller-1
Creating service traefikee_controller-2

To monitor the controller boot process, use:

docker service ls
ID                  NAME                     MODE                REPLICAS            IMAGE                            PORTS
kdekcc86kf5s        traefikee_api-proxies    replicated          2/2                 traefik/traefikee:v2.12.0         *:55055->55055/tcp
zs1q0yzs3ni7        traefikee_controller-0   replicated          1/1                 traefik/traefikee:v2.12.0
06qn1eqb8e4a        traefikee_controller-1   replicated          1/1                 traefik/traefikee:v2.12.0
p1k7pwdyhwyc        traefikee_controller-2   replicated          1/1                 traefik/traefikee:v2.12.0

Then, once the controllers are ready (all replicas are running and stable), deploy the proxies:

docker stack deploy -c ./proxies.yaml traefikee
Creating network traefikee_ingress
Creating service traefikee_proxies

To check that the proxies are running:

docker service ls
ID                  NAME                     MODE                REPLICAS            IMAGE                            PORTS
[...]
p1k7pwdyhwyc        traefikee_proxies        replicated          2/2                 traefik/traefikee:v2.12.0         *:80->80/tcp *:443->443/tcp

When all the pods are running, ensure that the cluster is properly installed using teectl get nodes:

teectl get nodes
ID                         NAME           STATUS  ROLE
7tw8nppypruy0iqgfyqf2jvz5  controller-1   Ready   Controller
i8lmlridgrihknvfpp9a4ckyo  78877d77d9     Ready   Proxy
j8e9luok9ksoj35v80mjasies  7d9bf4w238     Ready   Proxy
lufljo7vjyqptmsczfcxb4ljm  controller-2   Ready   Controller
rb74v1d548petedlqny2n76cm  controller-0   Ready   Controller (Leader)

Applying a Static Configuration

A cluster is created without any default configuration. To allow the controller to listen to a provider and proxies to manage incoming traffic, it is necessary to apply a static configuration.

The following example defines two entry-points (listening on 80 and 443), and enables the docker provider with swarm mode enabled.

---
providers:
  docker:
    swarmMode: true

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"
[providers]
  [providers.docker]
    swarmMode=true

[entryPoints]
  [entryPoints.web]
    address = ":80"
  [entryPoints.websecure]
    address = ":443"

Apply the configuration using the following command:

teectl apply --file="./static.yaml"
teectl apply --file="./static.toml"
On DockerEE

To enable UCP, please refer to the DockerEE guide below.

At any time, it is possible to get the currently applied static configuration of a cluster using:

teectl get static-config
---
configuration:
  global:
    checkNewVersion: true
  serversTransport:
    maxIdleConnsPerHost: 200
  entryPoints:
    web:
      address: :80
      transport:
        lifeCycle:
          graceTimeOut: 10s
        respondingTimeouts:
          idleTimeout: 3m0s
      forwardedHeaders: {}
    websecure:
      address: :443
      transport:
        lifeCycle:
          graceTimeOut: 10s
        respondingTimeouts:
          idleTimeout: 3m0s
      forwardedHeaders: {}
  providers:
    providersThrottleDuration: 2s
    docker:
      swarmmode: true
cluster:
  cleanup:
    gracePeriod: 1h0m0s

Deploying a Test Service

To validate your setup, it is possible to deploy a test application using the following docker compose file:

version: '3.4'
networks:
  traefikee_ingress:
    external: true

services:
  whoami:
    image: traefik/whoami:v1.6.1
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - "traefik.http.routers.whoami.rule=Path(`/whoami`)"
        - "traefik.http.services.whoami.loadbalancer.server.port=80"
    networks:
      - traefikee_ingress

Then, access the application:

curl <your-cluster-hostname-or-ip>/whoami
Hostname: 06c13be38c61
IP: 127.0.0.1
IP: 172.17.0.3
RemoteAddr: 172.17.0.1:33684
GET / HTTP/1.1
Host: localhost
User-Agent: curl/7.68.0
Accept: */*

Congratulations! Your Traefik Enterprise cluster is ready.

What's next?

Now that the cluster is ready, we recommend reading the various operating guides to dive into the features that Traefik Enterprise provides.

We also recommend getting familiar with the various concepts of Traefik Enterprise.

Installing on DockerEE

Traefik Enterprise supports leveraging the Universal Control Plane to access the Docker API which brings the following benefits:

  • Access control on the Docker API: UCP provides access control to the Docker API, improving the security of the installation.
  • Workload on workers only: Using UCP removes the need of accessing the Docker socket, removing the constraint of executing controllers on managers.

To install Traefik Enterprise with UCP support enabled follow the DockerCE installation procedure above except the two following steps.

Generating the Controllers Docker Compose File

To enable UCP support, pass the --swarm.ucpbundle option, which is a path to an UCP client bundle previously generated.

teectl setup gen --license="${TRAEFIKEE_LICENSE}" --controllers=3 --swarm.ucpbundle="./path/to/bundle.zip" > controllers.yaml
Allowing execution on managers

If it is needed to allow executing controllers on swarm managers, pass --swarm.allowmanagers option to setup gen.

Applying a Static Configuration

To support access to the Docker API through UCP, the docker provider has to be configured properly in the static configuration:

providers:
  docker:
    endpoint: "tcp://address-of-the-UCP-api:port"
    swarmMode: true
    tls:
      ca: "/var/run/ucp/ca.pem"
      key: "/var/run/ucp/key.pem"
      cert: "/var/run/ucp/cert.pem"
[providers]
  [providers.docker]
    endpoint = "tcp://address-of-the-UCP-api:port"
    swarmMode = true
    [providers.docker.tls]
      ca = "/var/run/ucp/ca.pem"
      key = "/var/run/ucp/key.pem"
      cert = "/var/run/ucp/cert.pem"

License Monitoring

When a Traefik Enterprise controller starts for the first time, it checks the license validity.

If the license is valid, another check is done once every 24 hours.

If the controller can't communicate with the license server, a 72-day grace period starts to recover from this situation.

Once the grace period is over, the controller stops updating the proxies configuration.

Please look at the FAQ to know how to implement the license monitoring.