[SOLVED] Traefik's router uses a non-existent resolver

Traefik Traefik v2
1

Hi y'all!
I've got an ansible configuration for the traefik docker container:

        - name: start traefik
          docker_container:
            name: reverse-proxy
            image: traefik:v2.0
            command:
              - "--log.level=DEBUG"
              - "--api"
              - "--accesslog"
              - "--providers.file.filename={{ traefik.docker_dir }}{{ traefik.conf_file }}"
              - "--providers.docker=true"
              - "--providers.docker.exposedbydefault=false"
              - "--entrypoints.http.address=:80"
              - "--entrypoints.https.address=:443"
              - "--certificatesresolvers.{{ env }}.acme.httpchallenge=true"
              - "--certificatesresolvers.{{ env }}.acme.httpchallenge.entrypoint=http"
              #- "--certificatesresolvers.{{ env }}.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
              - "--certificatesresolvers.{{ env }}.acme.email=letsencrypt@mydomain.io."
              - "--certificatesresolvers.{{ env }}.acme.storage={{ traefik.docker_dir }}{{ traefik.acme_file }}"
            restart_policy: always
            network_mode: host
            keep_volumes: yes
            env:
              AWS_ACCESS_KEY_ID: "{{ route53_accesskey }}"
              AWS_SECRET_ACCESS_KEY: "{{ route53_secretkey }}"
              AWS_HOSTED_ZONE_ID: "{{ route53_hostedzone }}"
              AWS_REGION: "{{ aws_region }}"
            ports:
              - "80:80"     # The HTTP port
              - "443:443"   # The HTTPS port
              - "8080:8080" # The Web UI (enabled by --api)
            volumes:
              - /etc/localtime:/etc/localtime:ro
              - /var/run/docker.sock:/var/run/docker.sock
              - "{{ traefik.host_dir }}{{ traefik.conf_file }}:{{ traefik.docker_dir }}{{ traefik.conf_file }}"
              - "{{ traefik.host_dir }}{{ traefik.acme_file }}:{{ traefik.docker_dir }}{{ traefik.acme_file }}"
            labels:
              # Dashboard
              traefik.http.routers.traefik.rule: "Host(`rproxy.s.{{ env }}.mydomain.io`)"
              traefik.http.routers.traefik.service: "api@internal"
              traefik.http.routers.traefik.middlewares: "admin"
              traefik.http.routers.traefik.tls.certresolver: "{{ env }}"
              traefik.http.routers.traefik.entrypoints: "https"
              traefik.http.middlewares.admin.basicauth.users: "admin:ch1environment"
            log_driver: awslogs
            log_options:
              awslogs-region: "{{ aws_region }}"
              awslogs-group: "/mydomain/{{ env }}/traefik"
              awslogs-stream: "mydomain-traefik-{{ tag }}"

...and my application:

- name: action on present container
  docker_container:
    name: "myapp-{{ tag }}"
    image: "{{ docker_ecr }}/mydomain/myapp:{{tag}}"
    state: started
    detach: true
    restart_policy: always
    network_mode: host
    ports:
      - "{{ app_port }}:{{ app_port }}"
    env:
      AWS_ACCESS_KEY_ID: "{{ my_aws_access_key }}"
      AWS_SECRET_ACCESS_KEY: "{{ my_aws_secret_key }}"

    volumes:
      - "/home/{{ user }}/config:{{ config_path_container }}"
      - "/home/{{ user }}/tmp:/tmp"

    # this is documented here: https://docs.traefik.io/configuration/backends/docker/
    labels:
      traefik.enable: "true"
      traefik.http.services.rhttp.loadbalancer.server.port: "{{ app_port | string }}"
      traefik.http.routers.rhttp.entrypoints: "http"
      traefik.http.routers.rhttp.rule: "HostRegexp({{ traefik.frontend_rules.myapp }} {{ traefik.frontend_rules.other | default(None) }})"

      traefik.http.routers.rhttps.entrypoints: "https"
      traefik.http.routers.rhttps.rule: "HostRegexp({{ traefik.frontend_rules.myapp }} {{ traefik.frontend_rules.other | default(None) }})"
      traefik.http.routers.rhttps.tls: "true"
      traefik.http.routers.rhttps.tls.certresolver: "{{ env }}"
      # traefik.http.routers.rhttps.tls.options: "ch1secure@file"
      # traefik.http.routers.rhttps.tls.domains[0].main: "{{ acme_domains_main }}"
      # traefik.http.routers.rhttps.tls.domains[0].sans: "{{ acme_domains_sans|join(', ') }}"

      #traefik.backend: "docker-{{ env }}"
      #traefik.enable: "true"
      #traefik.port: "{{ app_port | string }}"
      #traefik.frontend.rule: "HostRegexp: {{ traefik.frontend_rules.myapp }} {{ traefik.frontend_rules.other | default(None) }}"

      # enable below for LOCAL testing:
      #traefik.frontend.rule: "Host:localhost"

    #log_driver: journald
    log_driver: awslogs
    log_options:
      awslogs-region: "{{ aws_region }}"
      awslogs-group: "/myapp/{{ env }}"
      awslogs-stream: "myapp-{{ tag }}"

with the static config file:

[tls.options]
  [tls.options.ch1secure]
    minVersion = "VersionTLS12"
    cipherSuites = [
      "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
      "TLS_RSA_WITH_RC4_128_SHA"
      "TLS_RSA_WITH_3DES_EDE_CBC_SHA"
      "TLS_RSA_WITH_AES_128_CBC_SHA"
      "TLS_RSA_WITH_AES_256_CBC_SHA"
      "TLS_RSA_WITH_AES_128_CBC_SHA256"
      "TLS_RSA_WITH_AES_128_GCM_SHA256"
      "TLS_RSA_WITH_AES_256_GCM_SHA384"
      "TLS_ECDHE_ECDSA_WITH_RC4_128_SHA"
      "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA"
      "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA"
      "TLS_ECDHE_RSA_WITH_RC4_128_SHA"
      "TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA"
      "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA"
      "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA"
      "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"
      "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256"
      "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
      "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
      "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
      "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
      "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305"
      "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305"
    ]

Why does traefik logs say it cannot get ACME account upon start?

time="2019-11-26T18:42:44Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"http\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}},\"https\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":15000000000},\"file\":{\"watch\":true,\"filename\":\"/etc/traefik.toml\"}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"accessLog\":{\"format\":\"common\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"drop\"}}},\"certificatesResolvers\":{\"ch1test\":{\"acme\":{\"email\":\"letsencrypt@deep-impact.ch\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"/etc/acme.json\",\"keyType\":\"RSA4096\",\"httpChallenge\":{\"entryPoint\":\"http\"}}}}}"
time="2019-11-26T18:42:44Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/v2.0/contributing/data-collection/\n"
time="2019-11-26T18:42:44Z" level=error msg="Unable to add ACME provider to the providers list: unable to get ACME account: json: cannot unmarshal array into Go value of type acme.StoredData"
...
time="2019-11-26T18:42:44Z" level=error msg="the router rhttps uses a non-existent resolver: ch1test"
2

ok, solved this thing by myself. The culprit was the acme.json file that remained from the previous installation, and that apparently does not conform to the newest 2.0 traefik spec. As soon as I deleted it and restarted my traefik:v2.0 container - everything worked like a charm, new certs were released and my servers went up; yay!

@admins: PLEASE DOCUMENT THIS!
This was not obvious at all, the only pointer that suggested me what the issue could be was the fact that it was a json parser that failed, not yaml/toml, which is what I use everywhere.

3

It's an open source projet, feel free to improve the migration guide: https://docs.traefik.io/v2.0/migration/v1-to-v2/#acme-letsencrypt