Unable to find the IP address for the container : the server is ignored

Traefik Traefik v1
1

Do you want to request a feature or report a bug?

Bug

What did you do?

I get this error in log :

What did you expect to see?

Not getting this error and my api working

What did you see instead?

I see this :

time="2019-10-22T08:26:51Z" level=warning msg="unable to find the IP address for the container \"/am-api-test\": the server is ignored"
time="2019-10-22T08:26:51Z" level=error msg="undefined backend 'backend-am-api-test' for frontend frontend-Host-test-am-api-alexyfondeville-com-0. Skipping frontend frontend-Host-test-am-api-alexyfondeville-com-0..."

After that, my backend up and down immediatly

Output of traefik version: (What version of Traefik are you using?)

Version:      v1.7.6
Codename:     maroilles
Go version:   go1.11.3
Built:        2018-12-14_06:43:37AM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

# defaultEntryPoints must be at the top
# because it should not be in any table below
defaultEntryPoints = ["http", "https"]

logLevel = "INFO"

# Entrypoints, http and https
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
  [entryPoints.traefik]
  address=":8080"
    [entryPoints.traefik.auth]
    headerField = "X-WebAuth-User"
      [entryPoints.traefik.auth.basic]
      # touch /auth
      # htpasswd -c ./auth afondevi
      # cat auth
      users = [
          "user:XXXXXXXXXXXXXXXXXX",
      ]

# Enable ACME (Let's Encrypt): automatic SSL
[acme]
email = "myEmail@gmail.com"
storage = "acme.json"
entryPoint = "https"
    [acme.dnsChallenge]
    provider = "ovh"
    delayBeforeCheck = 10

  [[acme.domains]]
    main = "*.alexyfondeville.com"
    sans = [
        "sub.sub.domain.com",
        "sub2.sub2.domain.com"
    ]

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "alexyfondeville.com"
watch = true
exposedbydefault = false
network = "web"

[api]
  entryPoint = "traefik"
  dashboard = true
  debug = true

traefik.yml

version: '3'

services:
  traefik:
    image: traefik:1.7.6-alpine
    container_name: traefik
    environment:
      - OVH_ENDPOINT=XXXXX
      - OVH_APPLICATION_KEY=XXXXX
      - OVH_APPLICATION_SECRET=XXXXX
      - OVH_CONSUMER_KEY=XXXXX
    ports:
      - "80:80"
      - "443:443"
    networks:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
      - ./traefik.toml:/traefik.toml
      - ./acme.json:/acme.json
    labels:
      - "traefik.enable=true"
      - "traefik.entryPoint=traefik"
      - "traefik.frontend.rule=Host:traefik.domain.com"
      - "traefik.port=8080"
    restart: always

networks:
  web:
    external: true

app file (docker-compose.yml)

version: '3'

services:
  am-app:
    image: registry.gitlab.com/afondevi_projects/am/am-app:${AM_APP_VERSION}
    container_name: ${CONTAINER_NAME}
    networks:
      - web
    labels:
      - "traefik.enable=true"
      - "traefik.entryPoint=https"
      - "traefik.frontend.rule=Host:${SUB_DOMAIN}.domain.com"
      - "traefik.docker.network=web"
    restart: always

networks:
  web:
    external: true

api project (docker-compose.yml)

version: '3'

services:
  am-api:
    image: registry.gitlab.com/afondevi_projects/am/am-api:${TAG_VERSION}
    command: /app/node_modules/.bin/node index.js
    container_name: ${API_CONTAINER_NAME}
    networks:
      - web
      - internal
    labels:
      - "traefik.enable=true"
      - "traefik.entryPoint=https"
      - "traefik.backend=${API_CONTAINER_NAME}"
      - "traefik.port=${API_PORT}"
      - "traefik.frontend.rule=Host:${API_SUB_DOMAIN}.domain.com"
      - "traefik.docker.network=web"
    depends_on:
      - am-api-db
    restart: always

  am-api-db:
    image: mysql:latest
    container_name: ${DB_CONTAINER_NAME}
    environment:
      MYSQL_ROOT_PASSWORD:
    expose:
      - '3306'
    networks:
      - internal
    volumes:
      - /srv/databases/am/${ENVIRONEMENT}:/var/lib/mysql
    labels:
      - "traefik.enable=false"
    restart: always

networks:
  web:
    external: true
  internal:
    external: false

If applicable, please paste the log output in DEBUG level (--log.level=DEBUG switch)

time="2019-10-22T08:40:43Z" level=debug msg="allLabelsmap[:map[traefik.docker.network:web traefik.enable:true traefik.frontend.rule:Host:test.am-api.alexyfondeville.com traefik.backend:am-api-test traefik.entryPoint:https traefik.port:7041]]"
time="2019-10-22T08:40:43Z" level=debug msg="originLabelsmap[com.docker.compose.container-number:1 com.docker.compose.service:am-app maintainer:NGINX Docker Maintainers <docker-maint@nginx.com> traefik.docker.network:web traefik.enable:true traefik.entryPoint:https traefik.frontend.rule:Host:test.am.alexyfondeville.com com.docker.compose.config-hash:d99bc4a093e4c3482ee6dfdc88b7ebc635238e854f64aa8b5ea2b5163aeb47d3 com.docker.compose.oneoff:False com.docker.compose.project:am-app-test com.docker.compose.slug:f750fff3d92b9cca8eb35d4575ee4db23a9d144deaa9c60a272b5d641e8e000 com.docker.compose.version:1.23.1]"
time="2019-10-22T08:40:43Z" level=debug msg="allLabelsmap[:map[traefik.frontend.rule:Host:test.am.alexyfondeville.com traefik.docker.network:web traefik.enable:true traefik.entryPoint:https]]"
time="2019-10-22T08:40:43Z" level=warning msg="unable to find the IP address for the container \"/am-api-test\": the server is ignored"
time="2019-10-22T08:40:43Z" level=debug msg="Backend backend-traefik-traefik: no load-balancer defined, fallback to 'wrr' method"
time="2019-10-22T08:40:43Z" level=debug msg="Backend backend-am-app-am-app-test: no load-balancer defined, fallback to 'wrr' method"
time="2019-10-22T08:40:43Z" level=debug msg="Configuration received from provider docker: {\"backends\":{\"backend-am-app-am-app-test\":{\"servers\":{\"server-am-app-test-0ca42ef701fd7c6a621230fb67244a3f\":{\"url\":\"http://172.31.0.3:80\",\"weight\":1}},\"loadBalancer\":{\"method\":\"wrr\"}},\"backend-traefik-traefik\":{\"servers\":{\"server-traefik-9c2d11757b4c813f8ddfece1d549b205\":{\"url\":\"http://172.31.0.2:8080\",\"weight\":1}},\"loadBalancer\":{\"method\":\"wrr\"}}},\"frontends\":{\"frontend-Host-test-am-alexyfondeville-com-2\":{\"entryPoints\":[\"http\",\"https\"],\"backend\":\"backend-am-app-am-app-test\",\"routes\":{\"route-frontend-Host-test-am-alexyfondeville-com-2\":{\"rule\":\"Host:test.am.alexyfondeville.com\"}},\"passHostHeader\":true,\"priority\":0,\"basicAuth\":null},\"frontend-Host-test-am-api-alexyfondeville-com-1\":{\"entryPoints\":[\"http\",\"https\"],\"backend\":\"backend-am-api-test\",\"routes\":{\"route-frontend-Host-test-am-api-alexyfondeville-com-1\":{\"rule\":\"Host:test.am-api.alexyfondeville.com\"}},\"passHostHeader\":true,\"priority\":0,\"basicAuth\":null},\"frontend-Host-traefik-alexyfondeville-com-0\":{\"entryPoints\":[\"http\",\"https\"],\"backend\":\"backend-traefik-traefik\",\"routes\":{\"route-frontend-Host-traefik-alexyfondeville-com-0\":{\"rule\":\"Host:traefik.alexyfondeville.com\"}},\"passHostHeader\":true,\"priority\":0,\"basicAuth\":null}}}"
time="2019-10-22T08:40:43Z" level=info msg="Server configuration reloaded on :80"
time="2019-10-22T08:40:43Z" level=info msg="Server configuration reloaded on :443"
time="2019-10-22T08:40:43Z" level=info msg="Server configuration reloaded on :8080"
time="2019-10-22T08:40:44Z" level=debug msg="Wiring frontend frontend-Host-test-am-alexyfondeville-com-2 to entryPoint http"
time="2019-10-22T08:40:44Z" level=debug msg="Creating backend backend-am-app-am-app-test"
time="2019-10-22T08:40:44Z" level=debug msg="Adding TLSClientHeaders middleware for frontend frontend-Host-test-am-alexyfondeville-com-2"
time="2019-10-22T08:40:44Z" level=debug msg="Creating load-balancer wrr"
time="2019-10-22T08:40:44Z" level=debug msg="Creating server server-am-app-test-0ca42ef701fd7c6a621230fb67244a3f at http://172.31.0.3:80 with weight 1"
time="2019-10-22T08:40:44Z" level=debug msg="Creating route route-frontend-Host-test-am-alexyfondeville-com-2 Host:test.am.alexyfondeville.com"
time="2019-10-22T08:40:44Z" level=debug msg="Wiring frontend frontend-Host-test-am-alexyfondeville-com-2 to entryPoint https"
time="2019-10-22T08:40:44Z" level=debug msg="Creating backend backend-am-app-am-app-test"
time="2019-10-22T08:40:44Z" level=debug msg="Adding TLSClientHeaders middleware for frontend frontend-Host-test-am-alexyfondeville-com-2"
time="2019-10-22T08:40:44Z" level=debug msg="Creating load-balancer wrr"
time="2019-10-22T08:40:44Z" level=debug msg="Creating server server-am-app-test-0ca42ef701fd7c6a621230fb67244a3f at http://172.31.0.3:80 with weight 1"
time="2019-10-22T08:40:44Z" level=debug msg="Creating route route-frontend-Host-test-am-alexyfondeville-com-2 Host:test.am.alexyfondeville.com"
time="2019-10-22T08:40:44Z" level=error msg="undefined backend 'backend-am-api-test' for frontend frontend-Host-test-am-api-alexyfondeville-com-1. Skipping frontend frontend-Host-test-am-api-alexyfondeville-com-1..."
2

Hi @Afondevi, thanks for your detailed report.

Could you try the following and let us know if it works please?

  • Update Traefik to latest 1.7.x patch (aka. 1.7.18-alpine). Reason: a lot of bug (some related to docker) and security fixes.
  • Remove the label 'traefik.backend from the docker compose service am-api, as Traefik create a backend automatically for you, and forcing a name might be the source of the trouble here.

By the way, can you also check the state of the am-api container? The error message Unable to find the IP address for the container (...) is often triggerrer when Traefik received an event of docker container created, but when trying to retrieve its IP address, the container is stopped/crashed/unhealthy. It can also be a bug in Traefik when retrieving this IP, but we need insights:
So can you provide the result of docker ps and docker inspect <container for am-api> once you will have applied the changes please?