Service+Router only Registers After Long Delay

Traefik Traefik v2
1

Hi All,

I have a strange situation that I can't find any references to. Furthermore, it works correctly sometimes, just intermittently.

I'm launching three services (four if you count Traefik). For some reason, the "nakama_server" service/router doesn't populate in the Traefik dashboard consistently, often taking 5-10 minutes to appear, but the "whoami" service appears immediately, as well as the cockroachdb service (when enabled).

It seems that Traefik's discovery process is either being blocked, or is triggering on something strange with nakama server, but I think it's configured correctly.

Docker compose:

version: '3'
services:
  reverse-proxy:
    image: traefik:v2.4
    depends_on:
      - cockroachdb
      - nakama_server
      - whoami
    command:
      - "--api.insecure=true"
      - "--providers.docker"
      - "--accesslog" # Enables the web UI and tells Traefik to listen to docker
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    networks:
      - default
    ports:
      - "80:80"     # The HTTP port
      - "8080:8080" # The Web UI (enabled by --api)
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events

  nakama_server:
    image: heroiclabs/nakama:3.1.0
    entrypoint:
      - "/bin/sh"
      - "-ecx"
      - >
          /nakama/nakama migrate up --database.address root@cockroachdb:26257 &&
          exec /nakama/nakama --name nakama1 --database.address root@cockroachdb:26257 --logger.level DEBUG --session.token_expiry_sec 7200
    restart: "no"
    depends_on:
      - cockroachdb
    expose:
      - "7349"
      - "7350"
      - "7351"
    networks:
      - default
    volumes:
      - ./data:/nakama/data
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.nakama_server.service=nakama_server"
      - "traefik.http.routers.nakama_server.rule=Host(`nakama_server.docker.localhost`)"
      - "traefik.http.services.nakama_server.loadbalancer.server.port=7351"
      - "traefik.http.services.nakama_server.loadbalancer.server.scheme=http"

  cockroachdb:
    image: cockroachdb/cockroach:latest-v20.2
    command: start-single-node --insecure --store=attrs=ssd,path=/var/lib/cockroach/
    restart: "no"
    networks:
      - default
    volumes:
      - data:/var/lib/cockroach
    expose:
      - "8080"
      - "26257"
    labels:
      - "traefik.enable=false"
      - "traefik.http.services.cockroachdb.loadbalancer.server.port=8080"
      - "traefik.http.routers.cockroachdb.service=cockroachdb"
      - "traefik.http.routers.cockroachdb.rule=Host(`cockroachdb.docker.localhost`)"

  whoami:
    image: traefik/whoami # A container that exposes an API to show its IP address
    networks:
      - default
    labels:      
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami.docker.localhost`)"
       
volumes:
  data:

Anyone else seen this or run into a similar issue?

Best,
John

2

Hi @johnmarianhoffman

Most likely this is due to the container's healthcheck.

The 5-10 minutes line up with that. You can override/create your own healthcheck in your composefile.

1 Like
3

Thank you SO much @cakiwi. I seriously spent hours chasing that down and forgot that I had cleared out the healthcheck in an effort to simplify the compose file.

To go one step further, I'm curious why the failed health check (or at least delayed health check) was causing Traefik to not enumerate the service. Does Traefik require the docker health check? Additionally the "standard" health check in Nakama (curl -f {nakama_url}) would require the Traefik router to be functioning to work at all, leading to a kind of chicken/egg problem.

Is there a place in the documentation I could go to read a bit more about this?

Thanks again!

4

I don't think there is, I just know from experience / the log line in this code. Might be worth a mention in the docs though.

5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.