# Traefik cannot obtain service containers of other nodes on the manager node

**URL:** <https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203>\
**Category:** Traefik v3 (latest)\
**Tags:** docker-swarm\
**Created:** [June 17, 2024, 6:43am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203 "2024-06-17T06:43:57Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sqh539](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/sqh539/32/8899_2.png) [@sqh539](https://community.traefik.io/u/sqh539)\
**Post date:** [June 17, 2024, 6:43am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/1 "2024-06-17T06:43:57Z")

</div>

traefik-stack.yml

```auto
version: '3.3'

services:
  traefik:
    image: traefik:v3.0
    container_name: traefik
    restart: unless-stopped
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.swarm=true"
      - "--providers.docker.network=dms-overlay-network"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
    ports:
      - "9080:80"
      - "9880:8080"
      - "9443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.yml:/traefik.yml
      - ./certs/certfile.crt:/certs/certfile.crt
      - ./certs/keyfile.key:/certs/keyfile.key
    networks:
      - dms-overlay-network
    deploy:
      placement:
        constraints:
          - node.role == manager

networks:
  dms-overlay-network:
    external: true

```

traefik.yml

```auto
entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"

# Docker configuration backend
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    network: "dms-overlay-network"
    watch: true
    defaultRule: "Host(`{{ trimPrefix `/` .Name }}.demo.com`)"

# API and Dashboard settings
api:
  insecure: true
  dashboard: true

# Log settings
log:
  level: DEBUG 
accessLog:
  format: json 
  filters:
    statusCodes: "400-599"  
  fields:
    defaultMode: keep
    names:
      "ClientUsername": drop
      "ClientHost": keep

tls:
  stores:
    default:
      defaultCertificate:
        certFile: /certs/certfile.crt
        keyFile: /certs/keyfile.key

http:
  routers:
    dashboard:
      rule: "Host(`dashboard.demo.com`)"
      service: "api@internal"
      entryPoints:
        - "websecure"
      tls:
        certFile: /certs/certfile.crt
        keyFile: /certs/keyfile.key
      middlewares:
        - auth

  middlewares:
    auth:
      basicAuth:
        users:
          - "admin:haitai@123"

```

model-server.yml

```auto
version: '3.3'

services:
  model-server:
    image: model_run_sklearn:1.0
    user: "8888:981"
    environment:
      - ORG_CODE=100001
      - IMAGE_TYPE=model_run_sklearn:1.0
    volumes:
      - /home/dms_data/run/base_file/config.yaml:/dms/conf/config.yaml
      - /home/dms_data/run/api/run_file/e5205e88889c49c4815c27f8def82d7f/hander.py:/work_dir/actuator/hander.py
      - /home/dms_data/run/task/model/d5f1fe3f843f4deebc46a5a1b63d0223.model:/work_dir/actuator/run.model
    networks:
      - dms-overlay-network
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.model-servers.rule=PathPrefix(`/model-servers`)"
      - "traefik.http.services.model-servers.loadbalancer.server.port=5000"
      - "traefik.http.routers.model-servers.entrypoints=websecure"
      - "traefik.http.routers.model-servers.tls=true"
    deploy:
      replicas: 3

networks:
  dms-overlay-network:
    external: true

```

按照上面的配置启动model-server.yml后，dashboard中只能看到主节点启动的容器ip。其他两个节点启动的容器ip无法获取。这三个节点机器的dms-overlay-network网络id是相同的，三个容器可以相互访问。

---

<div class="post-metadata">

**Author:** ![bluepuma77](https://avatars.discourse-cdn.com/v4/letter/b/a9adbd/32.png) [@bluepuma77](https://community.traefik.io/u/bluepuma77)\
**Post date:** [June 17, 2024, 7:01am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/2 "2024-06-17T07:01:56Z")

</div>

English language, please.

---

<div class="post-metadata">

**Author:** ![sqh539](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/sqh539/32/8899_2.png) [@sqh539](https://community.traefik.io/u/sqh539)\
**Post date:** [June 17, 2024, 7:14am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/3 "2024-06-17T07:14:22Z")

</div>

> [@sqh539](#):
>
> 按照上面的配置启动model-server.yml后，dashboard中只能看到主节点启动的容器ip。其他两个节点启动的容器ip无法获取。这三个节点机器的dms-overlay-network网络id是相同的，三个容器可以相互访问。

After starting model-server.yml according to the above configuration, only the container IP started by the master node can be seen in the dashboard servers. The container IP started by the other two nodes cannot be obtained. The Docker network `dms-overlay-network` ID of these three node machines are the same, and the three containers can access each other.

---

<div class="post-metadata">

**Author:** ![bluepuma77](https://avatars.discourse-cdn.com/v4/letter/b/a9adbd/32.png) [@bluepuma77](https://community.traefik.io/u/bluepuma77)\
**Post date:** [June 17, 2024, 8:04am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/4 "2024-06-17T08:04:01Z")

</div>

Note that you are mixing multiple things:

1. Traefik static config (`entrypoints`, `providers`, `api`, `log`, `certresolver`) can be in `traefik.yml` or `command:`, but not both. Decide for one ([doc](https://doc.traefik.io/traefik/getting-started/configuration-overview/#the-static-configuration)).

2. Traefik dynamic config (`tls`, `http` and `tcp`, `routers`, `middlewares` and `services`) is better in a separate file, which you load with `providers.file`. Or `routers`/`middlewares`/`services` in labels via `providers.docker`.

---

<div class="post-metadata">

**Author:** ![sqh539](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/sqh539/32/8899_2.png) [@sqh539](https://community.traefik.io/u/sqh539)\
**Post date:** [June 17, 2024, 8:47am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/5 "2024-06-17T08:47:06Z")

</div>

traefik-stack.yml

```auto
version: '3.3'

services:
  traefik:
    image: traefik:v3.0
    container_name: traefik
    restart: unless-stopped
    command:
      - --api.insecure=true
      - --api.debug=true
      - --providers.docker=true
      - --providers.docker.exposedByDefault=false
      - --providers.docker.network=dms-overlay-network
      - --log.level=DEBUG
      - --accesslog=true
      - --accesslog.filters.statuscodes=400-599
      - --accesslog.format=json
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - "9080:80"
      - "9880:8080"
      - "9443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./certs/certfile.crt:/certs/certfile.crt
      - ./certs/keyfile.key:/certs/keyfile.key
    networks:
      - dms-overlay-network
    deploy:
      placement:
        constraints:
          - node.role == manager

networks:
  dms-overlay-network:
    external: true

```

model-server.yml

```auto
version: '3.3'

services:
  model-server:
    image: model_run_sklearn:1.0
    user: "8888:981"
    environment:
      - ORG_CODE=100001
      - IMAGE_TYPE=model_run_sklearn:1.0
    volumes:
      - /home/dms_data/run/base_file/config.yaml:/dms/conf/config.yaml
      - /home/dms_data/run/api/run_file/e5205e88889c49c4815c27f8def82d7f/hander.py:/work_dir/actuator/hander.py
      - /home/dms_data/run/task/model/d5f1fe3f843f4deebc46a5a1b63d0223.model:/work_dir/actuator/run.model
    networks:
      - dms-overlay-network
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.model-servers.rule=PathPrefix(`/model-servers`)"
      - "traefik.http.services.model-servers.loadbalancer.server.port=5000"
      - "traefik.http.routers.model-servers.entrypoints=web"
    deploy:
      replicas: 3

networks:
  dms-overlay-network:
    external: true

```

Servers

Status URL

--- [http://10.0.1.224:5000](http://10.0.1.224:5000)

```auto
[root@liuwenhua tmp]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
6jbzx7zov6du model_model-server replicated 3/3 docker.haitai.com/dms/model_run_sklearn:1.0   
wcs28kd9a0ns stack_traefik replicated 1/1 docker.haitai.com/dms/traefik:v3.0 *:9080->80/tcp, *:9443->443/tcp, *:9880->8080/tcp
[root@liuwenhua tmp]# docker service ps model_model-server
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
g7vd590i7o2n model_model-server.1 docker.haitai.com/dms/model_run_sklearn:1.0 zhoujiancheng.haitaichina.net Running Running 10 minutes ago             
y71ce93qrd1r model_model-server.2 docker.haitai.com/dms/model_run_sklearn:1.0 zuojunxiang-nb.haitaichina.net Running Running 10 minutes ago             
wk9nrndtt78d model_model-server.3 docker.haitai.com/dms/model_run_sklearn:1.0 liuwenhua.haitaichina.net Running Running 10 minutes ago 

```

When I switched to booting without using static configuration, the situation remained the same. only the container IP started by the master node can be seen in the dashboard servers.

---

<div class="post-metadata">

**Author:** ![bluepuma77](https://avatars.discourse-cdn.com/v4/letter/b/a9adbd/32.png) [@bluepuma77](https://community.traefik.io/u/bluepuma77)\
**Post date:** [June 17, 2024, 10:36am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/6 "2024-06-17T10:36:12Z")

</div>

Config seems ok.

You deploy with `docker stack deploy`?

What does Traefik debug log tell you?

When you want to use your custom TLS certs, you need to use a dynamic config file, loaded with `providers.file` in static config with `tls` section, then enable TLS on `entrypoint` or `router` (no `certresolver`).

---

<div class="post-metadata">

**Author:** ![sqh539](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/sqh539/32/8899_2.png) [@sqh539](https://community.traefik.io/u/sqh539)\
**Post date:** [June 17, 2024, 12:34pm UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/7 "2024-06-17T12:34:52Z")

</div>

```auto
version: '3.3'

services:
  traefik:
    image: docker.haitai.com/dms/traefik:v3.0
    container_name: traefik
    restart: unless-stopped
    command:
      - --providers.swarm=true
      - --providers.swarm.endpoint=unix:///var/run/docker.sock
      - --api.insecure=true
      - --api.debug=true
      - --log.level=DEBUG
      - --accesslog=true
      - --accesslog.filters.statuscodes=400-599
      - --accesslog.format=json
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - "9080:80"
      - "9880:8080"
      - "9443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - dms-overlay-network
    deploy:
      placement:
        constraints:
          - node.role == manager

networks:
  dms-overlay-network:
    external: true

```

```auto
version: '3.3'

services:
  model-server:
    image: docker.haitai.com/dms/model_run_sklearn:1.0
    user: "8888:981"
    environment:
      - ORG_CODE=100001
      - IMAGE_TYPE=model_run_sklearn:1.0
    volumes:
      - /home/dms_data/run/base_file/config.yaml:/dms/conf/config.yaml
      - /home/dms_data/run/api/run_file/e5205e88889c49c4815c27f8def82d7f/hander.py:/work_dir/actuator/hander.py
      - /home/dms_data/run/task/model/d5f1fe3f843f4deebc46a5a1b63d0223.model:/work_dir/actuator/run.model
    networks:
      - dms-overlay-network
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.model-servers.rule=PathPrefix(`/model-servers`)"
      - "traefik.http.services.model-servers.loadbalancer.server.port=5000"
      - "traefik.http.routers.model-servers.entrypoints=web"
    deploy:
      replicas: 3

networks:
  dms-overlay-network:
    external: true

```

[stack\_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net](mailto:stack_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net) | 2024-06-17T12:33:45Z ERR [github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81](http://github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81) \> error="service "stack-traefik" error: port is missing" container=stack-traefik-dk04ah1atmy3wy791thuxxvzn providerName=swarm  
[stack\_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net](mailto:stack_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net) | 2024-06-17T12:33:45Z ERR [github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81](http://github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81) \> error="service "model-model-server" error: port is missing" container=model-model-server-p3wvigshoyjl5vmq7hgj4broz providerName=swarm  
[stack\_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net](mailto:stack_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net) | 2024-06-17T12:33:45Z ERR [github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81](http://github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81) \> error="service "model-model-server" error: port is missing" container=model-model-server-p6ll1be0w6ekjg1kmnci4t4ju providerName=swarm  
[stack\_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net](mailto:stack_traefik.1.dk04ah1atmy3@liuwenhua.haitaichina.net) | 2024-06-17T12:33:45Z ERR [github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81](http://github.com/traefik/traefik/v3/pkg/provider/docker/config.go:81) \> error="service "model-model-server" error: port is missing" container=model-model-server-q00eg65553v6fj7eoxd72xwh8 providerName=swarm

why does this configuration report that the port cannot be found

---

<div class="post-metadata">

**Author:** ![bluepuma77](https://avatars.discourse-cdn.com/v4/letter/b/a9adbd/32.png) [@bluepuma77](https://community.traefik.io/u/bluepuma77)\
**Post date:** [June 17, 2024, 4:46pm UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/8 "2024-06-17T16:46:13Z")

</div>

When using Swarm, labels need to go inside `deploy:` section.

---

<div class="post-metadata">

**Author:** ![sqh539](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/sqh539/32/8899_2.png) [@sqh539](https://community.traefik.io/u/sqh539)\
**Post date:** [June 18, 2024, 12:52am UTC](https://community.traefik.io/t/traefik-cannot-obtain-service-containers-of-other-nodes-on-the-manager-node/23203/9 "2024-06-18T00:52:44Z")

</div>

The adjustment has been successful. Thank you very much.