Cannot get Traefik to find macvlan container network

Traefik Traefik v2
1

I have been working on this for 2 days now, and I'm completely stumped.

I have an external macvlan network (macvlan-5) assigned to all containers and traefik through labels, and defined networks. All containers successfully communicate over this network but Traefik floods the logs with.

level=warning msg="Could not find network named 'macvlan-5' for container 'portainer_portainer.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network." serviceName=api-dummy providerName=docker container=portainer-portainer-btbjwfcoeyb5q8kbvnb3ddlqp

Stack:

version: '3.3'
services:
  traefik:
    image: traefik:latest
    container_name: traefik
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8080
        published: 8080
        protocol: tcp
    command:
      - --log.level=WARN
#      - --ping=true
#      - --entrypoints.web.http.redirections.entryPoint.to=https
#      - --entrypoints.web.http.redirections.entryPoint.scheme=https
#      - --entrypoints.web.http.redirections.entrypoint.permanent=true
#      - --ping.entryPoint=ping
      - --providers.docker=true
      - --providers.docker.network=macvlan-5
      - --providers.docker.swarmMode=true
      - --providers.docker.exposedbydefault=false
      - --entrypoints.http.address=:80
      - --entrypoints.https.address=:443
 #    - --docker.domain=moderncaveman.us
      - --api.dashboard=true
      - --api.insecure=true
      - --certificatesresolvers.letsencrypt.acme.dnschallenge=true
      - --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.letsencrypt.acme.email=certs@email.com
      - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
      - --certificatesresolvers.letsstaging.acme.tlschallenge=true
      - --certificatesresolvers.letsstaging.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
      - --certificatesresolvers.letsstaging.acme.email=certs@email.com     
      - --accesslog
      - --log
      - --accesslog.filepath=/logs/traefik-access.log
      - --certificatesresolvers.letsencrypt.acme.storage=/ssl/acme.json
      - --certificatesresolvers.letsstaging.acme.storage=/ssl/acme-staging.json

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - traefik-ssl:/ssl/
      - traefik-logs:/logs/
    networks:
#        traefik_public:
       - macvlan-5

    deploy:
      mode: global
      labels:
        - "traefik.docker.network=macvlan-5"
        - "traefik.http.routers.api.rule=Host(`traefik.moderncaveman.us`)"
        - "traefik.http.routers.api.entrypoints=https"    
        - "traefik.http.routers.api.tls=true"
        - "traefik.http.routers.api.tls.certresolver=letsencrypt"
        - traefik.http.routers.api.service=api@internal
        - "traefik.http.routers.traefik-rtr.tls=true" 
        - "traefik.http.services.traefik.loadbalancer.server.port=8080"

        # uncomment this to enable forward authentication on the traefik api/dashboard
  #      - traefik.http.routers.api.middlewares=forward-auth     
      placement:
        constraints: [node.role == manager]    
  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    networks:
      - "macvlan-5"
    labels:
      - traefik.enable=true
      - "traefik.http.services.whoami.loadbalancer.server.port=80"
      - "traefik.http.routers.whoami.rule=Host(`whoami.moderncaveman.us`)"
      - "traefik.http.routers.whoami.entrypoints=https"
      - "traefik.http.routers.whoami.tls.certresolver=letsstaging"

networks:
  macvlan-5:
    external: 
      name: macvlan-5
    driver: macvlan

volumes:
    traefik-ssl:
        driver_opts:
            type: nfs 
            o: addr=192.168.40.91,nfsvers=4
            device: :/volume2/docker-pool/traefik/ssl/ 
    traefik-logs:
        driver_opts:
            type: nfs 
            o: addr=192.168.40.91,nfsvers=4
            device: :/volume2/docker-pool/traefik/logs/

and docker network ls

root@Mule:/volume2/docker-pool/portainer# docker network ls
NETWORK ID     NAME                            DRIVER    SCOPE
28a788a3e680   Granfa                          bridge    local
6usoindfq290   PiSwarmpit_net                  overlay   swarm
mdwcz24y1da5   autopi_internal                 overlay   swarm
37e95bf537bc   bridge                          bridge    local
85d17e152818   docker_gwbridge                 bridge    local
bd1460ef14d6   host                            host      local
tttgkozf7pmu   ingress                         overlay   swarm
oh228za1gjxz   librespeed_default              overlay   swarm
qfz4ofhtwee1   librespeed_librespeed_default   overlay   swarm
mrvtqodx38y6   macvlan-5                       macvlan   swarm
71514f1a1ccc   macvlan-5-config                null      local
e22d5c155428   mcvlan-config                   null      local
0796fe2ae89b   none                            null      local
2ad4425cd4c5   overseerr_correct               null      local
ea42e05db547   overseerr_default               bridge    local
87b3d79f9c95   photonix_default                bridge    local
42f5552e7304   photoprisim_default             bridge    local
yb2fk0hfh9e0   portainer_agent_network         overlay   swarm
stpmkzkjn3xr   portainer_internal              overlay   swarm
be7dfa3f2036   sabnzbd_default                 bridge    local
s0gshxjh9spw   traefik_internal                          swarm
k43botw9wqlf   traefik_public                  overlay   swarm
25105ac697f7   unpackerr_default               bridge    local
root@Mule:/volume2/docker-pool/portainer#

What am I doing wrong here?

2

Do you have infra available where you can replace VLAN with overlay just to see if there is a difference?

3

Probably not the issue: you define two different challenges on the same resolver

4

I don’t think the name should be spaced in, docs.

5

That was me grasping at straws. The name shouldn't need to be defined at all, but I gave it a shot.

Not sure what you mean here, are you referring to the macvlan?

Right you are, there are a lot of bugs in that config. I was trying to focus on the macvlan part of it but ended up cleaning it up some when it killed my spirit :sob:

Here is the config I am working with currently, but macvlan still has the same errors.

version: '3.3'
services:
  traefik:
    image: traefik:latest
    container_name: traefik
    ports:
      - target: 80
        published: 80
        protocol: tcp
#        mode: host
      - target: 443
        published: 443
        protocol: tcp
#        mode: host
#      - target: 8080
#        published: 8080
#        protocol: tcp
    command:
    
    
      - --log.level=WARN
#      - --ping=true
#      - --ping.entryPoint=ping
      #entrypoints
#      - --entrypoints.web.http.redirections.entryPoint.to=websecure
#      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --entrypoints.https.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22
      - --entrypoints.websecure.http.tls=true
      - --entryPoints.web.forwardedHeaders.insecure
      - --entryPoints.web.address=:80
      - --entryPoints.websecure.address=:443  
      - --providers.docker=true
#      - --providers.docker.network=macvlan-5
      - --providers.docker.swarmMode=true
      - --providers.docker.exposedbydefault=false

 #    - --docker.domain=moderncaveman.us
      - --api.dashboard=true
#      - --api.insecure=true
      #Resolvers
#      - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
#      - --certificatesresolvers.letsstaging.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.letsencrypt.acme.dnschallenge=true
      - --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.letsencrypt.acme.email=cory@email.com
      - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
      - --certificatesresolvers.letsstaging.acme.tlschallenge=true
      - --certificatesresolvers.letsstaging.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
      - --certificatesresolvers.letsstaging.acme.email=cory@email.com
      - --certificatesresolvers.letsencrypt.acme.storage=/ssl/acme.json
      - --certificatesresolvers.letsstaging.acme.storage=/ssl/acme-staging.json

      #logging
      - --accesslog
      - --accesslog.format=json
      - --log
      - --accesslog.filepath=/logs/traefik-access.log
      - --log.filepath=/logs/traefik.log
      - --log.format=json


    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - traefik-ssl:/ssl/
      - traefik-logs:/logs/
    networks:
       - public
       - macvlan-5

    deploy:
      mode: global
      labels:
        - traefik.enable=true
        - traefik.docker.network=macvlan-5
        - traefik.http.routers.api.rule=Host(`traefik.moderncaveman.us`)
        - traefik.http.routers.api.entrypoints=websecure,web
        - traefik.http.routers.api.tls=true
        - traefik.http.routers.api.tls.certresolver=letsencrypt
        - traefik.http.routers.api.service=api@internal
        - traefik.http.services.traefik.loadbalancer.server.port=8080
        - traefik.http.services.traefik.loadbalancer.server.scheme=https
#        - traefik.http.routers.traefik-tls.tls.domains[0].main=moderncaveman.us
#        - 'traefik.http.routers.traefik-tls.tls.domains[0].sans=*.moderncaveman.us'
        #Authela
#        - 'traefik.http.middlewares.authelia.forwardAuth.address=http://authelia:9091/api/verify?rd=https%3A%2F%2Fauth.example.com%2F'
#        - 'traefik.http.middlewares.authelia.forwardAuth.trustForwardHeader=true'
#        - 'traefik.http.middlewares.authelia.forwardAuth.authResponseHeaders=Remote-User,Remote-Groups,Remote-Name,Remote-Email'


 # uncomment this to enable forward authentication on the traefik api/dashboard
  #      - traefik.http.routers.api.middlewares=forward-auth     
      placement:
        constraints: [node.role == manager]    
  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    command:
      - --name=whoami
    networks:
      - macvlan-5
      - public
    deploy:
      labels:
         traefik.enable: "true"
         traefik.docker.network: macvlan-5    
         traefik.http.services.whoami.loadbalancer.server.port: 80
         traefik.http.routers.whoami.rule: "Host(`whoami.moderncaveman.us`)"
         traefik.http.routers.whoami.entrypoints: websecure,web
#         traefik.http.routers.whoami.tls: "true"
#         traefik.http.routers.whoami.tls.certresolver: letsencrypt    
 


 

      
      
networks:
  public:
    driver: overlay
  macvlan-5:
    external: true
    driver: macvlan  #"<- tried with and without driver. Volume works either way, but traefik can't find it based off the label"

volumes:
    traefik-ssl:
        driver_opts:
            type: nfs 
            o: addr=192.168.40.91,nfsvers=4
            device: :/volume2/docker-pool/traefik/ssl/ 
    traefik-logs:
        driver_opts:
            type: nfs 
            o: addr=192.168.40.91,nfsvers=4
            device: :/volume2/docker-pool/traefik/logs/

whoami in the above stack is still resolving to it's internal network IP, and not macvlan-5

image
image1017×507 21.4 KB

image
image1803×1155 68.7 KB

{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:07-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:07-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:40:16-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:16-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:16-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:40:22-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:22-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:22-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:40:37-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:37-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:37-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:40:46-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:46-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:46-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:40:52-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:52-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:40:52-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:41:07-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:41:07-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:41:07-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:41:16-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:41:16-06:00"}
{"container":"traefik-traefik-kst3ov08i79t5ffzl3cpd4p5z","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.kst3ov08i79t5ffzl3cpd4p5z'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:41:16-06:00"}
{"container":"traefik-whoami-u63zm5bvkwgh8b4b7f2ibd778","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_whoami.1'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"whoami","time":"2022-11-25T01:41:22-06:00"}
{"container":"traefik-traefik-hwwekml7tc9kfaidruzqej26o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.hwwekml7tc9kfaidruzqej26o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T01:41:22-06:00"}
6

Your container names all get the prefix traefik_, probably from compose or stack. So you need to make sure your network name is correct.

With external you are stating it’s not created by compose/stack, but using an existing one, driver shouldn’t matter in that case. Is that existing network using the driver you want?

With infra I am asking if you have similar infrastructure available to test your setup with overlay network instead of VLAN.

7

Right, which unless I am missing something means I have it defined correctly as that is the true docker network. As an external network, the project prefix should not apply.

mrvtqodx38y6   macvlan-5                       macvlan   swarm

Yes, it is using the macvlan driver and the network itself works fine.

The compose "public" / docker "traefik_public" in the config is an overlay network and works as expected; it is the one I am using while debugging macvlan.

8

Just some ideas:

I think the docker.network belongs to static config, needs to go into command.

Check the real final name of the network with docker network ls. Try adding that in the line.

You only need docker.network when your target service is linked to (multiple) networks, of which Traefik is not part of. Otherwise the Service Discovery might choose a target IP that Traefik can not reach.

Why do you have your whoami container on multiple networks, anyway? I would say it only needs to be in an internal network that also connects to Traefik.

9

Mostly just for ease of use of testing as I've been at this for a few days now. These are good tips and I am giving them a go now, but these are just convenient examples because they're in the same stack. I have tried the same network with other stacks and get the same error only when referring to a macvlan network.

According to the docs it can be overidden with a label per container, the command is just a catch-all unless I am misunderstanding

network

Optional, Default=""

Defines a default docker network to use for connections to all containers.

This option can be overridden on a per-container basis with the traefik.docker.network label.

I have shown the output of docker network ls in the OP and a focused version of the macvlan-5 output in my last post, it is correct as far as I can tell but let me know if if you are referring to something else.

10

In this example I removed all labels referring to the macvlan network, the extra network on whoami, and uncommented my command referring to it.

version: '3.3'
services:
  traefik:
    image: traefik:latest
    container_name: traefik
    ports:
      - target: 80
        published: 80
        protocol: tcp
#        mode: host
      - target: 443
        published: 443
        protocol: tcp
#        mode: host
#      - target: 8080
#        published: 8080
#        protocol: tcp
    command:
    
    
      - --log.level=WARN
#      - --ping=true
#      - --ping.entryPoint=ping
      #entrypoints
#      - --entrypoints.web.http.redirections.entryPoint.to=websecure
#      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --entrypoints.https.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22
      - --entrypoints.websecure.http.tls=true
      - --entryPoints.web.forwardedHeaders.insecure
      - --entryPoints.web.address=:80
      - --entryPoints.websecure.address=:443  
      - --providers.docker=true
      - --providers.docker.network=macvlan-5
      - --providers.docker.swarmMode=true
      - --providers.docker.exposedbydefault=false

 #    - --docker.domain=moderncaveman.us
      - --api.dashboard=true
#      - --api.insecure=true
      #Resolvers
#      - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
#      - --certificatesresolvers.letsstaging.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.letsencrypt.acme.dnschallenge=true
      - --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.letsencrypt.acme.email=con@gmail.com
      - --certificatesresolvers.letsencrypt.acme.tlschallenge=true
      - --certificatesresolvers.letsstaging.acme.tlschallenge=true
      - --certificatesresolvers.letsstaging.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
      - --certificatesresolvers.letsstaging.acme.email=cn@gmail.com 
      - --certificatesresolvers.letsencrypt.acme.storage=/ssl/acme.json
      - --certificatesresolvers.letsstaging.acme.storage=/ssl/acme-staging.json

      #logging
      - --accesslog
      - --accesslog.format=json
      - --log
      - --accesslog.filepath=/logs/traefik-access.log
      - --log.filepath=/logs/traefik.log
      - --log.format=json


    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - traefik-ssl:/ssl/
      - traefik-logs:/logs/
    networks:
       - public
       - macvlan-5


    deploy:
      mode: global
      labels:
        - traefik.enable=true
#        - traefik.docker.network=macvlan-5
        - traefik.http.routers.api.rule=Host(`traefik.moderncaveman.us`)
        - traefik.http.routers.api.entrypoints=websecure,web
        - traefik.http.routers.api.tls=true
        - traefik.http.routers.api.tls.certresolver=letsencrypt
        - traefik.http.routers.api.service=api@internal
        - traefik.http.services.traefik.loadbalancer.server.port=8080
        - traefik.http.services.traefik.loadbalancer.server.scheme=https
#        - traefik.http.routers.traefik-tls.tls.domains[0].main=moderncaveman.us
#        - 'traefik.http.routers.traefik-tls.tls.domains[0].sans=*.moderncaveman.us'
        #Authela
#        - 'traefik.http.middlewares.authelia.forwardAuth.address=http://authelia:9091/api/verify?rd=https%3A%2F%2Fauth.example.com%2F'
#        - 'traefik.http.middlewares.authelia.forwardAuth.trustForwardHeader=true'
#        - 'traefik.http.middlewares.authelia.forwardAuth.authResponseHeaders=Remote-User,Remote-Groups,Remote-Name,Remote-Email'


 # uncomment this to enable forward authentication on the traefik api/dashboard
  #      - traefik.http.routers.api.middlewares=forward-auth     
      placement:
        constraints: [node.role == manager]    
  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    command:
      - --name=whoami
    networks:
      - macvlan-5
#      - public
    deploy:
      labels:
         traefik.enable: "true"
#         traefik.docker.network: macvlan-5    
         traefik.http.services.whoami.loadbalancer.server.port: 80
         traefik.http.routers.whoami.rule: "Host(`whoami.moderncaveman.us`)"
         traefik.http.routers.whoami.entrypoints: websecure,web
         traefik.http.routers.whoami.tls: "true"
#         traefik.http.routers.whoami.tls.certresolver: letsencrypt    
 


 

      
      
networks:
  public:
    driver: overlay
  macvlan-5:
    external: true
    driver: macvlan

volumes:
    traefik-ssl:
        driver_opts:
            type: nfs 
            o: addr=192.168.40.91,nfsvers=4
            device: :/volume2/docker-pool/traefik/ssl/ 
    traefik-logs:
        driver_opts:
            type: nfs 
            o: addr=192.168.40.91,nfsvers=4
            device: :/volume2/docker-pool/traefik/logs/  

{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:22-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:28-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:37-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:37-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:52-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:52-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:58-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:22:58-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:07-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:07-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:22-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:22-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:28-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:28-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:37-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:37-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:52-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:52-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:58-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:23:58-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:24:07-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:24:07-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:24:13-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:24:13-06:00"}
{"container":"traefik-traefik-8sy9a059vflnpmow95tnhv97o","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.8sy9a059vflnpmow95tnhv97o'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:24:28-06:00"}
{"container":"traefik-traefik-vl5vkujtosb843iqun51bmgmr","level":"warning","msg":"Could not find network named 'macvlan-5' for container 'traefik_traefik.vl5vkujtosb843iqun51bmgmr'! Maybe you're missing the project's prefix in the label? Defaulting to first available network.","providerName":"docker","serviceName":"traefik","time":"2022-11-25T10:24:28-06:00"}

This caused whoami to drop off the dashboard completely.

11

Hmm, maybe it’s really a bug. But then you would need to provide a really simple example for the maintainers to be able to reproduce