Unable to Route Quadlets Through Traefik

I recently made the decision to change from Docker to Podman in my homelab, and in the process, I’ve converted my docker-compose.yaml files to .container files (i.e., quadlets).

As a result, I have the following folder structure in ~/.config/containers:

.
├── containers.conf.d
│   └── engine.conf
├── storage
│   ├── portainer-ce
│   │   ├── bin
│   │   ├── certs
│   │   │   ├── cert.pem
│   │   │   └── key.pem
│   │   ├── chisel
│   │   │   └── private-key.pem
│   │   ├── compose
│   │   ├── docker_config
│   │   │   └── config.json
│   │   ├── portainer.db
│   │   ├── portainer.key
│   │   ├── portainer.pub
│   │   └── tls
│   ├── traefik
│   │   ├── acme.json
│   │   ├── config.yaml
│   │   └── traefik.yaml
│   └── uptime-kuma
│       ├── docker-tls
│       ├── error.log
│       ├── kuma.db
│       ├── kuma.db-shm
│       ├── kuma.db-wal
│       ├── screenshots
│       └── upload
└── systemd
    ├── chartdb
    │   └── chartdb.container
    ├── glance
    │   └── glance.container
    ├── portainer-ce
    │   └── portainer-ce.container
    ├── traefik
    │   ├── traefik.container
    │   ├── traefik.env
    │   └── traefik.network
    └── uptime-kuma
        └── uptime-kuma.container

My traefik.container looks like the following:

[Unit]
Description=Traefik
After=local-fs.target
Wants=network-online.target
After=network-online.target
Requires=podman.socket
After=podman.socket

[Container]
Image=docker.io/traefik:v3.5.1
AutoUpdate=registry

ContainerName=traefik
HostName=traefik

NoNewPrivileges=true

Network=traefik.network

PublishPort=80:80/tcp
PublishPort=443:443/tcp

Environment=CF_API_EMAIL=${CLOUDFLARE_EMAIL}
Environment=CF_DNS_API_TOKEN=${CLOUDFLARE_API_KEY}
Environment=TRAEFIK_DASHBOARD_CREDENTIALS=${TRAEFIK_DASHBOARD_CREDENTIALS}

Label=traefik.enable=true
Label=traefik.http.routers.traefik.entrypoints=http
Label=traefik.http.routers.traefik.rule=Host(`traefik.mydomain.tld`)
Label=traefik.http.middlewares.traefik-auth.basicauth.users=${TRAEFIK_DASHBOARD_CREDENTIALS}
Label=traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https
Label=traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https
Label=traefik.http.routers.traefik.middlewares=traefik-https-redirect
Label=traefik.http.routers.traefik-secure.entrypoints=https
Label=traefik.http.routers.traefik-secure.rule=Host(`traefik.mydomain.tld`)
Label=traefik.http.routers.traefik-secure.middlewares=traefik-auth
Label=traefik.http.routers.traefik-secure.tls=true
Label=traefik.http.routers.traefik-secure.tls.certresolver=cloudflare
Label=traefik.http.routers.traefik-secure.tls.domains[0].main=mydomain.tld
Label=traefik.http.routers.traefik-secure.tls.domains[0].sans=*.mydomain.tld
Label=traefik.http.routers.traefik-secure.service=api@internal
Label=traefik.http.services.traefik.loadbalancer.server.port=80

Volume=%h/.config/containers/storage/traefik/traefik.yaml:/etc/traefik/traefik.yaml:ro,Z
Volume=%h/.config/containers/storage/traefik/config.yaml:/etc/traefik/config.yaml:ro,Z
Volume=%h/.config/containers/storage/traefik/acme.json:/etc/traefik/acme.json:rw,Z
Volume=/etc/localtime:/etc/localtime:ro
Volume=%t/podman/podman.sock:/var/run/docker.sock:ro

[Service]
Restart=on-failure
TimeoutStartSec=300

EnvironmentFile=%h/.config/containers/systemd/traefik/traefik.env

[Install]
WantedBy=multi-user.target default.target

Similarly, my traefik.yaml:

global:
  checkNewVersion: false
  sendAnonymousUsage: false

log:
  level: DEBUG

api:
  dashboard: true
  debug: true

entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https

  https:
    address: ":443"

serversTransport:
  insecureSkipVerify: true

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false

  file:
    filename: /etc/traefik/config.yaml

certificatesResolvers:
  cloudflare:
    acme:
      email: REDACTED

      storage: /etc/traefik/acme.json

      # Production (Default)
      caServer: https://acme-v02.api.letsencrypt.org/directory

      # Staging
      # caServer: https://acme-staging-v02.api.letsencrypt.org/directory

      keyType: EC256

      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

It should be noted that the traefik.container and traefik.yaml are pretty much identical to what I’ve used for Traefik for the past 3 years, with no changes to the labels or the files, etc.

I created uptime-kuma.container with the following content:

[Unit]
Description=Uptime Kuma
After=local-fs.target
Wants=network-online.target
After=network-online.target
Requires=podman.socket
After=podman.socket
Wants=traefik.service
After=traefik.service

[Container]
Image=docker.io/louislam/uptime-kuma:latest
AutoUpdate=registry

ContainerName=uptime-kuma
HostName=uptime-kuma

Network=traefik

PublishPort=3131:3001/tcp

Label=traefik.enable=true
Label=traefik.http.routers.uptime-kuma.rule=Host(`monitor.mydomain.tld`)
Label=traefik.http.routers.uptime-kuma.entrypoints=https
Label=traefik.http.routers.uptime-kuma.tls=true
Label=traefik.http.services.uptime-kuma.loadbalancer.server.port=3131

Volume=%E/containers/storage/uptime-kuma/:/app/data:rw,Z
Volume=%t/podman/podman.sock:/var/run/docker.sock:ro

[Service]
Restart=on-failure
TimeoutStartSec=300

[Install]
WantedBy=multi-user.target default.

I have all these .containers running in a VM (podman.mydomain.tld - 10.10.20.2), which has CNAME aliases pointing to it for traefik, monitor and portainer-ce (the latter will come in to topic shortly). However, when I navigate to https://monitor.mydomain.tld, I get the following:

Sep 03 23:33:11 podman traefik[220044]: 2025-09-03T23:33:11+03:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:175 > Service selected by WRR: http://10.89.0.3:3131
Sep 03 23:33:11 podman traefik[220044]: 2025-09-03T23:33:11+03:00 DBG github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121 > 502 Bad Gateway error="dial tcp 10.89.0.3:3131: connect: connection refused"

Similary, with portainer-ce, which consists of the same flags, albeit with portainer instead of monitor, and to Port 9443 instead of 3131, throws this error:

Sep 03 23:35:51 podman traefik[220044]: 2025-09-03T23:35:51+03:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:175 > Service selected by WRR: http://10.89.0.4:9443
Sep 03 23:35:51 podman traefik[220044]: 2025-09-03T23:35:51+03:00 DBG log/log.go:245 > httputil: ReverseProxy read error during body copy: read tcp 10.89.0.2:39532->10.89.0.4:9443: read: connection reset by peer
Sep 03 23:35:51 podman traefik[220044]: 2025-09-03T23:35:51+03:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/recovery/recovery.go:45 > Request has been aborted [10.89.0.2:43954 - /]: net/http: abort Handler middlewareName=traefik-internal-recovery middlewareType=Recovery

I used to have these connections in my config.yaml, as Portainer and Uptime-Kuma used to be in their own LXC containers, which is no longer the case.

Checking portainer, I can see that all 3 containers are indeed in the traefik network, as seen below:

I have Promox and my iLO in config.yaml, which route perfectly fine, so I know Traefik is working as it should (to a certain extent).

I would really appreciate having some assistance in solving this issue, as I spent a whole week trying to figure it out, but to no avail.

TIA!

Podman, Quadlets - that's all kind of new to Traefik. They support Docker, but I think most development and new features are going to k8s.

I would enable and check Traefik debug log (or check in Traefik dashboard), check how your target services are recognized, what target IPs are recognized. Then you can try from Traefik container to connect to them with installed wget.

But in general I don't know if Traefik providers.docker is compatible with Podman and Quadlets.

In many contexts, podman.sock can be used in place of docker.sock as podman should be able to have a translation layer for it (AFAIK). If Portainer and Uptime Kuma can, then Traefik should very likely have it.

When I start Portainer (for example), I can see the following in journalctl --user -xeu traefik.service:

Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/provider/docker/pdocker.go:111 > Provider event received {Status:start ID:4e7d1057c3a6ff6af3d579e9b64b277e50e5d45081f18020df104930a313719e From:docker.io/portainer/portainer-ce:lts Type:container Action:start Actor:{ID:4e7d1057c3a6ff6af3d579e9b64b277e50e5d45081f18020df104930a313719e Attributes:map[PODMAN_SYSTEMD_UNIT:portainer-ce.service com.docker.desktop.extension.api.version:>= 0.2.2 com.docker.desktop.extension.icon:https://portainer-io-assets.sfo2.cdn.digitaloceanspaces.com/logos/portainer.png com.docker.extension.additional-urls:[{"title":"Website","url":"https://www.portainer.io?utm_campaign=DockerCon&utm_source=DockerDesktop"},{"title":"Documentation","url":"https://docs.portainer.io"},{"title":"Support","url":"https://join.slack.com/t/portainer/shared_invite/zt-txh3ljab-52QHTyjCqbe5RibC2lcjKA"}] com.docker.extension.detailed-description:<p data-renderer-start-pos="226">Portainer&rsquo;s Docker Desktop extension gives you access to all of Portainer&rsquo;s rich management functionality within your docker desktop experience.</p><h2 data-renderer-start-pos="374">With Portainer you can:</h2><ul><li>See all your running containers</li><li>Easily view all of your container logs</li><li>Console into containers</li><li>Easily deploy your code into containers using a simple form</li><li>Turn your YAML into custom templates for easy reuse</li></ul><h2 data-renderer-start-pos="660">About Portainer&nbsp;</h2><p data-renderer-start-pos="680">Portainer is the worlds&rsquo; most popular universal container management platform with more than 650,000 active monthly users. Portainer can be used to manage Docker Standalone, Kubernetes and Docker Swarm environments through a single common interface. It includes a simple GitOps automation engine and a Kube API.&nbsp;</p><p data-renderer-start-pos="1006">Portainer Business Edition is our fully supported commercial grade product for business-wide use. It includes all the functionality that businesses need to manage containers at scale. Visit <a class="sc-jKJlTe dPfAtb" href="http://portainer.io/" title="http://Portainer.io" data-renderer-mark="true">Portainer.io</a> to learn more about Portainer Business and <a class="sc-jKJlTe dPfAtb" href="http://portainer.io/take-3?utm_campaign=DockerCon&amp;utm_source=Docker%20Desktop" title="http://portainer.io/take-3?utm_campaign=DockerCon&amp;utm_source=Docker%20Desktop" data-renderer-mark="true">get 3 free nodes.</a></p> com.docker.extension.publisher-url:https://www.portainer.io com.docker.extension.screenshots:[{"alt": "screenshot one", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-1.png"},{"alt": "screenshot two", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-2.png"},{"alt": "screenshot three", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-3.png"},{"alt": "screenshot four", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-4.png"},{"alt": "screenshot five", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-5.png"},{"alt": "screenshot six", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-6.png"},{"alt": "screenshot seven", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-7.png"},{"alt": "screenshot eight", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-8.png"},{"alt": "screenshot nine", "url": "https://portainer-io-assets.sfo2.digitaloceanspaces.com/screenshots/docker-extension-9.png"}] image:docker.io/portainer/portainer-ce:lts io.containers.autoupdate:registry io.portainer.server:true name:portainer-ce org.opencontainers.image.description:Docker container management made simple, with the world’s most popular GUI-based container management platform. org.opencontainers.image.title:Portainer org.opencontainers.image.vendor:Portainer.io podId: traefik.enable:true traefik.http.routers.portainer-ce.entrypoints:https traefik.http.routers.portainer-ce.rule:Host(`portainer.,mydomain.tld`) traefik.http.routers.portainer-ce.tls:true traefik.http.services.portainer-ce.loadbalancer.server.port:9443]} Scope:local Time:1756984565 TimeNano:1756984565263966577} providerName=docker
Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{"middlewares":{"sslheader":{"headers":{"customRequestHeaders":{"X-Forwarded-Proto":"https"}}},"traefik-auth":{"basicAuth":{"users":["admin:$$apr1$$3aEToBAB$$nXV.5t.TOTTHviILG8mww."]}},"traefik-https-redirect":{"redirectScheme":{"scheme":"https"}}},"routers":{"portainer-ce":{"entryPoints":["https"],"rule":"Host(`portainer.,mydomain.tld`)","service":"portainer-ce","tls":{}},"traefik":{"entryPoints":["http"],"middlewares":["traefik-https-redirect"],"rule":"Host(`traefik.,mydomain.tld`)","service":"traefik"},"traefik-secure":{"entryPoints":["https"],"middlewares":["traefik-auth"],"rule":"Host(`traefik.,mydomain.tld`)","service":"api@internal","tls":{"certResolver":"cloudflare","domains":[{"main":",mydomain.tld","sans":["*.,mydomain.tld"]}]}}},"services":{"portainer-ce":{"loadBalancer":{"passHostHeader":true,"responseForwarding":{"flushInterval":"100ms"},"servers":[{"url":"http://10.89.0.6:9443"}],"strategy":"wrr"}},"traefik":{"loadBalancer":{"passHostHeader":true,"responseForwarding":{"flushInterval":"100ms"},"servers":[{"url":"http://10.89.0.5:80"}],"strategy":"wrr"}}}},"tcp":{},"tls":{},"udp":{}} providerName=docker

[...]

Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/server/service/service.go:319 > Creating load-balancer entryPointName=https routerName=portainer-ce@docker serviceName=portainer-ce@docker
Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/server/service/service.go:362 > Creating server URL=http://10.89.0.6:9443 entryPointName=https routerName=portainer-ce@docker serverIndex=0 serviceName=portainer-ce@docker
Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/service.go:26 > Added outgoing tracing middleware entryPointName=https middlewareName=tracing middlewareType=TracingService routerName=portainer-ce@docker serviceName=portainer-ce@docker
Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/router.go:37 > Added outgoing tracing middleware entryPointName=https middlewareName=tracing middlewareType=TracingRouter routerName=portainer-ce@docker serviceName=portainer-ce@docker
Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/semconv.go:40 > Creating middleware entryPointName=https middlewareName=tracing middlewareType=SemConvServerMetrics routerName=portainer-ce@docker
Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/entrypoint.go:37 > Creating middleware entryPointName=https middlewareName=tracing middlewareType=TracingEntryPoint routerName=portainer-ce@docker

[...]

Sep 04 14:16:05 podman traefik[226999]: 2025-09-04T14:16:05+03:00 DBG github.com/traefik/traefik/v3/pkg/server/router/tcp/manager.go:237 > Adding route for portainer.mydomain.tld with TLS options default entryPointName=https

So from what I can gather, Traefik does recognize and pull the configuration by the labels I’ve set in portainer-ce.container file.

I can’t even access Traefik’s dashboard, as highlighted in Unable to Login to the Dashboard - #2 by bluepuma77

I am assuming you’re talking about using wget https://portainer.mydomain.tld inside the container (or using the podman network IP address instead of the FQDN?) I’d appreciate it if you could elaborate further.

I’ve managed to fix my issue highlighted in Unable to Login to the Dashboard - so now I have access to the dashboard. The following are images from the dashboard for my Uptime-Kuma and Portainer containers:

I am still not able to identify what is actually wrong with these 2 containers. One thing I did notice is that Portainer is http://<IP>:9443, instead of https://<IP>:9443, which could be the reason for it to cause the issues highlighted before. However, I cannot figure out why uptime-kuma returns “connection refused”

Sep 04 19:17:36 podman traefik[235394]: 2025-09-04T19:17:36+03:00 DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:175 > Service selected by WRR: http://10.89.0.19:3131
Sep 04 19:17:36 podman traefik[235394]: 2025-09-04T19:17:36+03:00 DBG github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121 > 502 Bad Gateway error="dial tcp 10.89.0.19:3131: connect: connection refused"

If Podman behaves like Docker, and you use a "container network", then you would need to connect to the internal port 3001, not the published port.

While that might be logical, that doesn’t explain why Traefik fails for Portainer then:

[Unit]
Description=Portainer CE
After=local-fs.target
Wants=network-online.target
After=network-online.target
Requires=podman.socket
After=podman.socket
Requires=traefik.service
After=traefik.service

[Container]
Image=docker.io/portainer/portainer-ce:lts
AutoUpdate=registry

ContainerName=portainer-ce
HostName=portainer-ce

PodmanArgs=--privileged

Network=traefik.network

PublishPort=8000:8000/tcp
PublishPort=9443:9443/tcp

Label=traefik.enable=true
Label=traefik.http.routers.portainer-ce.rule=Host(`portainer.arszilla.network`)
Label=traefik.http.routers.portainer-ce.entrypoints=https
Label=traefik.http.routers.portainer-ce.tls=true
Label=traefik.http.services.portainer-ce.loadbalancer.server.port=9443

Volume=%E/containers/storage/portainer-ce/:/data
Volume=%t/podman/podman.sock:/var/run/docker.sock

[Service]
Restart=on-failure
TimeoutStartSec=300

#ExecStartPre=mkdir -p %E/containers/%N/config

[Install]
WantedBy=multi-user.target default.target

The ports are not modified in Portainer (unlike Uptime Kuma or my other containers for example), yet Traefik tries accessing http://<IP>:9443 instead of https://<IP>:9443 according to the dashboard (as seen in the images above).

So after debugging further and researching, adding the following label to portainer-ce.container helped:

Label=traefik.http.services.portainer-ce.loadbalancer.server.scheme=https

Other than that, as you said @bluepuma77, pointing uptime-kuma to 3001 instead of my actual expose port was the way to go. Thanks for pointing that out!

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.