Traefik 3 - Only Traefik Dashboard works

JeanS · May 5, 2024, 3:39pm

I am getting Traefik3 to work only partly.
I have admit that I am complete newbie on Traefik, and just tried to follow these instructions with some changes:

Main difference that I used Portainer - Stacks instead of docker-compose.yaml.
This Traefik is running on Proxmox - Ubuntu 22.04 Server VM, where is just Docker with Portainer and Traefik containers.

https://traefik-dashboard.mydomain/ is working fine, connection is secure and certificate looks ok to me.

But all other services or such (Proxmox, Homeassistant) running on different VMs / Servers does not work.

https://proxmox.mydomain/ => I am getting "404 page not found" error, but connection is secure.
https://homeassistant.mydomain/ => I am getting "400: Bad Request". Also got "Internal Server Error" after small modification on config.yml. Homeassistant is HAOS VM install. Also secure connection.

Debug logs did not show any errors or warnings. I think port handling does not work correctly. Proxmox is using 8006 and Homeassistant 8123.

I have defined those DNS names to PiHole and using Traefik IP instead of actual VM IP.

Is some additional configuration needed when service is not running on Same VM (same IP) as Traefik?

I don't need access service outside of my local network at the moment, so I have not defined any DNS entry on Cloudflare.

bluepuma77 · May 5, 2024, 6:27pm

Share your full Traefik static and dynamic config, and docker-compose.yml or equivalent if used.

JeanS · May 5, 2024, 7:07pm

I have been trying to get work only js-zen-prox Proxmox server, homeassistant and truenas.
mydomain.net is fake here, but I have reserved own xxxx.net domain from CloudFlare that I am using also below configurations.


#### Portainer Stacks: ####
version: '3'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - proxy
    ports:
      - 80:80
      - 443:443

    environment:
      - CF_DNS_API_TOKEN=xxxxx
      - CF_API_EMAIL=xxxx@gmail.com
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /home/js/docker_volumes/traefik/data/traefik.yml:/traefik.yml:ro
      - /home/js/docker_volumes/traefik/data/acme.json:/acme.json
      - /home/js/docker_volumes/traefik/data/config.yml:/config.yml:ro
      - /home/js/docker_volumes/trafik/data/logs:/var/log/traefik
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.mydomain.net`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.users=xxxx"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.mydomain.net`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik-secure.tls.domains[0].main=mydomain.net"
      - "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.mydomain.net"
      - "traefik.http.routers.traefik-secure.service=api@internal"
      #- "traefik.http.middlewares.my-service-ipallowlist.ipallowlist.sourcerange=192.168.50.0/24"
      #- "traefik.http.routers.my-service.middlewares=my-service-ipallowlist@docker"

networks:
  proxy:
    external: true

#### traefik.yml: ####
api:
  dashboard: true
  debug: true

entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entrypoint:
          to: https
          scheme: https

  https:
    address: ":443"

serversTransport:
  insecureSkipVerify: false

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    filename: /config.yml
    watch: true

log:
  level: "DEBUG"
  filePath: "/var/log/traefik/traefik.log"
accessLog:
  filePath: "/var/log/traefik/access.log"

certificatesResolvers:
  cloudflare:
    acme:
      email: xxxx@gmail.com
      storage: acme.json
      caServer: https://acme-v02.api.letsencrypt.org/directory # production
      dnsChallenge:
        provider: cloudflare
        #disablePropagationCheck: true # uncomment this if you have issues pulling certificates through cloudflare, By>
        #delayBeforeCheck: 60s # uncomment along with disablePropagationCheck if needed to ensure the TXT record is re>
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

#### config.yml ####
http:
 #region routers
  routers:
    js_zen_proxmox:
      entryPoints:
        - "https"
      rule: "Host(`js-zen-prox.mydomain.net`)"
      middlewares:
        - default-headers
        - https-redirectscheme
      tls: {}
      service: js_zen_proxmox

    homeassistant:
      # For Homeassistant config, check: https://www.home-assistant.io/integrations/http/#reverse-proxies
      # This relies on Homeassistant using http. No certs are needed in the Homeassistant config.
      entryPoints:
        - "https"
      rule: "Host(`homeassistant.mydomain.net`)"
      middlewares:
        - default-headers
        - https-redirectscheme
      tls: {}
      service: homeassistant

    truenas:
      entryPoints:
        - "https"
      rule: "Host(`truenas.mydomain.net`)"
      middlewares:
        - default-headers
        - https-redirectscheme
      tls: {}
      service: truenas
#endregion

#region services
  services:
    js_zen_proxmox:
      loadBalancer:
        servers:
          - url: "https://192.168.50.130:8006"
        passHostHeader: true

    homeassistant:
      loadBalancer:
        servers:
          - url: "https://192.168.50.60:8123"
        passHostHeader: true

    truenas:
      loadBalancer:
        servers:
          - url: "https://192.168.50.85"
        passHostHeader: true

#endregion

  middlewares:
    https-redirectscheme:
      redirectScheme:
        scheme: https
        permanent: true

    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https

    default-whitelist:
      ipAllowList:
        sourceRange:
        - "192.168.50.0/24"

[quote="bluepuma77, post:2, topic:22586, full:true"]
Share your full Traefik static and dynamic config, and `docker-compose.yml` or equivalent if used.
[/quote]

bluepuma77 · May 6, 2024, 4:55am

You mount traefik.yml to /. Use target /etc/traefik/traefik.yml (doc) or add

command: --configFile=/traefik.yml

in docker-compose.yml (doc).

Also

storage: acme.json

should use an absolute path (/acme.json) for the mounted file.

JeanS · May 6, 2024, 12:58pm

I could not add command: or storage: to stacks -> it did not start then.
However, I got working all services that won't require port on address field:

services:
proxmoxzen:
loadBalancer:
servers:
- url: "http://192.168.50.130:8006"
passHostHeader: true

truenas:
  loadBalancer:
    servers:
      - url: "http://192.168.50.85"
    passHostHeader: true

homeassistant:
  loadBalancer:
    servers:
      - url: "http://192.168.50.60:8123"
    passHostHeader: true

For example: Truenas is working fine via Traefik, but ProxmoxZen and Homeassistant does not.
I am getting different errors on those;
Homeassistant: 400: Bad Request
ProxmoxZen: ERR_TOO_MANY_REDIRECTS

badabing · May 27, 2024, 9:31pm

I can concur that with Traefik 3, Home Assistant does not work, I get http 404
Every other service is working in my case except Home Assistant.
If I revert back to an older version (the latest before 3) it works fine.

bluepuma77 · May 28, 2024, 5:55am

Check and compare to simple Traefik example.

If your configuration is not working, you should share Traefik static and dynamic config, config of the target service, and docker-compose.yml if used.

badabing · May 30, 2024, 5:57pm

Found the issue:

error while adding rule Host(`homeassistant.REDACTED.com`, `haos.REDACTED.com`): error while adding rule Host: unexpected number of parameters; got 2, expected one of [1]

And it's not documented in:

However this fixes it:

core:
  defaultRuleSyntax: v2

Or: Host rule does not support multiple hostnames anymore

JeanS · June 10, 2024, 3:05pm

I have got working all services that I wanted, except homeassistant.
I tried to add this core: rule, but not sure where to configure it.

Can you share full homeassistant config?

And also if I try to enable homeassistant to config.yml, also all other services stops to work (homeassistant is defined exactly same way as all other services)

Edit: I did minor change and now it is not crashing other services. On Traefik Dashboard Homeassistant looks fine. But when trying to access HA, I will get 'Internal Server Error'

bluepuma77 · June 10, 2024, 4:07pm

Share your full Traefik static and dynamic config, and docker-compose.yml or equivalent if used.

badabing · June 10, 2024, 4:11pm

Was it working with v2?
If you never used v2, you don't need that core rule.
In my case it was working for v2 but broke in v3 because of host rule format for multiple hostnames. (See the link above)

JeanS · June 10, 2024, 4:22pm

I just started to use Traefik on version 3. No past experience.

badabing · June 10, 2024, 4:38pm

then get rid of that v2 compatibility part.
As requested by @bluepuma77 share your config.

Do you have traefik IP added to trusted_proxies?

http:
  use_x_forwarded_for: true
  trusted_proxies:
    - 192.168.1.130/32 # Host IP where Traefik container is running
    - 172.23.0.2/32 # Container IP of Traefik

subhobose0003 · June 27, 2024, 5:20pm

How did you get it working? Can you share the config files. I am running into the same problem. I have followed the same guide by technotim and the same thing is happening. The sevices running on the same docker server are all getting resolved but not my proxmox server or any other VM.

JeanS · June 28, 2024, 1:54pm

Some of services does not work at all like homeassistant, but e.g. proxmox on three different servers works fine. Using pihole as local DNS. Here is my config file:
http:
#region routers
routers:
...
proxmoxzen:
entryPoints:
- "https"
rule: Host(proxzen.js-homelab.net)
middlewares:
- default-headers
- https-redirectscheme
tls: {}
service: proxmoxzen

...
#region services
services:

proxmoxzen:
  loadBalancer:
    servers:
      - url: "https://192.168.50.130:8006"
    passHostHeader: true

...
#region middlewares

middlewares:

https-redirectscheme:
  redirectScheme:
    scheme: https
    permanent: true

default-headers:
  headers:
    frameDeny: true
    browserXssFilter: true
    contentTypeNosniff: true
    forceSTSHeader: true
    stsIncludeSubdomains: true
    stsPreload: true
    stsSeconds: 15552000
    customFrameOptionsValue: SAMEORIGIN
    customRequestHeaders:
      X-Forwarded-Proto: https

default-whitelist:
  ipAllowList:
    sourceRange:
    - "192.168.50.0/24"

secured:
  chain:
    middlewares:
    - default-whitelist

JeanS · June 28, 2024, 1:59pm

To add:
So in pfSense router primary DNS Server is my local piHole IP address.
In piHole is then DNS entry for Proxmox and pointing to IP where Traefik is running.

bluepuma77 · June 28, 2024, 6:42pm

Share your full Traefik static and dynamic config, and docker-compose.yml if used.

subhobose0003 · June 29, 2024, 2:26pm

I have created a post where I have added all of my config files -Traefik does not redirect to my Proxmox URL.
For my DNS settings, I have a local DNS running where I have A records and PTR records created for my internal domain and then a CNAME record for the domain that will get the cert for Traefik to forward. This is working fine as when I do nslookup I am able to get proper results.

subhobose0003 · June 30, 2024, 12:35pm

Another thing I have observed recently is, I have a NAS server running on a different VM and for that server I am able to access the web portal but in that I am not getting valid certificate from lets encrypt. But receiving valid cert for Traefik dashboard.

Topic		Replies	Views
Traefik + homeassistant (without TLS certificate) Traefik v2 docker	0	521	March 22, 2021
Traefik Routing or Redirect Traefik v2 docker	2	872	February 26, 2024
Can't get wildcard domains to work with Cloudflare and Docker Traefik v1 docker	0	428	October 6, 2019
Docker traefik + portainer Traefik v3 (latest) docker	0	181	May 30, 2024
[solved] Traefik resolving cloudflare domain/sub-domains to an old machine IP - can't access dashboard either Traefik v2 docker	2	891	January 5, 2023

Traefik 3 - Only Traefik Dashboard works

Related Topics