HTTP 404 errors (both http and https)

Traefik Traefik v3 (latest)
,
1

Hi all,
I'm new to traefik (used nginx previously), so please bear with me.
I'm running traefik 3.5.2 in a LXC debian-based container in proxmox. I run authelia in another LXC container.
I have a domain (A and CNAME records), proxied through cloudflare.
I can reach any of those services through their LAN IPs; issue is, I cannot reach either when using the domain.
I have checked traefik's log, but I need some guidance since it looks like traefik fails to find a router for the services?

debian version 
root@traefik:~# cat /etc/*-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
traefik version 
root@traefik:~# traefik version
Version:      3.5.2
Codename:     chabichou
Go version:   go1.24.7
Built:        2025-09-09T10:17:00Z
OS/Arch:      linux/amd64
traefik systemd service 
root@traefik:~# cat /etc/systemd/system/traefik.service 
[Unit]
Description=Traefik
Documentation=https://doc.traefik.io/traefik/
After=network-online.target

[Service]
Environment=CLOUDFLARE_EMAIL=ps@<domain>.com
Environment=CLOUDFLARE_API_KEY=<redacted>
Type=simple
User=root
Group=root
ExecStart=/usr/bin/traefik --configfile=/etc/traefik/traefik.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target
traefik static configuration 
# Traefik global configuration
global:
  checkNewVersion: true
  sendAnonymousUsage: false

# Enable traefik ui dashboard
api:
  dashboard: true
  insecure: true  # Consider securing this in production

# Log level INFO|DEBUG|ERROR
log:
  level: DEBUG
  filePath: /var/log/traefik/traefik.log
  format: json
  maxSize: 100
  maxBackups: 5
  compress: true

# Configuring access logs
accessLog:
  filePath: /var/log/traefik/access.log
  format: json
  filters:
    statusCodes:
      - '200-299' # log successful http requests
      - '400-599' # log failed http requests
    retryAttempts: true
    minDuration: '10ms'
  # collect logs as in-memory buffer before writing into log file
  bufferingSize: 10
#  fields:
#    headers:
#      defaultMode: drop # drop all headers per default
#      names:
#          User-Agent: keep # log user agent strings

# The setting below is to allow insecure backend connections
serversTransport:
  insecureSkipVerify: true

# Traefik entrypoints (network ports) configuration
# Note: These ports are NOT exposed to the internet, only to Cloudflare Tunnel
entryPoints:
  web:
    address: :80
    forwardedHeaders:
      trustedIPs: &trustedIps
        # Allow these IPs to set the X-Forwarded-* headers - Cloudflare IPs: https://www.cloudflare.com/ips/
        - 173.245.48.0/20
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 141.101.64.0/18
        - 108.162.192.0/18
        - 190.93.240.0/20
        - 188.114.96.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 162.158.0.0/15
        - 104.16.0.0/13
        - 104.24.0.0/14
        - 172.64.0.0/13
        - 131.0.72.0/22
        - 2400:cb00::/32
        - 2606:4700::/32
        - 2803:f800::/32
        - 2405:b500::/32
        - 2405:8100::/32
        - 2a06:98c0::/29
        - 2c0f:f248::/32
        # Local IPs
        - 192.168.1.0/24
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          permanent: true
#    observability:
#      accessLogs: true
#      metrics: false
#      tracing: false

  websecure:
    address: :443
    forwardedHeaders:
      # Reuse list of Cloudflare Trusted IP's above for HTTPS requests
      trustedIPs: *trustedIps
    http3: true
    http:
#      tls: {}
      tls:
        # TLS Options File inside rules folder
        options: tls-options@file
        certResolver: cloudflare
        domains:
          - main: '<domain>.com'
            sans:
              - '*.<domain>.com'

providers:
  providersThrottleDuration: 2s

  # File provider for defining services and middleware
  file:
    directory: '/etc/traefik/dynamic'
    watch: true

# Optional: Certificates resolver for internal services or development
certificatesResolvers:
  cloudflare:
    acme:
      email: ps@<domain>.com
      storage: /etc/traefik/acme.json
#      caServer: 'https://acme-staging-v02.api.letsencrypt.org/directory'  # Use staging for testing
      caServer: 'https://acme-v02.api.letsencrypt.org/directory'  # Use for production
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - '1.1.1.1:53'
          - '1.0.0.1:53'
        # To delay DNS check and reduce LE hitrate
        delayBeforeCheck: 90

# Plugins
#experimental:
#  plugins:
#    cloudflarewarp:
#      moduleName: 'github.com/PseudoResonance/cloudflarewarp'
#      version: 'v1.4.2'
authelia dynamic configuration 
http:
  routers:
    authelia:
      rule: "Host(`auth.<domain>.com`)"
      entryPoints:
#         - web
         - websecure
      service: authelia
      middlewares:
        - ch-no-auth

  services:
    authelia:
      loadBalancer:
        servers:
          - url: "https://192.168.1.205:9091"
        passHostHeader: true
TLS options dynamic configuration 
tls:
  options:
    tls-options:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384
        - TLS_CHACHA20_POLY1305_SHA256
        - TLS_FALLBACK_SCSV # Client is doing version fallback. See RFC 7507
      curvePreferences:
        - CurveP521
        - CurveP384
      sniStrict: true
secure headers middleware 
http:
  middlewares:
    mw-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlMaxAge: 100
        hostsProxyHeaders:
          - 'X-Forwarded-Host'
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        # forceSTSHeader: true # This is a good thing but it can be tricky. Enable after everything works.
        customFrameOptionsValue: 'SAMEORIGIN' # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
        contentTypeNosniff: true
        browserXssFilter: true
        referrerPolicy: 'same-origin'
        permissionsPolicy: 'camera=(), microphone=(), geolocation=(), payment=(), usb=(), vr=()'
        customResponseHeaders:
          X-Robots-Tag: 'none,noindex,nofollow,noarchive,nosnippet,notranslate,noimageindex' # disable search engines from indexing home server
          server: '' # hide server info from visitors
        customRequestHeaders:
          X-Forwarded-Proto: 'https'
rate limit middleware 
ttp:
  middlewares:
    mw-rate-limit:
      rateLimit:
        average: 100
        burst: 50
chain no-auth 
http:
  middlewares:
    ch-no-auth:
      chain:
        middlewares:
          - mw-rate-limit
          - mw-secure-headers

Last line of traefik.log:

{
  "level":"debug",
  "providerName":"cloudflare.acme",
  "acmeCA":"https://acme-v02.api.letsencrypt.org/directory",
  "providerName":"cloudflare.acme",
  "ACME CA":"https://acme-v02.api.letsencrypt.org/directory",
  "domains":["<domain>.com","*.<domain>.com"],
  "time":"2025-09-17T18:31:50+02:00",
  "caller":"github.com/traefik/traefik/v3/pkg/provider/acme/provider.go:1001",
  "message":"No ACME certificate generation required for domains"
}

Last line of access.log:

{
  "ClientAddr":"192.168.1.201:44892",
  "ClientHost":"<WAN IP>",
  "ClientPort":"44892",
  "ClientUsername":"-",
  "DownstreamContentSize":19,
  "DownstreamStatus":404,
  "Duration":233252,
  "GzipRatio":0,
  "OriginContentSize":0,
  "OriginDuration":0,
  "OriginStatus":0,
  "Overhead":233252,
  "RequestAddr":"auth.<domain>.com",
  "RequestContentSize":0,
  "RequestCount":23,
  "RequestHost":"auth.<domain>.com",
  "RequestMethod":"GET",
  "RequestPath":"/",
  "RequestPort":"-",
  "RequestProtocol":"HTTP/1.1",
  "RequestScheme":"http",
  "RetryAttempts":0,
  "StartLocal":"2025-09-17T18:06:47.077754009+02:00",
  "StartUTC":"2025-09-17T16:06:47.077754009Z",
  "entryPointName":"websecure",
  "level":"info",
  "msg":"",
  "time":"2025-09-17T18:06:47+02:00"
}

What confuses me here, are these lines in the access.log's log entry:

  "OriginStatus":0,
  "RequestScheme":"http",
  "entryPointName":"websecure",

I'm no expert, but I understand that "OriginStatus":0 means that traefik itself is denying the request? And also, there's no "RouterName" in the log entry, so it means that traefik cannot find a suitable router?
When I reach out to traefik's console (via the LAN IP) I can see no errors in routers or services:

image
image1858×960 103 KB

What am I doing wrong?
Thanks in advance!

2

Very well analyzed.

It seems you get a http request on websecure entrypoint on port 443. It is not matched by Traefik with a router rule, probably because the router is tls enabled, so no RouterName and OriginStatus: 0.

Are you trying to access the domain locally (hosts file) or via CloudFlare? I would check CF proxy/forward settings.

3

Hi,

Thanks!

You mean my config is wrong? I suspect that, but cannot find where. Should I move the tls options out of the websecure entrypoint? If so, where to?
Also, I'm confused by the "RequestScheme":"http" in the access log. Shouldn't it be https? But then again, no router matches...

Via cloudflare, through my domain. The CNAME reords are proxied by cloudflare. Can you point me to some example DNS configuration? I'm skeptical, because this DNS setup worked with nginx.

Thanks for your support!

4

You stated you proxy through CF. I would assume the setting is wrong there, not in your local Traefik installation.

Didn’t you enter a target URL there?