Service(s) randomly get 307 Temp Redirects to root domain, then start working again

Traefik Traefik v2 (latest)
,
1

I've recently moved my services to a subdomain structure (from using subpaths before), and I'm currently experiencing some strange redirects. I've just updated to Traefik v2.8.4 and the problem persists, if maybe worse now: even the dashboard is exhibiting this behavior.

I've purchased my own domain that supports ACME SSL. On my LAN, I have an AdGuardHome container that has a rewrite specified for a subdomain that encompasses my private docker containers. Occasionally, when I try to access a service, Traefik will send my browser a 307 Temporary Redirect code to my top domain. Other services may continue working fine, and after about ~10 minutes of this, the original problem service(s) will resolve them selves and begin working normally. The problem is almost entirely random, affects a random number of services, and I have no idea how to trigger it or where to start diagnosing this from. I'd appreciate input from the community, thanks in advance!

Static config: traefik.yml

global:
  checkNewVersion: true
  sendAnonymousUsage: true

entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: "websecure"

  websecure:
    address:	:443
    http:
      tls: true

serversTrasnport:
  insecureSkipVerify: true

certificatesResolvers:
  porkbun:
    acme:
      email: [email]
      storage: /etc/traefik/acme.json
      
      dnsChallenge:
        provider: porkbun

log:
  level: INFO

accessLog:
  filePath: /var/log/traefik/access.log

api:
  dashboard: true

providers:
  docker:
    defaultRule: Host(`{{ normalize .Name }}.[my private subdomain]`)

  file:
    directory: /etc/traefik/providers
    watch: true

Docker-applied labels (Audiobookshelf container):

  • traefik.http.routers.audiobookshelf.entrypoints websecure
  • traefik.http.routers.audiobookshelf.rule Host(abs.[my private subdomain])
  • traefik.http.routers.audiobookshelf.tls true
  • traefik.http.routers.audiobookshelf.tls.certresolver porkbun

FreshRSS container:

  • traefik.docker.network traefik_default
  • traefik.http.routers.freshrss.entrypoints websecure
  • traefik.http.routers.freshrss.rule Host(rss.[my private subdomain])
  • traefik.http.routers.freshrss.tls true
  • traefik.http.routers.freshrss.tls.certresolver porkbun
1 Like
2

I am facing a similar kind of problem. After some random time, browser shows ERR_CONNECTION_CLOSED error. I try to ping my remote instances from client using the domain name and every packet drops. But the same ping works fine on remote instance. I kept my head scratching to find out what went wrong and after wasting 2 days, I cannot find the root cause.
I am using Godaddy as domain name provider, configured traefik such that it wildcard certificate for my subdomains, and everything seems to work fine initially.

There are some log entries that look suspicious to me.

time="2022-09-05T20:31:22+05:30" level=debug msg="Serving default certificate for request: \"\""
time="2022-09-05T20:31:22+05:30" level=debug msg="http: TLS handshake error from 212.102.40.218:24426: read tcp 172.31.0.2:443->212.102.40.21
8:24426: read: connection reset by peer"

Any help would be greatly appreciated.

4

@abhishek0010 It looks like we're experiencing different issues. It looks like you're having connection problems between your client and server, but for me it just seems like Traefik is exhibiting strange behavior.

5

@glenb The problem may be different, but the behaviour is the same. I am also having the same experience with the latest version. The configured routers work as configured initially, but after about 10 minutes, the connection resets and I cannot access any of my services. But it again goes live after some time then again resets.
Since you are having the same behaviour, I thought it would be more appropriate to increase conversation on this question for swift response from community.

UPDATE:
Try adding headers.X-Forwarded-Proto=https middleware.

These are my labels for traefik:latest image:

- "traefik.enable=true"
- "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.http-catchall.entrypoints=web"
- "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
- "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
- "traefik.http.middlewares.traefik-auth.basicauth.users=USER:PWD_HASH"
- "traefik.http.routers.traefik-secure.rule=Host(`SUBDOMAIN.DOMAIN.TLD`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.entrypoints=websecure"
- "traefik.http.routers.traefik-secure.tls.certresolver=godaddy"
- "traefik.http.routers.traefik-secure.tls.domains[0].main=DOMAIN.TLD"
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.DOMAIN.TLD"
- "traefik.http.routers.traefik-secure.service=api@internal"
- "traefik.docker.network=reverse_proxy_nw"
1 Like
6

Since you are having the same behaviour, I thought it would be more appropriate to increase conversation on this question for swift response from community.

You're right, I'm sorry - I appreciate you adding to the conversation.

I think my redirect is defined in my static config in the OP, although you reminded me that I made another static configuration for middlewares specifically, and one being adding secure headers. I'll paste it below, but I think we're accomplishing the same thing? If I still have issues I'll try to take a closer look and see how our changes differentiate.

[http.middlewares]
        [http.middlewares.chain-no-auth]
                [http.middlewares.chain-no-auth.chain]
                        middlewares = [ "middlewares-secure-headers", "middlewares-sslheader"]

    [http.middlewares.middlewares-secure-headers]
        [http.middlewares.middlewares-secure-headers.headers]
            hostsProxyHeaders = ["X-Forwarded-Host"]
            sslRedirect = true
            stsSeconds = 63072000
            stsIncludeSubdomains = true
            stsPreload = true
            forceSTSHeader = true
            contentTypeNosniff = true
            browserXssFilter = true
            referrerPolicy = "same-origin"
            featurePolicy = "camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
            customFrameOptionsValue = "allow-from [old domain I don't use anymore]" #

        [http.middlewares.middlewares-secure-headers.headers.sslproxyheaders]
            X-Forwarded-Proto = "https"

        [http.middlewares.middlewares-secure-headers.headers.customResponseHeaders]
            X-Robots-Tag = "none,noarchive,nosnippet,notranslate,noimageindex,"
            server = ""

        [http.middlewares.middlewares-sslheader.headers]
        [http.middlewares.middlewares-sslheader.headers.customrequestheaders]
            X-Forwarded-Proto = "https"
1 Like
7

Just for an update, I haven't restarted my traefik container for few days now and the problem is not occurring as frequent as before. Maybe the DNS provider was having record propagation issue earlier.
I am using GoDaddy as my provider with dns challenge for generating wild card ssl certificates.