Dynamic config breaking all sites?

ZerkerEOD · January 27, 2024, 1:22am

I have a dynamic config which I will include below. I updated my traefik and added a UDP entrypoint, which I thought was what broke it. Looking at my logs I saw the following error:

time="2024-01-27T00:55:03Z" level=error msg="Error while building configuration (for the first time): /etc/traefik/config/nextcloud-dyn.yml: field not found, node: nextcloud-chain" providerName=file

So, before submitting an issue for breaking my entire config I decided to rename the nextcloud to add .bak behind and then everything started working again. Originally if I tried to go to any site behind traefik including the dashboard I would get a 404 error. Renaming that one dynamic config and poof everything is reachable again. I would like to know why if anyone can see it easily.

http:
  services:
    nextcloud-ser:
      loadBalancer:
        servers:
          - URL: "[redacted]"

  middlewares:
    nextcloud-secure-headers:
      headers:
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        referrerPolicy: "same-origin"

      nextcloud-chain:
        chain:
          middlewares:
            - nextcloud-secure-headers

  routers:
    nextcloud-route:
      entryPoints:
        - "websecure"
      rule: "HostSNI(`[redacted]`)"
      tls:
        certResolver: production
        domains:
          - main: "[redacted]"
        options: "modern"
        passthrough: true
      service: "nextcloud-ser"
      middlewares:
        - nextcloud-chain

As a note, I had a redirector in there also but already removed that as originally the error referenced the redirector, and since it is only listening on the websecure anyway, I didn't care about redirecting.

    test-redirectscheme:
      redirectScheme:
        scheme: https

bluepuma77 · January 27, 2024, 9:47am

It seems nextcloud-chain is spaced in under nextcloud-secure-headers, that's probably not intended. It should be below middlewares.

ZerkerEOD · January 27, 2024, 10:45am

I'm going to look into that when I get home. I moved that section up, originally behind my router section. My concern is that is the file was working until I updated and I wasn't in the file until after everything broke, so I'm baffled at how the spacing just started causing an issue. I see what you're talking about and will report back when I get a chance.

ZerkerEOD · January 27, 2024, 12:58pm

Looks like the space was it, but now I ran into another problem with the file. Not sure why it worked one day then broke the next. Here is my updated yml:

http:
  services:
    nextcloud-ser:
      loadBalancer:
        servers:
          - url: "[redacted]"

  routers:
    nextcloud-route:
      entryPoints:
        - "websecure"
      rule: "HostSNI(`[redacted]`)"
      tls:
        certResolver: production
        domains:
          - main: "[redacted]"
        options: "modern"
        passthrough: true
      service: "nextcloud-ser"
      middlewares:
        - nextcloud-chain

  middlewares:
    nextcloud-secure-headers:
      headers:
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        referrerPolicy: "same-origin"

    nextcloud-chain:
      chain:
        middlewares:
          - nextcloud-secure-headers

Here is my new error:

time="2024-01-27T12:54:27Z" level=error msg="Error occurred during watcher callback: /etc/traefik/config/nextcloud-dyn.yml: field not found, node: passthrough" providerName=file
time="2024-01-27T12:54:27Z" level=error msg="Error occurred during watcher callback: /etc/traefik/config/nextcloud-dyn.yml: field not found, node: passthrough" providerName=file

Looking at the documentation it looks like passthrough shouldn't work with HTTP but it used to and if I comment it out I still can't connect to my nextcloud instance and it was working yesterday.

bluepuma77 · January 27, 2024, 1:17pm

Is this the full dynamic config file? passthrough looks ok (doc), have you defined modern (doc)?

ZerkerEOD · January 30, 2024, 5:40pm

Yes, this is the full dynamic configuration with the URL and redacted stuff. I am not sure what you mean by modern. I clicked the link and don't see a reference to modern. I also typed modern into the search bar at the top and don't see anything there either.

bluepuma77 · January 30, 2024, 8:19pm

You assigned this, but does it exist (doc)?

The options field enables fine-grained control of the TLS parameters. It refers to a TLS Options and will be applied only if a Host rule is defined.

modern is not a fixed setting, but a name for something you need to define.

## Dynamic configuration
http:
  routers:
    Router-1:
      rule: "Host(`foo-domain`) && Path(`/foo-path/`)"
      service: service-id
      # will terminate the TLS request
      tls:
        options: foo

tls:
  options:
    foo:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

ZerkerEOD · January 30, 2024, 8:59pm

Got it, yes I have another file in my config file called dynamic.yml which when I originally put all of it together a year or two ago, I pulled it from someone's github and changed my basic auth. Here is what is defined in that file.

http:
  middlewares:
#    # A basic authentification middleware, to protect the Traefik dashboard to anyone except myself
#    # Use with traefik.http.routers.myRouter.middlewares: "traefikAuth@file"
    traefikAuth:
      basicAuth:
        users:
          - "REDACTED"

    # Recommended default middleware for most of the services
    # Use with traefik.http.routers.myRouter.middlewares: "default@file"
    # Equivalent of traefik.http.routers.myRouter.middlewares: "default-security-headers@file,error-pages@file,gzip@file"
    default:
      chain:
        middlewares:
          - default-security-headers
#          - error-pages
          - gzip

    # Add automatically some security headers
    # Use with traefik.http.routers.myRouter.middlewares: "default-security-headers@file"
    default-security-headers:
      headers:
        browserXssFilter: true                            # X-XSS-Protection=1; mode=block
        contentTypeNosniff: true                          # X-Content-Type-Options=nosniff
        forceSTSHeader: true                              # Add the Strict-Transport-Security header even when the connection is HTTP
        frameDeny: true                                   # X-Frame-Options=deny
        referrerPolicy: "strict-origin-when-cross-origin"
        sslRedirect: true                                 # Allow only https requests
        stsIncludeSubdomains: true                        # Add includeSubdomains to the Strict-Transport-Security header
        stsPreload: true                                  # Add preload flag appended to the Strict-Transport-Security header
        stsSeconds: 63072000                              # Set the max-age of the Strict-Transport-Security header (63072000 = 2 years)

    # Serve the error pages when the status is included inside the following ranges
    # Use with traefik.http.routers.myRouter.middlewares: "error-pages@file"
#    error-pages:
#      errors:
#        query: "erreur{status}/"
#        service: traefik-error-pages
#        status:
#          - "403-404"
#          - "500"
#          - "503"

    # Enables the GZIP compression (https://docs.traefik.io/middlewares/compress/)
    #   if the response body is larger than 1400 bytes
    #   if the Accept-Encoding request header contains gzip
    #   if the response is not already compressed (Content-Encoding is not set)
    # Use with traefik.http.routers.myRouter.middlewares: "gzip@file"
    gzip:
      compress: {}

  serversTransports:
    internalinsecure:
      insecureSkipVerify: true

# See https://doc.traefik.io/traefik/https/tls/
tls:
  options:
    # To use with the label "traefik.http.routers.myrouter.tls.options=modern@file"
    modern:
      minVersion: "VersionTLS13"                          # Minimum TLS Version
      sniStrict: true                                     # Strict SNI Checking
#    
#    # To use with the label "traefik.http.routers.myrouter.tls.options=intermediate@file"
    intermediate:
      cipherSuites:
        - "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
        - "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
        - "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
        - "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
        - "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305"
        - "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305"
      minVersion: "VersionTLS12"                          # Minimum TLS Version
      sniStrict: true                                     # Strict SNI Checking
#    
#    # To use with the label "traefik.http.routers.myrouter.tls.options=old@file"
    old:
      cipherSuites:
        - "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
        - "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
        - "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
        - "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
        - "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305"
        - "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305"
        - "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"
        - "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256"
        - "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA"
        - "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA"
        - "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA"
        - "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA"
        - "TLS_RSA_WITH_AES_128_GCM_SHA256"
        - "TLS_RSA_WITH_AES_256_GCM_SHA384"
        - "TLS_RSA_WITH_AES_128_CBC_SHA256"
        - "TLS_RSA_WITH_AES_128_CBC_SHA"
        - "TLS_RSA_WITH_AES_256_CBC_SHA"
        - "TLS_RSA_WITH_3DES_EDE_CBC_SHA"
      minVersion: "TLSv1"                                 # Minimum TLS Version
      sniStrict: true                                     # Strict SNI Checking

ZerkerEOD · January 30, 2024, 9:28pm

I have also confirmed, if the nextcloud config is present at service start nothing works. If I rename it to have a .bak extension and restart the service (not just save it and it not seen anymore) then all my other services come up. If I comment out the passthrough line and restart the service everything works except for my nextcloud instance, I can no longer reach it because it isn't passing my TLS through.

bluepuma77 · January 31, 2024, 6:48am

Back to start. passthrough is intended for tcp routers, not http routers.

For http routers, usually Host() is used.

For your setup to work, both Traefik and target service need to have access to the TLS cert, otherwise HostSNI() does not work.

Maybe a different approach would be easier, to configure nextcloud to use no TLS internally.

According to nextcloud all-in-one (doc) the container provides plain http access.

ZerkerEOD · February 1, 2024, 11:32pm

It used to be a TCP socket, but I changed it about a year ago and don't remember why. It didn't break until just recently though.

Topic		Replies	Views
Secure Dashboard Code is not working Traefik v2 (latest) file	3	416	December 7, 2019
Error middleware in static Traefik v2 (latest) middleware	0	300	March 30, 2020
Field not found, node: stripPrefix Traefik v2 (latest) middleware , cli	1	410	May 17, 2023
Traefik reads but doesn't load dynamic configuration Traefik v2 (latest) file	3	240	December 13, 2023
One more Time: 404 for Dashboard Traefik v2 (latest) docker	2	105	April 1, 2024

Dynamic config breaking all sites?

Related Topics