Traefik completely unresponsive after 163 lines of log

Traefik Traefik v2
1

Hi,

I have no clue as to what happened, but traefik just stopped.
He served me with 163 lines of log and that's it.

He added routes for the different subs.
Then went through the certificates challenges part and stopped somewhere at the end of it.

The last thing was
No ACME certificate generation required for domains .... and that was the last of the subdomain, that I can confirm.

And that's it. Not a single actual subdomain is reachable.
Nothing happens.

I don't have the first single idea of what to do.

Any help would be greatly appreciated.

Cheers.

2

And we don’t have an idea of how to help you if you don’t share your Traefik static and dynamic config, and docker-compose.yml if used :slight_smile:.

Have you enabled Traefik dashboard to get some insights beside the log?

3

Here they are:

docker-compose :

 # TRAEFIK     TRAEFIK OK
  traefik:
    image: "traefik:2.7.3"
    container_name: "traefik"
    labels:
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true
      - traefik.http.routers.api.rule=Host(`heimdal.${DOMAINNAME}`)
      - traefik.http.routers.api.entrypoints=websecure
      - traefik.http.routers.api.middlewares=chain-authelia@file
      - traefik.http.routers.api.service=api@internal
      - traefik.http.routers.api.tls=true
      - traefik.http.routers.api.tls.certresolver=letsEncrypt
      - traefik.http.services.api.loadbalancer.server.port=8083
      - traefik.port=8083
    ports:
      - "80:80"
      #- "8080:8080"
      - "443:443"
    environment:
       - DOCKER_HOST=tcp://socket-proxy:2375
       - HTPASSWD_FILE=/run/secrets/.htpasswd
    #   - NAMECHEAP_API_KEY=/run/secrets/namecheap_api_key
    #   - NAMECHEAP_API_USER=/run/secrets/namecheap_api_user
       - NAMECHEAP_API_KEY=keymjvnblkmjvnmlvnldksfn
       - NAMECHEAP_API_USER=username
    volumes:
     # - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - ${APP_DATA_DIR}/traefik:/etc/traefik
      - ${APP_DATA_DIR}/traefik/acme.json:/acme.json
      - ${APP_DATA_DIR}/traefik/rules:/rules
      - ${SHARED_DIR}:/shared
      - ${APP_DATA_DIR}/secrets:/secrets
     # - TZ=${TZ}
    secrets:
      - htpasswd
      - namecheap_api_user
      - namecheap_api_key  
    networks:
      - socket_proxy
      - web
      - internal
    restart: unless-stopped

traefik.yml

global:
  checkNewVersion: false
  sendAnonymousUsage: false
entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
    http:
      tls:
        certResolver: letsEncrypt
        domains:
          - main: "mydomain.com"
            sans:
              - "*.mydomain.com"


certificatesResolvers:
  letsEncrypt:
    acme:
      #caServer: "https://acme-v02.api.letsencrypt.org/directory"
      #caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      email: myemail@myemailprovider.com
      storage: /acme.json
      #keyType: EC384
      dnsChallenge:
        provider: domainprovider
                

providers:
  docker:
    endpoint: tcp://socket-proxy:2375
    exposedByDefault: false
  file:
    directory: /rules
    watch: true
  #file:
  #  directory: /etc/traefik/dynamic/
api:
  #insecure: true
  dashboard: true
log:
  filePath: /etc/traefik/traefik.log
  format: common #json
  level: DEBUG

accessLog:
  filePath: /etc/traefik/access.log
  format: json
  filters:    
    statusCodes:
      - "200"
      - "300-302"
    retryAttempts: true
    minDuration: "10ms"

middleware-chains.yml

http:
  middlewares:
    chain-no-auth:
      chain:
        middlewares:
          - middlewares-rate-limit
          - middlewares-secure-headers

    chain-basic-auth:
      chain:
        middlewares:
          - middlewares-rate-limit
          - middlewares-secure-headers
          - middlewares-basic-auth

    chain-oauth:
      chain:
        middlewares:
          - middlewares-rate-limit
          - middlewares-secure-headers
          - middlewares-oauth
        
    chain-authelia:
      chain:
        middlewares:
          - middlewares-rate-limit
          - middlewares-secure-headers
          - middlewares-authelia

middlewares.yml:

http:
  middlewares:
    middlewares-basic-auth:
      basicAuth:
        # users:
        #   - "user:$apsdfs.$EntPC0w3FtswWvC/6fTVJ7IUVtX1"
        usersFile: "/secrets/.htpasswd" #be sure to mount the volume through docker-compose.yml
        realm: "Traefik 2 Basic Auth"

    middlewares-rate-limit:
      rateLimit:
        average: 100
        burst: 50

    middlewares-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlMaxAge: 100
        hostsProxyHeaders:
          - "X-Forwarded-Host"
          #- "HTTP_X_FORWARDED_PROTO"
        sslRedirect: true
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        forceSTSHeader: true
        # frameDeny: true #overwritten by customFrameOptionsValue
        #customFrameOptionsValue: "allow-from https:example.com" #CSP takes care of this but may be needed for organizr.
        contentTypeNosniff: true
        browserXssFilter: true
        # sslForceHost: true # add sslHost to all of the services
        # sslHost: "example.com"
        referrerPolicy: "same-origin"
        # Setting contentSecurityPolicy is more secure but it can break things. Proper auth will reduce the risk.
        # the below line also breaks some apps due to 'none' - sonarr, radarr, etc.
        # contentSecurityPolicy: "frame-ancestors '*.example.com:*';object-src 'none';script-src 'none';"
        featurePolicy: "camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
        customResponseHeaders:
          X-Robots-Tag: "none,noarchive,nosnippet,notranslate,noimageindex,"
          server: ""

    middlewares-oauth:
      forwardAuth:
        address: "http://oauth:4181" # Make sure you have the OAuth service in docker-compose.yml
        trustForwardHeader: true
        authResponseHeaders:
          - "X-Forwarded-User"

    middlewares-authelia:
      forwardAuth:
        address: "http://authelia:9091/api/verify?rd=https://authelia.mydomain.com"
        trustForwardHeader: true
        authResponseHeaders:
          - "Remote-User"
          - "Remote-Groups"

I have enabled the traefik dashboard but it won't load.

4

see my other reply for the information you requested, that being said. Everything was fine, until it wasn't.
I assumed that maybe the fact that it stops every time precisely at the end of the certificates challenges phase could be important.

Is it possible that my ip / domain is targeted by some sort of massive access requests and it just gets overloaded ?

Anyway thanks for your insights.

5

Where is your traefik.yml static config (entrypoints, providers, etc), as you don’t use command:?

If you use Traefik Dashboard in insecure mode, you need to expose port 8080 in docker-compose.yml.

You use secrets, so you use Docker Swarm?

Use Traefik latest release version 2.9.6.

7

OK so about traefik.yml, sorry completely overlooked it,
It's been added to the post.

About dashboard, I will open a port, but since I had to restart it it's gonna take a while.
I left home and when I came back, traefik had gone through his init. But I restarted it it was stuck again, exactly at the same spot.

Concerning dynamic rules, the folder is empty.
As for secrets, I just use them because it is recommended but that is the only reason.

As for the version. I use the latest, however, since I had those issues, I thought that it could be relevant to traefik version so I downgrade it.