Redis failing to connect with domain name in TLS passthrough

Hello!

I set an instance of Redis on a Docker Swarm on my homelab that works with TLS passthrough. That's working fine, except that I'm able connect to Redis only using its container name or it's internal IP assigned by Docker, not by the domain name. At first I thought it was a DNS resolution problem, but it's nslookup returns fine, but the connection just timeout and nothing is being recorded on the Traefik log and on Redis log. Please, I need fresh eyes to see what it's happening, because it seems like something very simple and obvious and I'm not able to see it, even discern if this is from the Redis or from the Traefik side.

I'm using Traefik 3.3.2, Redis 7.4.2, Redis Insight 2.64.1, Docker 27.3.1, Docker-compose v2.31.0, running on Alpine Linux 3.21 VM's.

Traefik static configuration file (/etc/traefik/traefik.yml)

providers:
  providersThrottleDuration: 10
  docker:
    exposedByDefault: false 
    #constraints: foobar
    allowEmptyServices: true 
    network: traefik_proxy 
    useBindPortIP: false
    watch: true #Watch Docker events.
    #endpoint: Default unix:///var/run/docker.sock
    #endpoint=tcp://socket-proxy:2375 # Enable for Socket Proxy. Disable otherwise.
    #endpoint=ssh://traefik@192.168.2.5:2022 # Enable for SSH.
    #tls:
    #  ca: foobar
    #  cert: foobar
    #  key: foobar
    #  insecureSkipVerify: true
    httpClientTimeout: 
  swarm:
    exposedByDefault: false 
    #constraints: foobar
    allowEmptyServices: true
    network: traefik_proxy
    useBindPortIP: false 
    watch: true
    #defaultRule: 
    #endpoint: # Traefik v3 Swarm. Default tcp://127.0.0.1:2377
    #tls:
    #  ca: foobar
    #  cert: foobar
    #  key: foobar
    #  insecureSkipVerify: true
    httpClientTimeout: 0 
    refreshSeconds: 15 
    directory: /config/ 
    watch: true 
    debugLogGeneratedTemplate: true

certificatesResolvers: 
  step-ca:
    acme:
      email: redacted
      storage: /acme/stepca-acme.json
      caServer: "https://step-ca.server/acme/acme-tls/directory" 
      certificatesDuration: 24 
      tlsChallenge: {}
      #httpChallenge: #Can't use more than one DNS Challenge on Traefik. See https://doc.traefik.io/traefik/https/acme/#dnschallenge

entryPoints:

  web:
    address: ":80" # Create the HTTP entrypoint on port 80 and redirects to HTTPS
    allowACMEByPass: true
    asDefault: false
    http:
      redirections:
        entryPoint:
          to: websecure # The target element
          scheme: https # The redirection target scheme
          permanent: true # To apply a permanent redirection.
          priority: 42 #Priority of the generated router. Default=MaxInt-1

  websecure:
    address: ":443"
    allowACMEByPass: true
    asDefault: true
    proxyProtocol:
      insecure: false
      trustedIPs:
        - 10.0.0.0/8
        - 192.168.200.0/24
    forwardedHeaders:
      insecure: false
      trustedIPs:
        - 10.0.0.0/8
        - 192.168.200.0/24

  redis:
    address: ":6379" 
    asDefault: false

serversTransport:
  insecureSkipVerify: true 
  rootCAs:
    - /certs/ca-certificates.crt
    - /certs/root_ca.crt

tcpServersTransport:
  tls:
    insecureSkipVerify: true
    rootCAs:
      - /certs/ca-certificates.crt
      - /certs/root_ca.crt

Traefik dynamic configuration

tls: #https://doc.traefik.io/traefik/https/tls/
  options:
    default: 
      minVersion: VersionTLS13
      sniStrict: true 
      cipherSuites: 
        # TLS 1.3 cipher suites
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384
        - TLS_CHACHA20_POLY1305_SHA256
        # TLS 1.0 - 1.2 cipher suites
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
      curvePreferences:
        - CurveP521
        - CurveP384
      clientAuth:
        caFiles: 
          - /certs/root_ca.crt
          #- /certs/ca-certificates.crt
        clientAuthType: RequestClientCert

tcp:
  routers:
    portainer:
      entryPoints:
        - "websecure"
      middlewares:
        - tcp-limitconnection
        #- tcp-ipallowlist
      service: portainer-service
      rule: "HostSNI(`portainer.server`)"
      tls:
        passthrough: true
        certResolver: ~
        domains:
          - main: "portainer.server"
    redis: 
      entryPoints:
        - "redis"
      #middlewares:
        #- tcp-limitconnection
        #- tcp-ipallowlist
      service: redis-service
      rule: "HostSNI(`redis.server`)"
      tls:
        passthrough: true
        certResolver: ~
        domains:
          - main: "redis.server"

  services:
     portainer-service:
      loadBalancer:
        servers:
          - address: "portainer_portainer:9443" #container_name and port. 
            tls: true
     redis-service:
      loadBalancer:
        servers:
          - address: "redis_server:6379" 
            tls: true

This is redis_stack.conf:

bind * -::*
protected-mode yes
tcp-backlog 511
timeout 0
tcp-keepalive 300
port 0
tls-port 6379
tls-cert-file /run/tls/server.crt 
tls-key-file /run/tls/server.key 
tls-ca-cert-file /run/tls/ca.crt
tls-auth-clients optional
tls-protocols "TLSv1.2 TLSv1.3"
daemonize no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "/data/redis.log"
syslog-enabled yes
databases 16
always-show-logo no
set-proc-title yes
proc-title-template "{title} {listen-addr} {server-mode}"
locale-collate ""
save 3600 1 300 100 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir ./
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync yes
repl-diskless-sync-delay 5
repl-diskless-sync-max-replicas 0
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100
acllog-max-len 128
aclfile /etc/redis/users.acl
maxclients 42
maxmemory 256mb
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
lazyfree-lazy-user-flush no
oom-score-adj no
oom-score-adj-values 0 200 800
disable-thp yes
appendonly yes
appendfilename "appendonly.aof"
appenddirname "appendonlydir"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 10mb
aof-load-truncated yes
aof-use-rdb-preamble yes
aof-timestamp-enabled no
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-listpack-entries 512
hash-max-listpack-value 64
list-max-listpack-size -2
list-compress-depth 0
set-max-intset-entries 512
set-max-listpack-entries 128
set-max-listpack-value 64
zset-max-listpack-entries 128
zset-max-listpack-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
jemalloc-bg-thread yes

This is the Redis stack deploy:

version: "3.9"

services:
  server:
    #image: redis/redis-stack:latest #contains both Redis Stack server and Redis Insight. This container is best for local development because you can use the embedded Redis Insight to visualize your data.
    image: redis/redis-stack-server #provides Redis Stack server only. This container is best for production deployment.
    container_name: redis_server
    security_opt:
      - no-new-privileges:true      
    deploy:
      mode: replicated #global or replicated
      replicas: 1 #If global, comment it!
      placement:
        constraints: 
         #- node.role == manager
         #- node.hostname == docker-manager01
          - node.platform.os == linux
      restart_policy:
        condition: on-failure
        delay: 15s
        max_attempts: 10
        window: 300s
      resources:
        limits:
          memory: 1G
    user: "5001:5001"      
    #healthcheck: # Healthcheck fails when using TLS. Investigate later.
      #test:
        #- CMD-SHELL
        #- redis-cli ping | grep PONG
      #start_period: 20s
      #interval: 30s
      #retries: 5
      #timeout: 3s
    #ports:
      #- '6379:6379' #Server
    volumes:
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/data:/data:rw 
      - ${NFS_MOUNT_VOLUME}/step-ca/certs/root_ca.crt:/run/tls/ca.crt:ro
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/certs/server.crt:/run/tls/server.crt:ro
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/certs/server.key:/run/tls/server.key:ro
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/config/redis-stack.conf:/redis-stack.conf:ro
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/config/users.acl:/etc/redis/users.acl:rw
    environment:
      - TZ = America/Sao_Paulo      
      - TLS_DOMAINS=redis.server
    networks:
      - traefik_proxy
    
networks:
  traefik_proxy:
    external: true

Enable and check Traefik debug log (doc) and Traefik access log in JSON format (doc).

I have never seen this in Traefik config:

For redis to work through Traefik, the client needs to support TLS with SNI.

Also Traefik needs to have TLS certs available. If not, you should use an dedicated port and just pass TCP through.

For TCP to pass through untouched, you should not use any TLS settings, as that can trigger Traefik to create a custom cert that is not trusted by the client/browser.

No need for tls.passthrough, as a plain TCP connection will do exactly that. And you need to use rule: HostSNI(`*`), which will not try to decode the request.

Maybe check simple Traefik TCP example.

Hi,

Already do that. I get no errors, the only thing Traefik notices when there is a connection it shows only that:

{"level":"debug","address":"redis_server:6379","remoteAddr":"10.0.0.2:49418","time":"2025-01-17T13:56:25-03:00","caller":"github.com/traefik/traefik/v3/pkg/tcp/proxy.go:41","message":"Handling TCP connection"}

You're right. I know I shouldn't use any settings there, probably I read somewhere to deactivate it and I forgot to comment it, but I didn't know that I don't need tls.passthrough also, I assumed that I should be explicit about that. Well, I commented both and also changed HostSNI to HostSNI(*) (w/ quotes around *), but nothing changed, the log shows the same message as above.

I looked at your examples and didn't notice anything different from what I'm using. If Traefik's response is the same as the one above, can I assume that Traefik is working fine and the problem is on the Redis side? I've read other posts, some of which you even answered yourself, and I don't think I oversight something.

What intrigues me is that it shouldn't be a problem with certificates, otherwise Redis wouldn't accept making the connection by IP or by container name, or at least I assume not.

Share your full Traefik static and dynamic config, and docker-compose.yml if used.

Ok, sharing full config:

Traefik Static

## Global
# ---------------------------------------------------------------------

global:
  checkNewVersion: true
  sendAnonymousUsage: false

# ---------------------------------------------------------------------
## Observability
# ---------------------------------------------------------------------

api:
  insecure: false
  dashboard: true
  debug: false
  disableDashboardAd: true

ping:
  #entrypoint: "traefik_ping"
  manualRouting: false
  terminatingStatusCode: 503

# ---------------------------------------------------------------------
## Observability
# ---------------------------------------------------------------------

log:
  level: DEBUG
  format: json #json or common
  noColor: false
  filePath: /logs/traefik.log
  maxSize: 5
  maxAge: 5
  maxBackups: 3
  compress: false

# Access Logs

accessLog:
  filePath: /logs/access.log
  format: common
  filters: 
    statusCodes:
      - "204-299"
      - "400-499"
      - "500-599"       
    retryAttempts: true
    minDuration: 42
  fields: #See the fields on https://doc.traefik.io/traefik/observability/access-logs/#limiting-the-fieldsincluding-headers
    defaultMode: keep
    names:
      StartUTC: drop #To correct access log presenting time in UTC instead of local.
      StartLocal: keep
      #ServiceName: keep
    headers: 
      defaultMode: drop
      #names: 
        #name0: foobar
        #name1: foobar
  bufferingSize: 100 # Configuring a buffer of 100 lines
  addInternals: true # Enables accessLogs for internal resources (e.g.: ping@internal).

# Metrics


#metrics:


# Tracing

# ---------------------------------------------------------------------
## Configuration Discovery - Providers
# ---------------------------------------------------------------------

providers:
  providersThrottleDuration: 10
  docker:
    exposedByDefault: false # Only expose container that are explicitly enabled (using label traefik.enabled)
    #constraints: foobar
    allowEmptyServices: true # If true, any servers load balancer defined for Docker containers is created regardless of the healthiness of the corresponding containers. If unhealthy, this results in 503 HTTP responses instead of 404 ones.
    network: traefik_proxy # Defines a default docker network to use for connections to all containers. This option can be overridden on a per-container basis with the traefik.docker.network label
    useBindPortIP: false #Traefik routes requests to the IP/port of the matching container. When true, you tell Traefik to use the IP/Port attached to the container's binding instead of its inner network IP/Port.
    watch: true #Watch Docker events.
    #defaultRule: #defines what routing rule to apply to a container if no rule is defined by a label.
    #endpoint: ${ENDPOINT_DOCKER} # Disable for Socket Proxy. Enable otherwise. Default unix:///var/run/docker.sock
    #endpoint=tcp://socket-proxy:2375 # Enable for Socket Proxy. Disable otherwise.
    #endpoint=ssh://traefik@192.168.2.5:2022 # Enable for SSH.
    #tls:
    #  ca: foobar
    #  cert: foobar
    #  key: foobar
    #  insecureSkipVerify: true
    httpClientTimeout: 0 #Defines the client timeout (in seconds) for HTTP connections. If its value is 0, no timeout is set.
  swarm:
    exposedByDefault: false # Only expose container that are explicitly enabled (using label traefik.enabled)
    #constraints: foobar
    allowEmptyServices: true
    network: traefik_proxy
    useBindPortIP: false #Traefik routes requests to the IP/port of the matching container. When true, you tell Traefik to use the IP/Port attached to the container's binding instead of its inner network IP/Port.
    watch: true
    httpClientTimeout: 0 #Defines the client timeout (in seconds) for HTTP connections. If its value is 0, no timeout is set.
    refreshSeconds: 15 #Defines the polling interval (in seconds) for Swarm Mode.
  file:
    directory: /config/ # Defines the path to the directory that contains the configuration files. The filename and directory options are mutually exclusive. It is recommended to use directory.
    watch: true # Allow Traefik to automatically watch for file changes. It works with both the filename and the directory options.
    #filename: /dynamic.yml # Link to the dynamic configuration
    debugLogGeneratedTemplate: true

# ---------------------------------------------------------------------
## HTTP & HTTPS
# ---------------------------------------------------------------------

certificatesResolvers: #acme.json files must have chmod 600, not more, not less - traefik will shutdown the resolver if something goes wrong.
  cloudflare:
			#(supressed, not being used)
  step-ca:
    acme:
      email: redacted
      storage: /acme/stepca-acme.json
      caServer: "https://step-ca.domain/acme/acme-tls/directory" 
      certificatesDuration: 24 # Means 24 hours, which should match with the CA configuration.
      tlsChallenge: {}
      #httpChallenge: #Can't use more than one DNS Challenge on Traefik. See https://doc.traefik.io/traefik/https/acme/#dnschallenge
        #entryPoint: web #When using the HTTP-01 challenge, entrypoint must be reachable by Let's Encrypt through port 80.

# ---------------------------------------------------------------------
## Routing & Load Balancing
# ---------------------------------------------------------------------

entryPoints:
  
  web:
    address: ":80" # Create the HTTP entrypoint on port 80 and redirects to HTTPS
    allowACMEByPass: true
    #reusePort: true
    asDefault: false

    http:
      redirections:
        entryPoint:
          to: websecure # The target element
          scheme: https # The redirection target scheme
          permanent: true # To apply a permanent redirection.
          priority: 42 #Priority of the generated router. Default=MaxInt-1

  websecure:
    address: ":443"
    allowACMEByPass: true
    #reusePort: true
    asDefault: true
    proxyProtocol:
      insecure: false
      trustedIPs:
        - 10.0.0.0/8
        - 192.168.200.0/24
    forwardedHeaders:
      insecure: false
      trustedIPs:
        - 10.0.0.0/8
        - 192.168.200.0/24

  redis:
    address: ":6379" 
    asDefault: false

serversTransport:
  insecureSkipVerify: true # Disables certificate verification for back-end servers. Accepts any certificate presented by the server regardless of the hostnames it covers - https://doc.traefik.io/traefik/providers/docker/#insecureskipverify
  rootCAs:
    - /certs/ca-certificates.crt
    - /certs/root_ca.crt

tcpServersTransport:
  tls:
    insecureSkipVerify: true
    rootCAs:
      - /certs/ca-certificates.crt
      - /certs/root_ca.crt

experimental:
  plugins:
    crowdsec-bouncer: #https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
      moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
      version: "v1.3.5"
    sablier: # https://acouvreur.github.io/sablier/#/
      moduleName: "github.com/sablierapp/sablier" 
      version: "v1.8.1"

Traefik Dynamic

# Traefik Dynamic Configuration File
# See some references and examples on those sites
# https://doc.traefik.io/traefik/reference/dynamic-configuration/file/

http:
 #region HTTP routers
  routers:
   
#Swarm services running on Docker. It is here and not on docker-compose labels because Sablier plugin

    freshrss-router: 
      entryPoints:
        - "websecure"
      rule: "Host(`freshrss.domain`)"
      middlewares:
        - freshrss-sablier
        - secured
      tls:
        certresolver: "step-ca"
        domains:
          - main: "freshrss.domain"
      service: freshrss-service 

    #redis:
      #entryPoints:
      #  - "redis"
      #rule: "Host(`redis.domain`)"
      #middlewares:
      #  - secured
      #tls:
      #  certresolver: "step-ca"
      #  domains:
      #    - main: "redis.domain"
      #service: redis-service

    redis_insight:
      entryPoints:
        - "websecure"
      rule: "Host(`redisinsight.domain`)"
      middlewares:
        - redis_insight-sablier
        - secured
      tls:
        certresolver: "step-ca"
        domains:
          - main: "redisinsight.domain"
      service: redis_insight-service

  services:  

    freshrss-service: 
      loadBalancer:
        servers:
          - url: "http://freshrss_freshrss:80" 
        passHostHeader: true

    #redis-service: 
      #loadBalancer:
        #servers:
        #  - url: "http://redis_server:6379" 
        #passHostHeader: true

    redis_insight-service: 
      loadBalancer:
        servers:
          - url: "http://redis_insight:5540" 
        passHostHeader: true

  middlewares:
    middlewares-rate-limit: #This protects from DDOS attacks - https://doc.traefik.io/traefik/middlewares/http/ratelimit/
      rateLimit:
        average: 100
        burst: 50

    middlewares-buffering: #The Buffering middleware limits the size of requests that can be forwarded to services. - https://doc.traefik.io/traefik/middlewares/http/buffering/
      buffering:
        maxRequestBodyBytes: 1000000  # Maximum allowed body size for the request (in bytes).
        memRequestBodyBytes: 1048576   # Threshold (in bytes) from which the request will be buffered on disk instead of in memory 
        maxResponseBodyBytes: 1000000 # Maximum allowed response size from the service (in bytes). If the response exceeds the allowed size, it is not forwarded to the client
        memResponseBodyBytes: 2097152  # Threshold (in bytes) from which the response will be buffered on disk instead of in memory
        retryExpression: "IsNetworkError() && Attempts() <= 2"

    middlewares-compression: # https://doc.traefik.io/traefik/middlewares/http/compress/
      compress: #supports Gzip, Brotli and Zstandard compression. Note that application/grpc is never compressed.
        defaultEncoding: brotli #specifies the default encoding if the Accept-Encoding header is not in the request or contains a wildcard (*). There is no fallback on the defaultEncoding when the header value is empty or unsupported.
        minResponseBodyBytes: 1024 #specifies the minimum amount of bytes a response body must have to be compressed. Responses smaller than the specified values will not be compressed.
        #excludedContentTypes:
          #- "image/jpeg"
          #- "image/png"
          #- "image/gif"
          #- "image/webp"
          #- "video/mp4"
          #- "video/webm"
          #- "application/zip"
          #- "application/gzip"
          #- "application/pdf"
          #- "application/octet-stream" # Generic binary files
          #- "application/x-bzip"       # Bzip archives
          #- "application/x-bzip2"      # Bzip2 archives
          #- "application/x-tar"        # Tar archives
          #- "audio/mpeg"               # MP3 files
          #- "audio/ogg"                # Ogg audio
          #- "audio/wav"                # Wav audio
          #- "audio/webm"               # WebM audio
        includedContentTypes:
          - "text/html"              
          - "text/css"               
          - "application/javascript" 
          - "application/json"       
          - "application/xml"        
          - "text/plain"             
          - "text/csv"               
          - "application/xhtml+xml"  
          - "application/rss+xml"    
          - "application/atom+xml"   
          - "application/wasm"       # WebAssembly binaries (may be pre-compressed, but still often compressed)        

    https-redirectscheme:
      redirectScheme:
        scheme: https
        permanent: true

    default-headers:  
      headers:
        # https://doc.traefik.io/traefik/middlewares/http/overview/#
        # https://github.com/unrolled/secure#available-options
        # https://securityheaders.com/ 
        # https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html 
        # https://www.hardenize.com/
        # CSP is not easy to implement - check https://www.paulsblog.dev/harden-your-website-with-traefik-and-security-headers/
        # https://github.com/unrolled/secure#available-options
                
        #XSS Filters
        frameDeny: true #This protects against clickjacking attacks, where a malicious site could trick users into interacting with your site without their knowledge.
        browserXssFilter: true #Enables the X-XSS-Protection header, which instructs the browser to activate a cross-site scripting (XSS) filter. 
        
        #HSTS Headers
        forceSTSHeader: true #Enables the Strict-Transport-Security (STS) header
        stsIncludeSubdomains: true #Extends the HSTS policy (set by Strict-Transport-Security) to include all subdomains, ensuring that they also require HTTPS connections.
        stsPreload: true #Indicates that your domain should be included in the HSTS preload list maintained by browsers. 
        stsSeconds: 15552000 #180d - This is the duration for which the browser should remember that your site is only accessible over HTTPS.
        
        #Cross-Origin Resource Sharing (CORS) Headers
        accessControlAllowMethods: GET, OPTIONS, PUT #new
        #accessControlAllowHeaders: "*"
        #accessControlAllowOriginList:
        #  - https://foo.bar.org
        #  - https://example.org
        accessControlMaxAge: 100
        addVaryHeader: true #New

        #contentsecuritypolicy=default-src 'none'; img-src 'self' https://i.postimg.cc; script-src 'self'; style-src 'self'

        sslproxyheaders:
          X-Forwarded-Proto: https #New
        hostsproxyheaders: X-Forwarded-Host #New
        PermissionsPolicy: ""
        referrerpolicy: same-origin #new
        contentTypeNosniff: true #This sets the X-Content-Type-Options header to nosniff, preventing browsers from trying to guess the MIME type of a file
        customFrameOptionsValue: SAMEORIGIN #This is an alternative to frameDeny, where instead of blocking all framing, it allows the content to be framed only by pages on the same origin. This still protects against clickjacking while allowing legitimate framing by your own pages.
        customResponseHeaders: # New
            X-Robots-Tag: none,noarchive,nosnippet,notranslate,noimageindex # New
        customRequestHeaders: 
          X-Forwarded-Proto: https #Adds or overrides the X-Forwarded-Proto header, setting its value to https. This informs backend services that the original request was made over HTTPS
          server: "" #This removes the Server header from responses, which usually reveals the web server software being used
          x-powered-by: "" #This removes the X-Powered-By header, which typically indicates the technology or framework used to build the application (like PHP, ASP.NET, etc.).    
    
    default-whitelist:
      ipWhiteList:
        sourceRange:
        - "192.168.200.0/24"
        - "10.0.0.0/8"
        - "172.16.0.0/12"
        #- "192.168.0.0/16"

    secured:
      chain:
       middlewares:
        #- default-whitelist
        - default-headers
        #- middlewares-compression
        - middlewares-rate-limit
        - middlewares-crowdsec-bouncer
        - https-redirectscheme
        
#region Sablier middlewares

    freshrss-sablier: #The name of middleware. One middleware per application, so repeat it for every app that will use that service. ---> ${MY_CONTAINER}-sablier-mid
          plugin:
            sablier: 
              sablierUrl: http://sablier:10000  # The sablier URL service, must be reachable from the Traefik instance
              #names: foobar              # Comma separated names of containers/services/deployments etc. They will start together!
              group: freshrss            # You can use groups, instead of names. This is easier because you don't need to know the name of each container.
              sessionDuration: 60m               # The session duration after which containers/services/deployments instances are shutdown
              dynamic: # Dynamic Middleware - This strategy provides a waiting page for your session. It's suited for a user that would access a frontend directly and expects to see a loading page.
                displayName: FreshRSS       # (Optional) Defaults to the middleware name
                showDetails: true           # (Optional) Set to true or false to show details specifcally for this middleware, unset to use Sablier server defaults
                theme: shuffle      # (Optional) The theme to use
                refreshFrequency: 5s        # (Optional) The loading page refresh frequency
              #blocking: # Blocking Middleware - Hang the request until services are up and running but will not wait more than `timeout`. It's suited for an API communication.
              #  timeout: 1m

    redis_insight-sablier:
          plugin:
            sablier: 
              sablierUrl: http://sablier:10000  
              #names: foobar              
              group: redis_insight           
              sessionDuration: 30m              
              dynamic: 
                displayName: Redis Insight 
                showDetails: true          
                theme: shuffle      
                refreshFrequency: 5s
              #blocking: 
              #  timeout: 1m

    middlewares-crowdsec-bouncer: #https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin#variables
          plugin:
            crowdsec-bouncer:
                # Those variables can be provided with the content as raw or through a file path that Traefik can read.
                # - CrowdsecLapiTlsCertificateBouncerKey
                # - CrowdsecLapiTlsCertificateBouncer
                # - CrowdsecLapiTlsCertificateAuthority
                # - CrowdsecCapiMachineId
                # - CrowdsecCapiPassword
                # - CrowdsecLapiKey
                # - CaptchaSiteKey
                # - CaptchaSecretKey 
                # The file variable will be used as preference if both content and file are provided for the same variable. Format is:
                #  - Content: VariableName: XXX
                #  - File : VariableNameFile: /path
              
              enabled: true #enable the plugin
              logLevel: DEBUG #default: INFO, expected values are: INFO, DEBUG, ERROR, log are written to stdout / stderr
              updateIntervalSeconds: 60 #default: 60 Used only in stream mode, the interval between requests to fetch blacklisted IPs from LAPI
              updateMaxFailure: 0 #default: 0 Used only in stream and alone mode, the maximum number of time we can not reach Crowdsec before blocking traffic (set -1 to never block)
              defaultDecisionSeconds: 60 #default: 60 Used only in live mode, maximum decision duration
              httpTimeoutSeconds: 10 #default: 10 Default timeout in seconds for contacting Crowdsec LAPI
              crowdsecMode: stream #default: live, expected values are: none, live, stream, alone, appsec
              
              crowdsecAppsecEnabled: false #Enable Crowdsec Appsec Server (WAF).
              crowdsecAppsecHost: crowdsec:7422 #Crowdsec Appsec Server available on which host and port. The scheme will be handled by the CrowdsecLapiScheme var.
              crowdsecAppsecFailureBlock: true #default true. Block request when Crowdsec Appsec Server have a status 500
              crowdsecAppsecUnreachableBlock: true
              
              crowdsecLapiKey: eYkBV+Dr2OhBk6fXgBY3kIVidMXpr5tAKITvf5v57Fs #Crowdsec LAPI key for the bouncer.
              #crowdsecLapiKeyFile: /run/docker/CROWDSEC_TRAEFIK_LAPI_KEY
              crowdsecLapiHost: 192.168.200.120:8080 #Crowdsec LAPI available on which host and port.
              crowdsecLapiScheme: http #default: http, expected values are: http, https
              crowdsecLapiTLSInsecureVerify: false #default: false. Disable verification of certificate presented by Crowdsec LAPI
              
              #crowdsecCapiMachineId: login #Used only in alone mode, login for Crowdsec CAPI
              #crowdsecCapiPassword: password #Used only in alone mode, password for Crowdsec CAPI
              #crowdsecCapiScenarios: #Used only in alone mode, scenarios for Crowdsec CAPI
              #  - crowdsecurity/http-path-traversal-probing
              #  - crowdsecurity/http-xss-probing
              #  - crowdsecurity/http-generic-bf
              
              #forwardedHeadersTrustedIPs: #List of IPs of trusted Proxies that are in front of traefik (ex: Cloudflare)
                #- 10.0.10.23/32
                #- 10.0.20.0/24
              clientTrustedIPs: #List of client IPs to trust, they will bypass any check from the bouncer or cache (useful for LAN or VPN IP)
                - 10.0.0.0/8
                - 192.168.200.0/24
                - 172.16.0.0/12
              forwardedHeadersCustomName: #default: "X-Forwarded-For" Name of the header where the real IP of the client should be retrieved
              
              #redisCacheEnabled: false #default: false enable Redis cache instead of in-memory cache
              #redisCacheHost: "redis:6379" #default: "redis:6379" hostname and port for the Redis service
              #redisCachePassword: password #Password for the Redis service
              #redisCacheDatabase: "5" #Database selection for the Redis service
            
              #crowdsecLapiTLSCertificateAuthority: #default: "" PEM-encoded Certificate Authority of the Crowdsec LAPI
              #crowdsecLapiTLSCertificateAuthorityFile: /etc/traefik/crowdsec-certs/ca.pem
              #crowdsecLapiTLSCertificateBouncer: #default: "" PEM-encoded client Certificate of the Bouncer
              #crowdsecLapiTLSCertificateBouncerFile: /etc/traefik/crowdsec-certs/bouncer.pem
              #crowdsecLapiTLSCertificateBouncerKey: #default: "" PEM-encoded client private key of the Bouncer
              #crowdsecLapiTLSCertificateBouncerKeyFile: /etc/traefik/crowdsec-certs/bouncer-key.pem
             
              #captchaProvider: hcaptcha #Provider to validate the captcha, expected values are: hcaptcha, recaptcha, turnstile
              #captchaSiteKey: FIXME #Site key for the captcha provider
              #captchaSecretKey: FIXME #Site secret key for the captcha provider
              #captchaGracePeriodSeconds: 1800 #default: 1800 (= 30 minutes) Period after validation of a captcha before a new validation is required if Crowdsec decision is still valid
              #captchaHTMLFilePath: /captcha.html #default: /captcha.html Path where the captcha template is stored
              #banHTMLFilePath: "" #default: "" Path where the ban html file is stored (default empty ""=disabled)

tls: #https://doc.traefik.io/traefik/https/tls/
  options:
    default: # Any store definition other than the default one (named default) will be ignored, and there is therefore only one globally available TLS store.
      minVersion: VersionTLS13
      sniStrict: true #If enabled Traefik won't allow connections from clients that do not specify a server_name extension or don't match any of the configured certificates. 
      cipherSuites: #https://pkg.go.dev/crypto/tls?utm_source=godoc#pkg-constants
        # TLS 1.3 cipher suites
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384
        - TLS_CHACHA20_POLY1305_SHA256
        # TLS 1.0 - 1.2 cipher suites
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
      curvePreferences:
        - CurveP521
        - CurveP384
      clientAuth:
        caFiles: # For containers and root CA. In PEM format. Each file can contain multiple CAs.
          - /certs/root_ca.crt
          #- /certs/ca-certificates.crt
        clientAuthType: RequestClientCert

tcp:

  routers:

    portainer:
      entryPoints:
        - "websecure"
      middlewares:
        - tcp-limitconnection
        #- tcp-ipallowlist
      service: portainer-service
      rule: "HostSNI(`portainer.domain`)"
      tls:
        passthrough: true
        #certResolver: ~
        domains:
          - main: "portainer.domain"
  
    redis: 
      entryPoints:
        - "redis"
      #middlewares:
        #- tcp-limitconnection
        #- tcp-ipallowlist
      service: redis-service
      rule: "HostSNI(`*`)"
      #tls:
        #passthrough: true - supress this
        #certResolver: ~ supress this
        #domains:
        #  - main: "redis.domain"

#region TCP services  
  services:
    portainer-service: # Docker swarm service.
      loadBalancer:
        servers:
          - address: "portainer_portainer:9443" #container_name and port. 
            tls: true

    redis-service:
      loadBalancer:
        #proxyProtocol:
          #version: 2
        servers:
          - address: "redis_server:6379" 
            tls: true
    
#region TCP Middleware
  middlewares:
    tcp-limitconnection: #To proactively prevent services from being overwhelmed with high load, the number of allowed simultaneous connections by IP can be limited.
      inFlightConn:
        amount: 10
    tcp-ipallowlist:
      ipAllowList:
        sourceRange:
          - "192.168.200.0/24"
          - "10.0.0.0/8"
          - "172.16.0.0/12"

Traefik Docker compose

# Traefik
version: "3.8"

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    security_opt:
      - no-new-privileges:true
    networks:
      - traefik_proxy
    deploy:
      mode: replicated #global or replicated
      replicas: 1 #If global, comment it!
      placement:
        constraints: 
         - node.role == manager
         - node.hostname == docker-manager01
         - node.platform.os == linux
      restart_policy:
        condition: on-failure
        delay: 15s
        max_attempts: 10
        window: 300s
      labels:

      - "traefik.enable=true"
      - "traefik.http.routers.${MY_CONTAINER}.entrypoints=websecure"
      - "traefik.http.routers.${MY_CONTAINER}.rule=Host(`${MY_CONTAINER}.${DOMAIN_NAME}`)"
      - "traefik.http.services.${MY_CONTAINER}.loadbalancer.server.port=8080"
      - "traefik.http.routers.${MY_CONTAINER}.tls=true"
      - "traefik.http.routers.${MY_CONTAINER}.tls.certresolver=step-ca"
      - "traefik.http.routers.${MY_CONTAINER}.service=api@internal"
      - "traefik.http.routers.${MY_CONTAINER}.middlewares=secured@file"  # Modify here in the future to include a secure login middleware
              
    ports:
      - 80:80     # HTTP
      - 443:443   # HTTPS
      - 6379:6379 # Redis
     #- 8086:8086 # InfluxDB
    environment:
      - TZ=${TZ}
      - CF_API_EMAIL_FILE=/run/secrets/CF_API_EMAIL
      - CF_DNS_API_TOKEN_FILE=/run/secrets/CF_DNS_API_TOKEN
      - CF_ZONE_API_TOKEN_FILE=/run/secrets/CF_ZONE_API_TOKEN
      - BASIC_AUTH_TRAEFIK_2=/run/secrets/BASIC_AUTH_TRAEFIK
      - INFLUX_SERVER=https://influxdb.${DOMAIN_NAME}:8086
      - INFLUX_ORG=ogatodustin
      - INFLUX_BUCKET_TRAEFIK=${MY_CONTAINER}
      - INFLUXDB_TOKEN_TRAEFIK=/run/secrets/INFLUXDB_TOKEN_TRAEFIK
      - LEGO_CA_CERTIFICATES=/certs/root_ca.crt
    secrets:
      - CF_API_EMAIL
      - CF_DNS_API_TOKEN
      - CF_ZONE_API_TOKEN
      - INFLUXDB_TOKEN_TRAEFIK
      - BASIC_AUTH_TRAEFIK

    healthcheck:
      test: ["CMD", "traefik", "healthcheck", "--ping"]
      interval: 10s
      retries: 3
    
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro                     # Give access to the UNIX Docker sockets so that Traefik can listen to the Docker events
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/acme/:/acme/:rw              # Set the location where my ACME certificates are saved to. Set permissions chmod/chown 600 root:root
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/config/static.yml:/traefik.yml:ro  # Set the static configuration file
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/config/:/config/:ro          # Set the dynamic configuration for the file provider
      - ${NFS_MOUNT_VOLUME}/${MY_CONTAINER}/logs:/logs:rw
      - /etc/ssl/certs/ca-certificates.crt:/certs/ca-certificates.crt:ro
      - /root/.step/certs/root_ca.crt:/certs/root_ca.crt:ro

    command: # CLI arguments
      - --configFile=/config/static.yml
      - --entrypoints.websecure.http.tls.domains[0].main=${DOMAIN_NAME}
      - --entrypoints.websecure.http.tls.domains[0].sans=*.${DOMAIN_NAME}
      - --certificatesresolvers.cloudflare.acme.email=${CF_API_EMAIL_FILE} # Email address used for registration.
      - --certificatesresolvers.cloudflare.acme.caserver=${CASERVER_LETSENCRYPT} # CA server to use. Let's Encrypt's staging server - Comment the line to go prod
      #- --certificatesresolvers.cloudflare.acme.dnschallenge.resolvers=${FIREWALL_IP}:53 #1.1.1.1:53,8.8.8.8:53 Use following DNS servers to resolve the FQDN authority.
      - --certificatesresolvers.stepca.acme.email=${CF_API_EMAIL_FILE} # Email address used for registration.
      
  sablier:
    image: acouvreur/sablier:latest
    networks:
      - traefik_proxy
      - traefik_authelia
    deploy:
      mode: replicated #global or replicated
      replicas: 1 #If global, comment it!
      placement:
        constraints: 
         - node.role == manager
         - node.hostname == docker-manager01
         - node.platform.os == linux
      restart_policy:
        condition: on-failure
        delay: 15s
        max_attempts: 10
        window: 300s
      labels:
        - io.portainer.accesscontrol.users=graywolfrs
        
    environment:
      - TZ=${TZ}
      - PROVIDER_NAME=docker_swarm
      - SERVER_PORT=10000
      # The base path for the API
      - SERVER_BASE_PATH=/
      # File path to save the state (default stateless)
      - SERVER_STORAGE_FILE=stateless
      # The default session duration (default 5m)
      - SESSIONS_DEFAULT_DURATION=5m
      # The expiration checking interval. Higher duration gives less stress on CPU. If you only use sessions of 1h, setting this to 5m is a good trade-off.
      - EXPIRATION_INTERVAL=1m
      - LOGGING_LEVEL=trace
    command:
        - start
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
  
# Cannot use labels because as soon as the container is stopped, the labels are not read by Traefik as
# the route doesn't exist anymore. Use dynamic-config.yml file instead.

networks:
  traefik_proxy:
    external: true
  
secrets:
  CF_API_EMAIL:
      external: true
  CF_DNS_API_TOKEN:
      external: true
  CF_ZONE_API_TOKEN:
      external: true
  INFLUXDB_TOKEN_TRAEFIK:
      external: true
  BASIC_AUTH_TRAEFIK:
      external: true
  CROWDSEC_TRAEFIK_LAPI_KEY:
      external: true

Redis docker compose and .conf in the first post.

Traefik static config can be in traefik.yml or command:, not both, decide for one (doc).

I must admit that I was a little frustrated with your simple answer, especially because I believed the settings in command section did not overlap and apparently unrelated to the problem, but I got the message; I rolled up my sleeves and cleaned up the three files above with the Traefik documentation open in the window next to it, checking every item. Basically, I didn't change any setting, I just cleaned up the command to leave only --configFile=/config/static.yml and moved the redis configurations from dynamic to their respective compose, making it as simple as possible as you said: just entrypoint, HostSNI(*) and port, but without mentioning TLS, as I had tried in the past. I removed the serversTransport section from static, but I don't think that was the problem.

A KISS method instead of looking for a needle is sometimes enough and is a lesson in humility, and peer review help us to see the obvious. Thanks.

1 Like