Traefik 2 on Synology unable to connect to services on the host network

Traefik Traefik v2
,
1

Hi, I'm hoping someone with a lot more expertise than myself can help, Ive been stuck on this for the last 2 weeks. I feel like im getting close to getting this working and just need to get over this last hurdle.

I work as a developer and we have started using docker in work and so am comfortable with docker and programing in general however networking and certificates are all very new to me. I will try and explain the best i can but please feel free to correct all my mistakes/ misconceptions as thats the only way i will get to learn. I wanted to impove my skills in these areas hence setting all this up in my home network so what to learn and understand what im doing rather than just get it working blindly if that makes anysense.
Anyway enough of the rambling here the details:

What ive trying to achieve:
Route traffic from over https (443)
ha.example.com > Home Assistant
dsm.example.com > Synology DSM home page.

Setup
Synology IP > 192.168.0.10 (changed built-in ngix poxy ports 80 >81 &443> 444)
Home Assistant > Installed in docker, uses host network exposing port 8123
DSM > uses port 5000
Domain and DNS setup in cloudflare
Create A and CName records with proxied status proxied (have also tryed DNS only)
and have set SSL/TLS to Full
traefik installed in docker on Synology
taefik docker network 192.168.5.0/24 (gw 192.168.5.254) - bridge mode
router port forward enabled 443 > 192.168.0.10
Using yaml file to configure Home Assistant Service Rule

Whats working
Staring with the positives....
Any container i build with docker compose works (if on same traefik network)
Any services not using docker that are not on the same ip as the docker host

The Problem
When i try an access ha.example.com or dsm.example.com i get error 504 gateway times out.
I know i must be close as if i change the forwarding address to say my router IP it works.
As a test (as i would do this via docker compose than a file) I have also tried to point at another container i have installed that exposes port 8989 and if try pointing at 192.168.1.100:8989 it doesnt work ... however if i change this to point at the dockers container IP (192.168.5.2:8989) it works.
So basically it appears any port that is on the Synology / Docker host IP (192.168.0.10:XXXX) doesnt seem to work.
If tried replacing the docker host with 127.0.0.1 and 0.0.0.0 and same result
I sure there is something silly im overlooking here around the routing or maybe around the fact its gong from https externaly to http internally so could it be something with certificate termination ??? forgive me as my knowlege on certs / tls is very very limited and to be honest im suprised i have gotten this far.
My docker compose and yaml config fies are attached ... in there i have setup google oauth and this appears to be working, i have also tried without to try and reduce complication but didnt make a difference.

I've tried following the docs on traefik, numerous google serarchs and forum posts and tried so many things that would make this long post even longer. (Happy to retry all test again though)
It seems like it should work and looking at so many examples i dont see what i have done that is different to others that have it working.

Any help or pointers would be greatly apreciated or if anyone has some good reading material on certs, and reverse proxies in gerneal that would also be welcomed.

Thanks
Steve

Docker Compose

version: "2.1"

## NETWORKS
networks:
  traefik_network:
    external:
      name: traefik_network
  default:
    driver: bridge

## SERVICES
services:
  traefik:
    container_name: "traefik"
    image: "traefik:v2.2"
    restart: "always"
    mem_swappiness: -1
    security_opt:
      - "no-new-privileges:true"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"    # <--- Allows acces to docker to talk to another containers
      - "/volume2/Docker/Traefik/certs:/certs"            # <--- Location for the cerrs
      - "/volume2/Docker/Traefik/logs:/logs"              # <--- Location of the Logs
      - "/volume2/Docker/Traefik/rules:/rules"            # <--- Location of the dynamic Rules
    networks:
      traefik_network:
        ipv4_address: 192.168.5.1
    ports:
      - "80:80"        # <--- Web
      - "443:443"      # <--- WebSecure
      - "8080:8080"    # <--- Traefik Dashboard
    environment:
      - "CF_API_EMAIL=xxx"                   #  <--- Cloudflare username
      - "CF_API_KEY=xxx"    #  <--- Cloudflare Api key
    command:
    ## Logs
      - "--log=true"                       #  <--- Enabled logs
      - "--log.level=DEBUG"                #  <--- Log values: DEBUG, INFO, WARN, ERROR, FATAL, PANIC (Default: error)
      - "--log.format=json"                #  <--- Log format
      - "--log.filePath=/logs/traefik.log" #  <--- Location of the Logs

    ## Access Logs
      - "--accessLog=true "                             # <--- enable access logs
      - "--accessLog.filePath=/logs/traefik_access.log" # <--- Location of Log files
      - "--accessLog.bufferingSize=100"                 # <--- Configuring a buffer of 100 lines
      - "--accessLog.filters.statusCodes=400-49"        # <--- Only get codes in the 400 range

    ## API
      - "--api.insecure=true"
      - "--api.dashboard=true"
      - "--api=true"

    ## Entry Points - listen for incoming traffic (ports, ...)
      - "--entrypoints.websecure.address=:443"        # <--- Listen on port 443 for incoming requests. Friendly name websecure has been given
      # - "--entrypoints.traefik.address=:8080"
      # - "--entrypoints.web.address=:80"

      #### Add cloudflare as default certresolver for all services. Also enables TLS and no need to specify on individual services.
      - "--entrypoints.websecure.http.tls.certresolver=cloudflare"
      - "--entrypoints.websecure.http.tls.domains[0].main=example.com"
      - "--entrypoints.websecure.http.tls.domains[0].sans=*.example.com"
      - "--entrypoints.websecure.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22"

    ## Providers - discover the services that live on your infrastructure (their IP, health, ...)
      - "--providers.docker.network=traefik_network"
      - "--providers.docker=true"                     #  <--- Enable Traefik on Docker
      - "--providers.docker.exposedbydefault=false"   #  <--- Do not expose all Docker Containers by default
      - "--providers.file.directory=/rules"           #  <--- Folder to store rules in
      - "--providers.file.watch=true"                 #  <--- Only works on top level files in the rules folder

    ## DNS Challenge Challenge
      #### (cloudflare can be anything you want, called it cloudfare to give it some meaning)
      - "--certificatesresolvers.cloudflare.acme.dnschallenge=true"                   #  <--- What type of Cert Challenge to LetEncrypt ... DNS is used not HTTP or TLS
      - "--certificatesresolvers.cloudflare.acme.dnschallenge.provider=cloudflare"    #  <--- Where is the DNS Hosted (Used Cloudflare in this case)
      - "--certificatesresolvers.cloudflare.acme.email=xxxx"                          #  <--- Username for Cloudfare
      - "--certificatesresolvers.cloudflare.acme.storage=/certs/acme.json"            #  <--- Location of to Store the Cert

    labels:
    ## Enable Traefik
      - "traefik.enable=true"    #  <--- Enable traefik to manage this container
    ## Router
      - "traefik.http.routers.api.rule=Host(`traefik.example.com`)"
      - "traefik.http.routers.api.service=api@internal"
      - "traefik.http.routers.api.middlewares=google_oauth@file"

  google_oauth:
    container_name: google_oauth
    image: thomseddon/traefik-forward-auth:latest
    restart: always
    mem_swappiness: -1
    security_opt:
      - no-new-privileges:true
    environment:
      - CLIENT_ID=xxxx
      - CLIENT_SECRET=xxx
      - SECRET=xxx
      - COOKIE_DOMAIN=example.com
      - INSECURE_COOKIE=false
      - AUTH_HOST=oauth.example.com
      - URL_PATH=/_oauth
      - WHITELIST=xxx
      - LOG_LEVEL=warn
      - LOG_FORMAT=text
      - LIFETIME=2592000 # 30 days
      - DEFAULT_ACTION=auth
      - DEFAULT_PROVIDER=google
    labels:
      - "traefik.enable=true"

      ## HTTP Routers
      - "traefik.http.routers.oauth_router.entrypoints=websecure"
      - "traefik.http.routers.oauth_router.rule=Host(`oauth.example.com`)"
      - "traefik.http.routers.oauth_router.service=oauth_service"

      ## Middlewares
      - "traefik.http.routers.oauth_router.middlewares=google_oauth@file"

      ## HTTP Services
      - "traefik.http.services.oauth_service.loadbalancer.server.port=4181"

  whoami:
    image: "containous/whoami"
    container_name: "simple-service"
    labels:
      - "traefik.enable=true" #  <--- Enable traefik to manage this container

      # Define the routers (frontend) - Match the router.xxx. to the container name for easy management
      - "traefik.http.routers.whoami_router.rule=Host(`example.com`) && PathPrefix(`/whoami`)"  #  <--- What rule to match entry point too.
      - "traefik.http.routers.whoami_router.entrypoints=websecure"                                       #  <--- What entry point to check

      # Middlewares
      - "traefik.http.routers.whoami_router.middlewares=google_oauth@file"    #  <--- Call middleware authenitication (google oAuth)

      # Define the Services (backend)
      - "traefik.http.services.whoami_service.loadbalancer.server.port=80"   #  <--- Where to direct the service too

dynamic config file .yaml

http:
  routers:
    homeassistant_router:
      rule: "Host(`homeassistant.example.com`)"
      entrypoints:
        - "websecure"
      #middlewares:
        # - google_oauth
        # - chain-no-auth
      service: "homeassistant_service"
  services:
    homeassistant_service:
      loadBalancer:
        passHostHeader: true
        servers:
         # - url: "http://192.168.0.10:8123" # Home assistant home page (not working)
         # - url: "http://192.168.0.10:5000" # Synology DSM home page  (not working)
         # - url: "http://192.168.0.1"       # routers home page (works)
         # - url: "http://192.168.5.3:8989" # sonarr internal docker network (works)
          - url: "http://0.0.0.0:9003"       # sonarr docker host  (not working)
2

Thanks for the information sharing
vmware

3

i have it configured as followed:

http:
  routers:
    hass:
      entryPoints:
        - https
      service: service-hass
      rule: Host("hass.blahblah.nl")
      middlewares:
        - "hsts"
      tls: {}

  services:
    service-hass:
      loadBalancer:
        servers:
          - url: https://192.168.2.9:8123
        passHostHeader: true
        passTLSCert: true

i have used the following guide:

4

Thanks for the pointer.
I double checked my config file and made a few tweaks for replicate yours and im still getting 504 Error - gateway time-out.
I'll take a look at the link you provided now and do a bit more reading.

I think there is something on my synology NAS that is blocking or conflicting with it, that is causing issues.
its weird as i can get to http://192.168.0.10:8123 from any browser on the LAN. traefik just wont redirect to it for some reason.

5

are you using the homeassistant.example.com from inside your network?
then it could be that your router is not supporting nat loopback.

6

ah yes i am ... let me flip to 4g and test

7

tried that and still no joys. I read that article and cant see anything obvious that im doing wrong. Im in the middle of redoing the whole compose file at moment and going to more a file based approach like the article. Not that i think it will make a difference but i just want to follow that article exactly at the moment, then if it works i can reverse engineer it back to my approach to see where i went wrong :stuck_out_tongue:

thanks so much for your input and feedback, its much apreciated as i dont really know what im doing haha

8

The only other think i can think about is the firewall on your synology.

Are Home Assistant and traefik on the same nas? Maybe you can try 127.0.0.1:8123

9

yes both traefik and home assistant are installed on the same nas on the same docker host.

I've tried using 127.0.0.1 and 0.0.0.0 (not sure the difference between the 2 to be honest) and still had the same issue. I'll try again now though as i have made some changes so it might now work.

i've had a look around on synology front and cant see anything obvious there. Havent got firewall set as far as i can tell.

i did try and replicating the article but failed massively and kept getting ssl handshake issues, so have reverted back to original for now, as that seemed in a more working state as i couldnt even access the dashboard, so clearly messed something up there... I will attempt again either later this evening or tomorrow.
the only thing i can really see that is massively different is the tls file. Is this needed?

tls:
  options:
    TLSv13:
      minVersion: VersionTLS13
      cipherSuites:
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384
        - TLS_CHACHA20_POLY1305_SHA256
      sniStrict: true
        
    default:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
      sniStrict: true

Following the article i cant see where this file is called? so no idea where this would fit into my current setup... the rules folder?
In the hass file it sets tls to null by the looks of it so im really confussed by the point of the above file

  tls: {}

I am leaning towards there being something on the synology side of things, blocking or conflicting with what im trying to do

10

ok think you might be onto something with firewall or something blocking it ... i logged into the console for the traefik container and tried to ping 192.168.0.10 (nas) and i couldnt.
I can ping the router 192.168.0.1 and also another machine on my network just not the nas/docker host network ... thing is my networking skill are terrible as are my linux skills so not sure where to go from here.
i did an iptables from the docker host and cant see anything blocking or denying and iptables is not installed on traefik so it cant be that which is blocking the connection. any ideas?

11

ok i've resolved the issue.
@jonathanmos massive thankyou for pointing me in the right direction

So i had to reference the gateway of the network that traefik container lived on. So for me this was 192.168.5.254.
That address is actually the same as my docker host 192.168.0.10,
So putting in 192.168.5.254 in my yaml file worked.
Below is my config for homeassistant should anybody be interested.

http:
  routers:
    hass:
      entryPoints:
        - websecure
      service: service-hass
      rule: Host("homeassistant.example.com")

  services:
    service-hass:
      loadBalancer:
        servers:
          - url: http://192.168.5.254:8123
        passHostHeader: true
12

No problem glad you got it working!