Need help limiting container access to private IPs when using HTTPS

Hello, I'm wondering if there is a way to limit access to a Docker container only to private IP ranges when using HTTPS.

Let me explain better. I have my Raspberry on which I setup Traefik v2 as a reverse proxy for my only Docker container, which is running bitwarden_rs. This container requires HTTPS to work correctly, so I'm using Let's Encrypt to provide certificates. Now I want to be able to access that container only from private IP ranges, or in other words, I don't want the Internet to be able to access my selfhosted password manager (even though it should be safe) and only access it from my LAN (or VPN when I'm not home).

I configured a middleware using "ipWhiteList" to specify the whitelisted private IP source ranges but the result is that I'm now not able to access the container anymore (via bitwarden.example.com), it says "forbidden".
I'm guessing that the issue is that since I'm using HTTPS, I'm actually going through the Internet and back to my server to reach the bitwarden container, hence using my public IP, so Traefik sees this request coming from a public IP (please tell me if I got it right or if that's not what happens when using HTTPS).

Does anyone know if there's a solution to this problem? I couldn't find anything elsewhere. I'm posting my configurations below if they're of any help. Thanks!

docker-compose.yml

version: '3.3'

services:
  traefik:
    container_name: traefik
    image: traefik:latest
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true

    ports:
      - 80:80
      - 443:443
      - 8080:8080

    networks:
      - traefik

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/acme.json:/acme.json
      - ./traefik/traefik.yml:/traefik.yml:ro
      - ./traefik/dynamic.yml:/dynamic.yml:ro

  bitwarden:
    image: bitwardenrs/server:raspberry
    container_name: bitwarden
    restart: unless-stopped

    volumes:
      - ./bitwarden/bw-data:/data

    environment:
      - SIGNUPS_ALLOWED=true
      - WEBSOCKET_ENABLED=true

    networks:
      - traefik

    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik
      - traefik.http.middlewares.redirect-https.redirectScheme.scheme=https
      - traefik.http.middlewares.redirect-https.redirectScheme.permanent=true
      - traefik.http.routers.bitwarden-https.rule=Host(`bitwarden.example.com`)
      - traefik.http.routers.bitwarden-https.entrypoints=https
      - traefik.http.routers.bitwarden-https.tls=true
      - traefik.http.routers.bitwarden-http.rule=Host(`bitwarden.example.com`)
      - traefik.http.routers.bitwarden-http.entrypoints=http
      - traefik.http.routers.bitwarden-http.middlewares=redirect-https
      - traefik.http.routers.bitwarden-ws-https.rule=Host(`bitwarden.example.com`) && Path(`/notifications/hub`)
      - traefik.http.routers.bitwarden-ws-https.entrypoints=https
      - traefik.http.routers.bitwarden-ws-https.tls=true
      - traefik.http.routers.bitwarden-ws-http.rule=Host(`bitwarden.example.com`) && Path(`/notifications/hub`)
      - traefik.http.routers.bitwarden-ws-http.entrypoints=http
      - traefik.http.routers.bitwarden-ws-http.middlewares=redirect-https
      - traefik.http.routers.bitwarden-http.middlewares=whitelist@file
      - traefik.http.routers.bitwarden-https.middlewares=whitelist@file
      - traefik.http.routers.bitwarden-ws-http.middlewares=whitelist@file
      - traefik.http.routers.bitwarden-ws-https.middlewares=whitelist@file

networks:
  traefik:
    external: true

traefik.yml

api:
  dashboard: true
  insecure: true
  debug: true

entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"
 
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"   
    exposedByDefault: false

  file:
    filename: "/dynamic.yml"
    watch: true

certificatesResolvers:
  letsencrypt:
    acme:
      email: mail@example.com
      storage: acme.json
      httpChallenge:
        entryPoint: http

dynamic.yml

http:
  middlewares:
    https-redirect:
      redirectScheme:
        scheme: https
        permanent: true

    whitelist:
      ipWhiteList:
        sourceRange:
          - "10.0.0.0/8"
          - "172.16.0.0/12"
          - "192.168.0.0/16"

Here is what I do: I do not expose my traefik to internet at all, so I do not have to worry about whitelisting - it's safer this way. I use dnsChallenge for obtaining the certs. I had to move my DNS to CloudFlare for that to work, but it was free, quick and easy.

Hey thanks for the reply! I'm sorry to bother you but I need more information before proceeding, if you could help I'd be happy. How do you not expose Traefik to the internet? I mean, configuration-wise, what is it that I must do, should I set my Traefik network to "no-internet" in docker-compose? I'm curious to know how you did it. How does the dns challenge help with what I want to do compared to the http challenge, is it the fact that Let's Encrypt must only contact the dns server and not my Raspberry server at home? Also, how should I configure Cloudflare DNS (I haven't looked into it yet but I'm planning to do that today), do you point bitwarden.example.com to a private IP? Again, sorry for asking so many questions, I'm just trying to understand. It should be simple but I must know how the mechanism works. By the way I'm hosting both public and private services, so some must be accessible from the internet, only bitwarden and a few other containers should be restricted to LAN access only. Thanks!

How do you not expose Traefik to the internet? I mean, configuration-wise, what is it that I must do, should I set my Traefik network to "no-internet" in docker-compose? I'm curious to know how you did it.

It's very simple. Internet comes to the dwelling via a cable that is plugged into a router. This means that any external connections come to the router, and it's up to router is to route or not to route them into the LAN. By default it does not, so really nothing is exposed on the internet, unless it's specifically configured so on the router.

How does the dns challenge help with what I want to do compared to the http challenge

If acme cannot reach your traefik instance, it cannot complete the http challenge. For dns challenge it does not need to reach your traefik instance.

is it the fact that Let's Encrypt must only contact the dns server and not my Raspberry server at home?

Exactly.

Also, how should I configure Cloudflare DNS (I haven't looked into it yet but I'm planning to do that today), do you point bitwarden.example.com to a private IP?

It's up to you. You do not have to configure this domain at all, traefik runs the protocol automatically. The point of the protocol is that you prove that you own the domain name. It's done by getting you (traefik in this case) to create a temporary text record that acme then checks. If it matches, it proves that you indeed own the domain.

The jury is still out if it's safe or not to put private ip on public dns. I have not done that but I personally see nothing wrong with that. Some people argue, that this way you give a potential hacker extra information that you could conceal, and thus they recommend not to this. I see that as pretty minimal risk.

Thanks for the clarification! So I just watched a video that explains how to setup the dns challenge with Cloudflare and it showed that, after Traefik is done getting its domain certificates, I can just configure my local dns (or hosts file) to point bitwarden's subdomain to my server's private IP instead of creating a record on Cloudflare's DNS. At the same time, I guess I can just create dns records for the internet-facing containers and open the ports on the router to access them from outside my LAN. I thought that TLS certificates wouldn't work if I'm accessing the website from a private IP, but apparently they do with dns challenge, or at least it did in the video. It seems simple enough, now I just need to try setting it all up. I'll post again if I run into more issues. Thanks again!

This approach only works properly if you're in control of your local DNS server, right?
Not all devices have an editable host file like PCs have.

What about the OPs original idea to limit source IP ranges? Why is that not a feasible solution?

That is correct, and this is also what I'm personally is doing. Before or after does not matter.

Good luck!

Sounds right. May be a practical example from you would help. We are talking about web servers specifically, and if you cannot change the host file on the device, you may be able to change it on the dns server on your LAN.

I'm not sure. You might be able to get it working. The main problem I'd say, is that if you are using http challenge you might need to white list the IP the challenge comes from. And you cannot control that IP. Assume they migrated service next month and your setup breaks. You also do not have a straightforward way of getting the list of all possible IPs the challenge can come from.

Another issue is if you running docker on Windows, and traefik in Docker, depending on your networking setup traefik might not get the source IP correctly, so could not filter. I've seen cases where traefik would think that source IP is an internal docker network IP, which would prevent the discussed scenario. I only saw this on Windows though, docker for Linux does not appear to have this problem.

So I would not say it's infeasible, I just personally did not have reasons to try and get this working. Your mileage may vary,

The IpWhitelist middleware obviously works, people on the forum reported using it successfully.

Okay I've got a few issues, I'll try to explain them the best I can and provide some info. I'll post my configs again since I've changed them a bit.

docker-compose.yml

version: '3.3'

services:
  traefik:
    container_name: traefik
    image: traefik:latest
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true

    ports:
      - 80:80
      - 443:443
      - 3012:3012

    networks:
      - traefik

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/acme.json:/acme.json
      - ./traefik/traefik.yml:/traefik.yml:ro
      - ./traefik/dynamic.yml:/dynamic.yml:ro

    labels:
      - traefik.enable=true
      - traefik.http.routers.dashboard.rule=Host(`monitor.example.com`)
      - traefik.http.routers.dashboard.entrypoints=https
      - traefik.http.routers.dashboard.service=api@internal

    environment:
      - CLOUDFLARE_EMAIL:mycloudflare@email.com
      - CF_DNS_API_TOKEN:my_cloudflare_global_api_token
      - CF_ZONE_API_TOKEN:my_cloudflare_global_api_token
      - CLOUDFLARE_API_KEY:my_cloudflare_global_api_token

  bitwarden:
    image: bitwardenrs/server:raspberry
    container_name: bitwarden
    restart: unless-stopped

    volumes:
      - ./bitwarden/bw-data:/data

    environment:
      - SIGNUPS_ALLOWED=true
      - WEBSOCKET_ENABLED=true

    networks:
      - traefik

    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik
      - traefik.http.middlewares.redirect-https.redirectScheme.scheme=https
      - traefik.http.middlewares.redirect-https.redirectScheme.permanent=true
      - traefik.http.routers.bitwarden-ui-https.rule=Host(`bw.example.com`)
      - traefik.http.routers.bitwarden-ui-https.entrypoints=https
      - traefik.http.routers.bitwarden-ui-https.tls=true
      - traefik.http.routers.bitwarden-ui-https.service=bitwarden-ui
      - traefik.http.routers.bitwarden-ui-http.rule=Host(`bw.example.com`)
      - traefik.http.routers.bitwarden-ui-http.entrypoints=http
      - traefik.http.routers.bitwarden-ui-http.middlewares=redirect-https
      - traefik.http.routers.bitwarden-ui-http.service=bitwarden-ui
      - traefik.http.services.bitwarden-ui.loadbalancer.server.port=80
      - traefik.http.routers.bitwarden-websocket-https.rule=Host(`bw.example.com`) && Path(`/notifications/hub`)
      - traefik.http.routers.bitwarden-websocket-https.entrypoints=https
      - traefik.http.routers.bitwarden-websocket-https.tls=true
      - traefik.http.routers.bitwarden-websocket-https.service=bitwarden-websocket
      - traefik.http.routers.bitwarden-websocket-http.rule=Host(`bw.example.com`) && Path(`/notifications/hub`)
      - traefik.http.routers.bitwarden-websocket-http.entrypoints=http
      - traefik.http.routers.bitwarden-websocket-http.middlewares=redirect-https
      - traefik.http.routers.bitwarden-websocket-http.service=bitwarden-websocket
      - traefik.http.services.bitwarden-websocket.loadbalancer.server.port=3012

  cloudflare-ddns:
    container_name: cf-ddns
    image: oznu/cloudflare-ddns:latest
    restart: unless-stopped
    networks:
      - traefik
    environment:
     - API_KEY=my_cloudflare_dns_api_token
     - ZONE=example.com
     - PROXIED=true

networks:
  traefik:
    external: true

traefik.yml

log:
  level: DEBUG

api:
  dashboard: true
  insecure: false
  debug: true

entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"
    http:
      tls:
        certResolver: cloudflare
      middlewares:
        - secureHeaders@file
 
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"   
    network: traefik
    exposedByDefault: false

  file:
    filename: /dynamic.yml
    watch: true

certificatesResolvers:
  letsencrypt:
    acme:
      email: mypersonal@email.com
      storage: /acme.json
      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      httpChallenge:
        entryPoint: http

  cloudflare:
    acme:
      storage: /acme.json
      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      dnsChallenge:
        provider: cloudflare
        delayBeforeCheck: 0

dynamic.yml

http:
  middlewares:
    https-redirect:
      redirectScheme:
        scheme: https
        permanent: true

    secureHeaders:
      headers:
        frameDeny: true
        sslRedirect: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 31536000        
        
    bw-stripPrefix:
      stripPrefix:
        prefixes:
          - "/notifications/hub"
        forceSlash: false

tls:
  options:
    default:
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
      minVersion: VersionTLS12

I cleaned my acme.json file that usually contains the generated certificates. My Cloudflare domain is currently set up like this for testing purposes:

When accessing monitor.example.com, I get a proper TLS connection thanks to a Cloudflare certificate, and I'm wondering, shouldn't I be getting a Let's Encrypt certificate instead or is it intended to work like this? I mean, isn't there a way for my browser to receive certificates signed by Let's Encrypt itself, instead of Cloudflare certificates? I thought that maybe it's due to the proxy function Cloudflare offers (the orangle cloud icon), so I disabled that on the "bw" subdomain to test things out. So now bw.example.com is pointing to my home public IP. It turns out that, when trying to access bw.example.com, I get a TRAEFIK DEFAULT CERT instead of a properly generated one. Do you have any idea why? This only happens when the subdomain is set to "DNS only" (gray cloud icon), because clearly monitor.example.com which is proxied by Cloudflare is working, I'm getting a Cloudflare certificate there.

I'm also getting this error in the logs:

level=error msg="Unable to obtain ACME certificate for domains \"monitor.example.com\": cannot get ACME client cloudflare: some credentials information are missing: CLOUDFLARE_EMAIL,CLOUDFLARE_API_KEY or some credentials information are missing: CLOUDFLARE_DNS_API_TOKEN,CLOUDFLARE_ZONE_API_TOKEN" routerName=dashboard@docker rule="Host(`monitor.example.com`)" providerName=cloudflare.acme

Am I passing the wrong API tokens, did I put the environment variables in the wrong place, or is it something else?

Sorry for the lengthy post, I've been trying to set this whole thing up for a week and I'm trying to understand how to solve these issues. Thanks!

Hello, I use http challenge to generate certificates and than limiting access to services using router ClientIP rules Something like this (snippet from docker compose):

  gitlab-web:
    image: gitlab/gitlab-ce:14.5.0-ce.0
    restart: always
    container_name: gitlab-server
    hostname: 'gitlab.example.sk'
    volumes:
      - '/serv/gitlab/config:/etc/gitlab'
      - '/serv/gitlab/logs:/var/log/gitlab'
      - '/serv/gitlab/data:/var/opt/gitlab'
    environment:
      VIRTUAL_HOST: gitlab.example.sk
    labels:
      - traefik.enable=true
      - traefik.http.routers.gitlabwebhttp.rule=(ClientIP(`172.26.0.0/24`, `::1`) || ClientIP(`10.1.0.0/24`, `::1`) || ClientIP(`192.168.1.0/24`, `::1`)) && Host(`gitlab.example.sk`, `registry.example.sk`)
      - traefik.http.routers.gitlabwebhttp.entrypoints=web
      - traefik.http.routers.gitlabwebsecure.rule=(ClientIP(`172.26.0.0/24`, `::1`) || ClientIP(`10.1.0.0/24`, `::1`) || ClientIP(`192.168.1.0/24`, `::1`)) && Host(`gitlab.example.sk`, `registry.example.sk`)
      - traefik.http.routers.gitlabwebsecure.entrypoints=websecure
      - traefik.http.routers.gitlabwebsecure.tls=true
      - traefik.http.services.gitlab-service.loadbalancer.server.port=80

Note that there is perhaps config issue when traefik does tls termination for gitlab (it should use tcp passthrough). Use this snippet as an example for rule config only.