Hey there, I have this peculiar setup and I can't seem to figure out a way to make this work.
So, the basics:
-
My ISP uses CGNAT (double nat), so I don't have a public ipv4. However, they allow me to use some high range ports in a different ipv4 which connects to my router. So, let's say, they redirect to my router traffic from this ISPaddress:15000 (they call it logical ports). It's like a reverse proxy. It works and I've hosted navidrome and a minecraft on it without issues, though I had to open the ports for these services.
-
Since the ISPaddress' range of ports reserved for me is very limited and high in numbers, I can't use domains DNS redirects.
-
I decided, then, to run a VPS with Caddy, which would receive the 80 and 443 traffic and reverse_proxy it to my ISPadress:15000, which would get to my router and be directed to my services in my HomeServer. I wanted to use traefik to receive this traffic on the edge of my network and, using subdomains, connect to my services at the backend. In this example I'll use navidrome, which is a pretty straightforward and simple music streaming service. I'll also test with whoami that comes with traefik's docker-compose example.
-
I create a DuckDNS A record that goes to my VPS. Let's say: example.duckdns.org.
-
On my VPS, I run Caddy with a extremely simple setup for each reverse proxy subdomain:
music.example.duckdns.org {
reverse_proxy --to ISPaddress:15000
}
whoami.example.duckdns.org {
reverse_proxy --to ISPaddress:15000
}
-
On my router, ISPaddress:15000 traffic goes to my HomeServer:800.
-
I install traefik and some services in my HomeServer with docker and docker-compose. They look like this:
traefik:
image: "traefik:v2.3"
container_name: "traefik"
command:
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.web-secured.address=:443"
ports:
- 800:80
- 8001:8080
- 4430:443
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
whoami:
image: "traefik/whoami"
container_name: "simple-service"
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`whoami.example.duckdns.org`)"
- "traefik.http.routers.whoami.entrypoints=web"
navidrome:
image: deluan/navidrome:latest
user: 1001:1001
ports:
- 3001:4533
restart: unless-stopped
environment:
volumes:
labels:
- "traefik.enable=true"
- "traefik.http.routers.navidrome.rule=Host(`music.example.duckdns.org`)"
- "traefik.http.routers.navidrome.entrypoints=web"
-
Everything loads up nicely in the backend. Accessing my services locally works fine, traefik is able to find them all.
-
Now it's time to see if acessing them from outside works.
Results: It partially works, but mostly not. I get 502 all the time. Services like whoami, after 2 or 3 502's, load up. If I refresh, 502s happen until it is able to finally load. Services like Navidrome, which have to load a bunch of stuff, will load just parts of it after dealing with 502s. After the main interface is able to load, I can see in the browser console that many of my requests to the server got 502s.
I wonder if traefik can actually work in these conditions or if there's something I'm configuring wrong.
I mean, it's like there's a reverse proxy from Caddy to my ISP and another one from my ISP to my router.
Maybe my ISP is interfering with the traffic? It's possible, but I ran a Minecraft server with a bunch of people directly in these ports without traffic loss.
Also, If I bypass traefik and caddy on the VPS and just use the ISPadress:15000 and point it to my server, allowing it through the firewall, the services work fine from the outer web.
I hope to see some ideas for this issue!
Thanks.