Hey there everybody!
I am currently trying to setup traefik in a portainer environment in my home, with port forwarding on my router, and a AAAA dns entry of my domain linking to the address. All then finished with sugar, namely a ssl-certificate.
So, this seems like a more or less default setup I'd say, but the IPv6 part seems to make it tricky. I think it is best to just dump all the existing config files in here first, after that I can do some explaining:
traefik docker-compose:
---
services:
traefik:
image: traefik:v2.11.0
container_name: traefik
ports:
- 80:80
- 443:443
- 8080:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /etc/docker-configs/traefik/traefik.yaml:/etc/traefik/traefik.yaml:ro
- /etc/docker-configs/traefik/conf/:/etc/traefik/conf/
- /etc/docker-configs/traefik/certs/:/etc/traefik/certs/
networks:
- traefik_network
restart: unless-stopped
networks:
traefik_network:
name: traefik_network
driver: bridge
enable_ipv6: true
ipam:
config:
- subnet: fd00:abcd::/64
This is pretty default I guess. I only defined an extra network for all future services to add which should be routed by traefic in the future. i enabled ipv6 on there and also did set a subnet. I'm actually not sure here whether the address space is valid, but I wasn't able to verify further.
It also works by itself, mostly, I can access the 8080 debug page locally.
I am getting this error in the logs:
msg="routers cannot be a standalone element (type map[string]*dynamic.Router)" container=webserver-paperless-ngx-d1e99826703ea575a0d7004b162bfd998892d956fd5f8645840faa3651dc4fa4 providerName=docker
but I think this comes from the traefik.yaml config shown later down below.
paperless-ngx docker-compose:
### NOT REALLY IMPORTANT FROM HERE ...
version: "3.4"
services:
broker:
image: docker.io/library/redis:7
environment:
PUID: 1000
PGID: 1000
restart: unless-stopped
volumes:
- /etc/docker-configs/paperless/redisdata:/data
db:
image: docker.io/library/postgres:15
stdin_open: true
tty: true
restart: unless-stopped
volumes:
- /etc/docker-configs/paperless/pgdata:/var/lib/postgresql/data
environment:
PUID: 1000
PGID: 1000
POSTGRES_DB: paperless
POSTGRES_USER: paperless
POSTGRES_PASSWORD: paperless
webserver:
image: ghcr.io/paperless-ngx/paperless-ngx:latest
restart: unless-stopped
depends_on:
- db
- broker
ports:
- "8010:8000"
healthcheck:
test: ["CMD", "curl", "-fs", "-S", "--max-time", "2", "http://localhost:8000"]
interval: 30s
timeout: 10s
retries: 5
volumes:
- /mnt/---removed---/paperless/data:/usr/src/paperless/data
- /mnt/---removed---/paperless/media:/usr/src/paperless/media
- /mnt/---removed---/paperless/export:/usr/src/paperless/export
- /mnt/---removed---/paperless/consume:/usr/src/paperless/consume
environment:
PAPERLESS_REDIS: redis://broker:6379
PAPERLESS_DBHOST: db
PUID: 1000
PGID: 1000
USERMAP_UID: 1000
USERMAP_GID: 1000
PAPERLESS_SECRET_KEY: ---removed---
PAPERLESS_TIME_ZONE: Europe/Berlin
PAPERLESS_OCR_LANGUAGE: deu
### ... TO HERE
labels:
- "traefik.enable=true"
- "traefik.http.routers=test"
networks:
- traefik_network
- default
networks:
traefik_network:
name: traefik_network
driver: external
This is also mostly standard and works by itself locally. I only added two labels to enable traefik and adding the test router. Then I added the traefik_network so that traefik and paperless are able to discover each other. In portainer it also seems like everything is fine with that, the network overview shows both services in this network.
traefik.yaml config:
global:
checkNewVersion: false
sendAnonymousUsage: false
log:
level: DEBUG
api:
dashboard: true
insecure: true
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
certificatesResolvers:
staging:
acme:
email: ---removed---
storage: /etc/traefik/certs/acme.json
caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
httpChallenge:
entryPoint: web
providers:
docker:
exposedByDefault: false
file:
directory: /etc/traefik
watch: true
http:
services:
test:
loadBalancer:
servers:
- url: "http://[fd00:abcd::3]:8000"
routers:
test:
rule: "Host(`foo.bar.com`)" # removed
service: test
entryPoints:
- "web"
- "websecure"
tls:
certresolver: "staging"
This is probably where the most is going on. Most of this is probably self-explanatory? I did not fully understand the purpose of services and routers, but most seems right. As I noted above, there is some issue with the routers key, but I don't see where it's coming from?
The test-Service url is the ipv6 which my paperless service got assigned within the traefik_network. I got this from within portainer and also checked via terminal, so I guess it's correct. Port is 8000, since the paperless-webserver is running on that internally.
The test-Router has the rule for the subdomain which I did setup. Talking about that below. The router is linked to the test-Service. (Is it okay to name service and router the same? Seems like it.) Then it has entrypoints and tls assigned, pretty default.
Are there any obvious hiccups here? To these configs, I also edited the /etc/docker/daemon.json (which didn't exist until then) to enable ipv6 within docker (as mentioned by docker themselves):
{
"experimental": true,
"ip6tables": true
}
I also have a domain, to which i added a dns AAAA entry with an 2a00:... address, which points to my network, and more specifically my server on which docker, traefik, etc. is running.
And to sum it up, I added port forwarding on my router for that device, for ipv4 and ipv6, for port 80 and 443. I'm pretty sure that this isn't the error point, since I also already have set up other services/devices before in a similar way, but without traefik yet.
To me this all seems fine. In general, is it correct that I need to route ipv6 also inside my traefik, because the request is coming in as ipv6? Or can I modify it somehow to just handle ipv4 inside and save me from this mess?
When starting/restarting traefik, in the log it tells me the following:
msg="Unable to obtain ACME certificate for domains \"foo.bar.com\": unable to generate a certificate for the domains [foo.bar.com]: error: one or more domains had a problem:\n[foo.bar.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: ---removed ipv6 from foo.bar.com---: Fetching http://foo.bar.com/.well-known/acme-challenge/aJxxxxxxx6P-F-ktxxxxxS_ULMiXxxxxxxxxxxxxTTg: Error getting validation data\n" rule="Host(foo.bar.com)" providerName=staging.acme ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" routerName=test@file
Also when browsing the subdomain myself, it seems to load for a little bit, but I will receive an ERR_ADDRESS_UNREACHABLE. In the first hours it would not be able to retrieve the actual ip because of address propagation, but this is now solved. Also the file does exist within the file system with content.
I tried several things, but overall nothing really seems to move forward (other than having duplicate definitions of services/routers between the .yaml-file and the docker-compose from traefik & paperless at some point, but this seems solved.)
Also the :8080 traefik site shows my paperless service, the router and rule, tls, etc., but no errors.
Do you have any tips for me on how to debug this further?
Thanks for any help 