Gettng information, not 404; tailscale client can't connect

Traefik Traefik v3 (latest)
1

Couple questions. Have a new install with traefik serving headscale. I read that if you hit your url with a browser that you should get a 404 message. When i hit mine, I get information like below. Is this expected? I am hitting it internally right now, so the 192.168 IPs are expected. When I hit it externally I get external addresses, but same behavior.

Hostname: 4b5647dd4b1d
IP: 127.0.0.1
IP: 172.19.0.3
RemoteAddr: 172.19.0.2:46610
GET / HTTP/1.1
Host: headscale.mydomain.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Te: trailers
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 192.168.100.7
X-Forwarded-Host: headscale.mydomain.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 8c8de1db49ce
X-Real-Ip: 192.168.100.7

I've tried an Android tailscale client and docker tailscale and neither can connect. On Andriod I change the server and click "Sign in with other" and it just spins. I see nothing in the traefik log.

Any suggestions for troubleshooting this?

Thanks

2

Ok, I think my problem is the ports. Taking a look at that.

Still looking for an answer for the first question.

Thanks

3

What you see there is the response of a whoami service :smile:

So somehow Traefik seems not to be configured correctly. Note that router/service names in labels of target services need to be different per service/container.

  whoami:
    image: traefik/whoami:v1.10
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.mywhoami.rule=Host(`whoami.example.com`)
                             ^
      - traefik.http.services.mywhoami.loadbalancer.server.port=80
                              ^
4

Still struggling with this. Maybe you can tell me what I'm missing?

headscale docker-compose.yaml:

version: '3.9'
services:
  headscale:
    container_name: headscale
    volumes:
      - /home/dockerconfigs/docker/headscale/config:/etc/headscale/
      - /home/dockerconfigs/docker/headscale/keys:/var/lib/headscale/
    ports:
      - 8081:8080
      - 9091:9090
    image: headscale/headscale:0.22.2
    command: headscale serve
    restart: unless-stopped

  headscale-ui:
    image: amwpfiqvy/headscale-ui
    pull_policy: always
    container_name: headscale-ui
    restart: unless-stopped
    ports:
      - 9999:80

traefik docker-compose.yaml:

version: "3.3"

services:

  traefik:
    image: "traefik:v3.0"
    container_name: "traefik"
    command:
      #- "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.myresolver.acme.tlschallenge=true"
      #- "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
      - "--certificatesresolvers.myresolver.acme.email=email@mydomain.com"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "443:443"
      - "8080:8080"
    volumes:
      - "./letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`headscale.mydomain.com`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls.certresolver=myresolver"
      - "traefik.http.services.whoami.loadbalancer.server.port=8081"

Also, I've kind of hosed myself for a few days with letsencrypt. Is there a way to stop it from requesting a certificate every time traefik restarts? If it already has one it should be good to go?

Appreciate your time.

Thanks!

5

Your headscale is completely independent of Traefik, you expose the ports and don’t have any Traefik labels or dynamic config for it.

Just use http://domain:9999 to access the GUI.

To avoid re-creation of LE TLS certs, you need to set the file location and make sure the Docker bind mount works, best with absolute paths.

6

Hello. Maybe I'm closer now, not sure to be honest. My url for traefik-dashboard does work, and prompts me for login, and I can login.

Is there any way to get to the traefik console?
How can I view my acme.json file to see if it has keys in it? I assume it does since the dashboard works.

Still can't get to headscale though.

Appreciate your patience!!!!

Thanks

Here's my current traefik docker-compose.yaml:

version: "3.9"
services:
traefik:
image: "traefik:v3.0"
container_name: traefik
hostname: traefik
command:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --providers.docker
- --providers.docker.exposedByDefault=false
- --api
- --certificatesresolvers.myresolver.acme.email=myemail@mydomain.com
- --certificatesresolvers.myresolver.acme.storage=./acme.json
- --certificatesresolvers.myresolver.acme.tlschallenge=true
ports:
- "80:80"
- "443:443"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./acme.json:/acme.json"
labels:
- "traefik.enable=true"
# Dashboard
- "traefik.http.routers.traefik.rule=Host(traefik-dashboard.mydomain.com)"
- "traefik.http.routers.traefik.service=api@internal"
- "traefik.http.routers.traefik.tls=true"
- "traefik.http.routers.traefik.tls.certresolver=myresolver"
- "traefik.http.routers.traefik.entrypoints=websecure"
- "traefik.http.routers.traefik.middlewares=authtraefik"
- "traefik.http.middlewares.authtraefik.basicauth.users=username:hashpassword"
# global redirect to https
- "traefik.http.routers.http-catchall.rule=hostregexp({host:.+})"
- "traefik.http.routers.http-catchall.entrypoints=web"
- "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
# middleware redirect
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
restart: unless-stopped
networks:
- proxy

whoami:
# A container that exposes an API to show its IP address
image: traefik/whoami
labels:
- "traefik.http.routers.whoami.rule=Host(whoami.docker.localhost)"
ports:
- 8088:80

networks:
proxy:
name: proxy
external: true

Here's my headscale docker-compase.yaml:
version: '3.9'
services:
headscale:
container_name: headscale
volumes:
- ./config:/etc/headscale/
- ./keys:/var/lib/headscale/
ports:
- 8081:8080
- 9091:9090
labels:
- "traefik.enable=true"
- "traefik.http.routers.headscale.rule=Host(headscale.mydomain.com)"
- "traefik.http.routers.headscale.entrypoints=websecure"
- "traefik.http.routers.headscale.tls=true"
- "traefik.http.routers.headscale.tls.certresolver=myresolver"

image: headscale/headscale:0.22.2
command: headscale serve
restart: unless-stopped

headscale-ui:
image: amwpfiqvy/headscale-ui
pull_policy: always
container_name: headscale-ui
restart: unless-stopped
ports:
- 9999:80

As a side note, when I start headscale I get this error, but DERP is clearly set to disabled in my headscale/config/config.yaml file.
2024-03-18T16:19:54Z ERR Could not load DERP map from path error="Get "https://controlplane.tailscale.com/derpmap/default\": tls: failed to verify certificate: x509: certificate signed by unknown authority" func=GetDERPMap url=https://controlplane.tailscale.com/derpmap/default

2024-03-18T16:19:54Z WRN DERP map is empty, not a single DERP map datasource was loaded correctly or contained a region

In headscale/config/config.yaml:

derp:
server:
# If enabled, runs the embedded DERP server and merges it into the rest of the DERP config
# The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place
enabled: false

7

Added this line to headscale labels section:
- "traefik.http.services.headscale.loadbalancer.server.port=8080"

No change though.

Thanks

8

I think I'm at a point where my headscale and traefik containers can't talk to each other. Also unsure if I need the proxy network or not. Right now I have proxy commented out.

9

partial success by running the following:

sudo docker network connect headscale_default traefik

Do I create one network and have it assigned to all the containers?

10

so close. I created a network and assigned to all the containers that need it. everything looks like it's working correctly, but still can't get to internal resources on my phone using the tailscale client as an exit node. I have the routes enabled in headscale

Thanks