How to configure a dockerized Traefik with a service on the host?

Traefik Traefik v3 (latest)
1

I am trying to create a basic Continous Integration setup with Traefik and Docker on a VPS in the cloud (Rocky Linux).

I have Traefik 3 (Docker Hub Container Image Library | App Containerization) running in a Docker container and my webservice running in another container - this part works. Now - to make the CI deploy step work my idea is to send a request with a Github webhook to a basic Python "webhook-listener" http server that is running directly on the host. This will then fetch and deploy the new Docker image for my webservice when I activate the "Deploy" webhook on Github.

But I can't figure out how to configure Traefik to route the Python webhook-listener server. The Python server is running as a systemd service on port 5000 and when I log into the Traefik container and try to reach it via the external IP of the host or the bridge / gateway IP this works fine:

# /usr/bin/curl 172.20.0.1:5000
<html><head><title>https://pythonbasics.org</title></head><p>Request: /</p><body><p>This is an example web server.</p></body></html>/

And when I open port 5000 on the VPS (and in firewalld!) I can reach the Python http server also.

But when I try to reach it from the outside via Traefik on the domain I have pointed to the server, I just get a 404 page not found. So my guess is something must be up with my Traefik config files or the networking setup.

There are no hints in the log - just complaints about the self-signed certificate that I use for now, but I guess that is expected and not part of the issue:

 http: TLS handshake error from <my-ip>:<my-port>: remote error: tls: unknown certificate

Here is my static config (/etc/traefik/traefik.yml):

---
global:
  checkNewVersion: false
  sendAnonymousUsage: false

log:
 level: DEBUG

api:
  dashboard: true
  insecure: true

entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: :443

serversTransport:
  insecureSkipVerify: true

providers:
  docker:
    exposedByDefault: false 
    network: proxy
  file:
    directory: /etc/traefik/dynamic
    watch: true

The webhook listener config:

/etc/traefik/dynamic/webhook_listener.yml

http:
  routers:
    python-app-router:
      rule: "Host(`webhook-listener.mysite.com`)"
      service: webhook-listener-service
      entryPoints:
        - websecure
  services:
    webhook-listener-service:
      loadBalancer:
        servers:
          - url: "http://172.20.0.1:5000"  # Python HTTP server

tls:
  certificates:
    - certFile: /var/traefik/certs/webhook-listener.mysite.com.crt
      keyFile: /var/traefik/certs/webhook-listener.mysite.com.key
  options:
    default:
      minVersion: VersionTLS12

My docker-compose for Traefik:

---
services:
  traefik:
    image: docker.io/library/traefik:v3.3.3
    container_name: traefik
    ports:
      - 80:80
      - 443:443
      - 8080:8080 # <- dashboard
    volumes:
      - /run/docker.sock:/run/docker.sock:ro
      - ./config/:/etc/traefik/:ro
      - ./certs/:/var/traefik/certs/:rw
    networks:
      - proxy
    restart: unless-stopped

networks:
  proxy:
    name: proxy

Any ideas on what I am missing? I am new to Traefik so it could be anything. My guess is the network part...

EDIT: The Python http server is listening on all IP-addresses.

PS: I considered putting the Python server inside a Docker container, as this would simplify the routing. But this leads into issues regarding security as I would have to allow the Docker process of the Python Docker container to run commands on the host eg. via a named pipe.

2

The 172.x.x.x IP is probably a gateway IP. If this is available on host and inside container, then you need to make sure your host script is listening to it, or to all IPs (0.0.0.0).

You could also try host.docker.internal from inside container, when using compose:

   extra_hosts:
      - "host.docker.internal:host-gateway"

Docker communication usually works via Docker socket, like Traefik uses it for configuration discovery. You can probably also mount it into a CI/CD container.

3

Thanks for you input @bluepuma77 ! I have added the extra_hosts lines and can now use "host.docker.internal" instead of the gateway IP, which is more elegant. But the core issue still remains - Traefik still won't route the access to the Python server...

4

Check with netstat -tulpn what IPs your host process is listening on.

5

When I run ss on the Rocky Linux VPS:

ss -tulwnp | grep python
tcp   LISTEN 0      5            0.0.0.0:5000      0.0.0.0:*    users:(("python3",pid=43406,fd=3))
6

Ok, it got it. It was a missing tls statement in my dynamic configuration:

http:
  routers:
    python-app-router:
      rule: "Host(`webhook-listener.mysite.com`)"
      service: webhook-listener-service
      entryPoints:
        - websecure
      tls: {} # <-- This part was missing
  services:
    webhook-listener-service:
      loadBalancer:
        servers:
          - url: "http://host.docker.internal:5000"  # Python HTTP server

tls:
  certificates:
    - certFile: /var/traefik/certs/webhook-listener.mysite.com.crt
      keyFile: /var/traefik/certs/webhook-listener.mysite.com.key
  options:
    default:
      minVersion: VersionTLS12

For a newbie like me it would be nice if Traefik debug logs could help out a bit more here. And maybe there are some dev tools I don't know?

Besides that - thanks for a great product, excited to get movin with the rest :smile:

7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.