Traefik Over DIND Containers

Traefik Traefik v2 (latest)
,
1

Hi folks -

Sort of stumped and wondering if this should be possible, spent too many hours on this and ready to give up.

I've a bunch of isolated containers running with docker:dind which contain my whole applications stack (these are basically review apps).

Each container is as follows:-

CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS            PORTS                                                       NAMES

869c4a94edfc        docker:dind        "dockerd-entrypoint.…"   2 hours ago         Up 2 hours          2375/tcp, 0.0.0.0:32964->443/tcp, 0.0.0.0:32963->2376/tcp   review-app-test-18996
ed9602048524        docker:dind        "dockerd-entrypoint.…"   2 hours ago         Up 2 hours          2375/tcp, 0.0.0.0:32968->443/tcp, 0.0.0.0:32968->2376/tcp   review-app-test-19012

Inside each of these dind containers there's a Traefik instance for the review app and everything is working fine e.g. I can access:-

All working fine.

I was now wanting to stop having to use the random ports, so figured I could put another Traefik instance over the top of this to split on a path name (i.e. the review app name, I can't use wildcard DNS subdomains rather than path names for other reasons).

So basically wanted:-

To redirect to each of the apps, doing away with the ports.
I thought I could do this by sticking these labels on the DIND instance:-

docker run \
  --network=traefik-public \
  --name $REVIEW_APP \
  --label "traefik.enable=true" \
  --label "traefik.docker.network=traefik-public" \
  --label "traefik.http.routers.$REVIEW_APP.rule=PathPrefix(\`/$REVIEW_APP\`)" \
  --label "traefik.http.routers.$REVIEW_APP.entrypoints=app" \
  --label "traefik.http.routers.$REVIEW_APP.tls=true" \
  --label "traefik.http.services.$REVIEW_APP.loadbalancer.server.port=443" \
  --label "traefik.http.routers.${REVIEW_APP}.middlewares=${REVIEW_APP}-prefix" \
  --label "traefik.http.middlewares.${REVIEW_APP}-prefix.stripprefix.prefixes=/${REVIEW_APP}" \
  --privileged \
  --detach \
  -p 443 \
  -p 2376 \
  docker:dind

And I'm deploying the top level outside Traefik instance like so:-


version: "3.5"
services:

  traefik:
    image: git.example.local:4567/docker/images/traefik:livarot
    command:
      - --entrypoints.app.address=:443
      - --providers.docker=true
      - --providers.docker.endpoint=unix:///var/run/docker.sock
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=traefik-public
      - --providers.file.filename=/dyn.yml
      - --providers.file.watch=true
      - --api.debug=${TRAEFIK_API_DEBUG:-true}
      - --log.level=${TRAEFIK_LOG_LEVEL:-DEBUG}
      - --accesslog=${TRAEFIK_ACCESS_LOG:-true}
    ports:
      - "443:443"
      - "8443:8443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    configs:
      - source: traefik_conf_v1
        target: /dyn.yml
        mode: 444
    networks:
      - traefik-public

(Note I had to remove:-


      - --providers.docker.swarmMode=true

due to deploying the DIND containers with docker run as traefik in swarm mode requires labels to be part of deploy configuration and not passed labels. And then worked out I can't go the other way and deploy the DIND containers as part of a swarm due to missing privileged option on my version of Docker.)

Anyway, hitting the top level Traefik for the review app:-

https://server.host/review-app-test-18996/

Gives:-

404 page not found

Yet is forwarding the request to the right container:-

traefik_traefik.1.4sh0jq6yjoz4@gye2vlxwsinsd01    | 10.255.0.2 - - [15/Oct/2021:12:52:54 +0000] "GET /review-app-test-18996/ HTTP/1.1" 404 19 "-" "-" 77 "review-app-t

est-18996@docker" "http://10.0.11.39:443" 11ms

Traefik_traefik.1.4sh0jq6yjoz4@server    | time="2021-10-15T12:51:14Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL

="http://10.0.11.39:443" Request="{\"Method\":\"GET\"...

However shelling into the DIND instance and looking at the Traefik within the DIND container log, it appears to be getting passed through fine:-

traefik_traefik.1.l1f3uj0wwanu@869c4a94edfc    | 10.0.0.2 - - [15/Oct/2021:12:53:39 +0000] "GET / HTTP/1.1" - - "-" "-" 132 "-" "-" 0ms

But no match on a route or anything, so I think it's the 404 being served up from inside the review app container.

Is there anything obvious I'm missing in this convoluted setup as to why it's not matching inside the review app container?

Cheers,

Gavin.

2

First I'd recommend using the accesslog.format=json option. Lots of great information by default.

I do note in your 'random ports' scenario you are using https:

With the setup fronted by traefik you still have the https port 443 for the backend service:

However the backend request is http to port 443:

You might just have to add the sheme to the labels:

- "traefik.http.services.<service_name>.loadbalancer.server.scheme=https"
# Also may need a servers transport too
# - "traefik.http.services.<service_name>.loadbalancer.serverstransport=foobar@file"

But you may need to configure a servers transport if you get error with the backend.

3

Hi cakiwi -

Thanks for the quick reply! Good eyes... I'd actually tried that before in my many attempts so I've put it back in.

Instead of the 404 though I'm now getting an "Internal Server Error" and it's moaning about some certificate expiry issues at the DIND Traefik side. So it's another error message to Google actually so thanks, appreciate the response.

Not sure what serversTransport involves but that's something else to read up on.

4

Yes this is likely the serversTransport. While I recommend investigating this and getting it to work with your setup, immediately you can try: --serverstransport.insecureskipverify=true

1 Like
5

Ahhhh, perfect! That's done the trick.

Thanks a million, spent too long banging my head off a wall with this.

Really appreciate the support!

Cheers,
Gavin.

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.