Handling internal and external apps with one instance of Traefik

Hello,

We have a docker swarm running on multiple nodes, using Traefik for service discovery and SSL termination. It is currently hosting a bunch of web applications that are just meant to be accessible from our internal network - not exposed to the public internet.

Everything works fine.

But now we want to also host some external (public internet) facing apps on the same swarm.

We would prefer to do this using multiple entrypoints rather than using middlewares to IP whitelist the internal apps. In other words, each entrypoint would be bound to a specific IP address, and one of those IP addresses would be open to the public internet.

I am running into a problem specifying specific IP addresses in my entrypoint definitions.

For example - right now I am just testing on my local Mac which has 2 (internal) IP addresses on the same interface: 192.168.0.51 and 192.168.0.87.

Defining entrypoints with a specific IP address, as documented here is not working:

      - --entrypoints.web.address=192.168.0.51:80
      - --entrypoints.web-secured.address=192.168.0.51:443

This gives me an error in the traefik logs:

traefik_traefik.1.pfvlfgylsf51@docker-desktop    | 2022/11/03 22:51:15 traefik.go:80: command traefik error: error while building entryPoint web: error preparing server: error opening listener: listen tcp 192.168.0.51:80: bind: cannot assign requested address
...
traefik_traefik.1.og710hgdnwx5@docker-desktop    | 2022/11/03 22:51:07 traefik.go:80: command traefik error: error while building entryPoint web-secured: error preparing server: error opening listener: listen tcp 192.168.0.51:443: bind: cannot assign requested address

Same thing if I use the other IP. The only address that I can plug into the entrypoint definitions is 0.0.0.0 which of course defeats the purpose as it listens on all IPs on the system.

By the way, we are using host mode networking:

    ports:
      - mode: host
        protocol: tcp
        published: 80
        target: 80
      - mode: host
        protocol: tcp
        published: 443
        target: 443

...so traefik should be able to see actual IP addresses out in the real world.

I tried the same thing with a swarm cluster that I spun up in AWS. It won't bind to the private IPv4 address of the swarm manager instance, same error as above.

I should add that in my traefik config I have:

    deploy:
      placement:
        # traefik can only run on swarm manager nodes.
        constraints: [node.role == manager]

and on both my Mac and my test AWS cluster, there is only one manager node, so Traefik is definitely running on the host that has the IP addresses that I am trying to specify.

Nothing else is running on ports 80 and 443. If I remove the IP address from the entrypoint definitions, Traefik starts up fine.

How can I get Traefik to listen on a specific IP? And really, what I ultimately want is to have 4 entrypoints (2 for http and 2 for https) with 2 each listening on a different IP.

Thanks in advance.

EDIT: I wonder if this has something to do with Docker networking?
In that case I should share my networking config.
Currently I am using a docker network called proxy which has scope swarm and driver overlay.
And by "using," I mean my traefik config has the following:

...
    command:
...
      - --providers.docker=true
      - --providers.docker.swarmMode=true
      - --providers.docker.network=proxy
...
    networks:
      - proxy
...
networks:
  proxy:
    external: true

I have tried using various other types of networks but none of them seem to solve this issue.

Have you tried using full network host for the Traefik container?

Docker host port might just skip the ingress network, but not give you direct access to the IPs.

Thanks for the response.

I have already tried using host mode networking (see yaml excerpt in my post), and that didn't do it.

I just tried using the actual host network (which has scope local) and got the following:

traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=web
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="Error while starting server: accept tcp [::]:80: use of closed network connection" entryPointName=web
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="accept tcp [::]:443: use of closed network connection" entryPointName=web-secured
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="Error while starting server: accept tcp [::]:443: use of closed network connection" entryPointName=web-secured
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=debug msg="Entry point web closed" entryPointName=web
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:07Z" level=debug msg="Entry point web-secured closed" entryPointName=web-secured

I tried creating another network with driver host and scope swarm and using that, and I got the same error.

And that's before even specifying an IP address to bind to.

OK, I had a bit of a breakthrough, I got something to work, but I'm not exactly sure how/what I did, or whether it will help solve my overall problem.

I ran a containous/whoami container in my swarm, and then I hit it with curl (from my swarm manager host) and got this output:

Hostname: 5b430102cc44
IP: 127.0.0.1
IP: 10.0.1.52
IP: 172.18.0.3
RemoteAddr: 10.0.1.54:52364
GET / HTTP/1.1
Host: dt-whoami.redacted.org
User-Agent: curl/7.81.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.18.0.1
X-Forwarded-Host: dt-whoami.redacted.org
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 34e1ad25eb86
X-Real-Ip: 172.18.0.1

What's odd is that AWS EC2 tells me that the private IP of my manager instance is
172.30.1.75, but the output above gives both 172.18.0.3 and 172.18.0.1.

If I tell traefik to bind to 172.18.0.3 as follows:

      - --entrypoints.web.address=172.18.0.3:80
      - --entrypoints.web-secured.address=172.18.0.3:443

it works! The other IP (172.18.0.1) does not.

So this is telling me (I guess) that Traefik still can't see the physical IP address of my swarm manager host.

I'm not sure where that "working" IP (172.18.0.3) comes from; I don't see if in the output of ip address show on the host, but I do in the container.
It's not in the output of docker network inspect proxy (the network I'm using) or docker network inspect host).

So, while I have got something to work, I don't think this solves my problem. I need to be able to bind to different physical IP addresses.

Thanks for reading through all this. Please let me know if you have any suggestions.