Handling internal and external apps with one instance of Traefik

Traefik Traefik v2
, ,
1

Hello,

We have a docker swarm running on multiple nodes, using Traefik for service discovery and SSL termination. It is currently hosting a bunch of web applications that are just meant to be accessible from our internal network - not exposed to the public internet.

Everything works fine.

But now we want to also host some external (public internet) facing apps on the same swarm.

We would prefer to do this using multiple entrypoints rather than using middlewares to IP whitelist the internal apps. In other words, each entrypoint would be bound to a specific IP address, and one of those IP addresses would be open to the public internet.

I am running into a problem specifying specific IP addresses in my entrypoint definitions.

For example - right now I am just testing on my local Mac which has 2 (internal) IP addresses on the same interface: 192.168.0.51 and 192.168.0.87.

Defining entrypoints with a specific IP address, as documented here is not working:

      - --entrypoints.web.address=192.168.0.51:80
      - --entrypoints.web-secured.address=192.168.0.51:443

This gives me an error in the traefik logs:

traefik_traefik.1.pfvlfgylsf51@docker-desktop    | 2022/11/03 22:51:15 traefik.go:80: command traefik error: error while building entryPoint web: error preparing server: error opening listener: listen tcp 192.168.0.51:80: bind: cannot assign requested address
...
traefik_traefik.1.og710hgdnwx5@docker-desktop    | 2022/11/03 22:51:07 traefik.go:80: command traefik error: error while building entryPoint web-secured: error preparing server: error opening listener: listen tcp 192.168.0.51:443: bind: cannot assign requested address

Same thing if I use the other IP. The only address that I can plug into the entrypoint definitions is 0.0.0.0 which of course defeats the purpose as it listens on all IPs on the system.

By the way, we are using host mode networking:

    ports:
      - mode: host
        protocol: tcp
        published: 80
        target: 80
      - mode: host
        protocol: tcp
        published: 443
        target: 443

...so traefik should be able to see actual IP addresses out in the real world.

I tried the same thing with a swarm cluster that I spun up in AWS. It won't bind to the private IPv4 address of the swarm manager instance, same error as above.

I should add that in my traefik config I have:

    deploy:
      placement:
        # traefik can only run on swarm manager nodes.
        constraints: [node.role == manager]

and on both my Mac and my test AWS cluster, there is only one manager node, so Traefik is definitely running on the host that has the IP addresses that I am trying to specify.

Nothing else is running on ports 80 and 443. If I remove the IP address from the entrypoint definitions, Traefik starts up fine.

How can I get Traefik to listen on a specific IP? And really, what I ultimately want is to have 4 entrypoints (2 for http and 2 for https) with 2 each listening on a different IP.

Thanks in advance.

EDIT: I wonder if this has something to do with Docker networking?
In that case I should share my networking config.
Currently I am using a docker network called proxy which has scope swarm and driver overlay.
And by "using," I mean my traefik config has the following:

...
    command:
...
      - --providers.docker=true
      - --providers.docker.swarmMode=true
      - --providers.docker.network=proxy
...
    networks:
      - proxy
...
networks:
  proxy:
    external: true

I have tried using various other types of networks but none of them seem to solve this issue.

2

Have you tried using full network host for the Traefik container?

Docker host port might just skip the ingress network, but not give you direct access to the IPs.

3

Thanks for the response.

I have already tried using host mode networking (see yaml excerpt in my post), and that didn't do it.

I just tried using the actual host network (which has scope local) and got the following:

traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=web
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="Error while starting server: accept tcp [::]:80: use of closed network connection" entryPointName=web
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="accept tcp [::]:443: use of closed network connection" entryPointName=web-secured
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=error msg="Error while starting server: accept tcp [::]:443: use of closed network connection" entryPointName=web-secured
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:06Z" level=debug msg="Entry point web closed" entryPointName=web
traefik_traefik.1.loby1bs0jerd@ip-172-30-1-75    | time="2022-11-07T17:43:07Z" level=debug msg="Entry point web-secured closed" entryPointName=web-secured

I tried creating another network with driver host and scope swarm and using that, and I got the same error.

And that's before even specifying an IP address to bind to.

4

OK, I had a bit of a breakthrough, I got something to work, but I'm not exactly sure how/what I did, or whether it will help solve my overall problem.

I ran a containous/whoami container in my swarm, and then I hit it with curl (from my swarm manager host) and got this output:

Hostname: 5b430102cc44
IP: 127.0.0.1
IP: 10.0.1.52
IP: 172.18.0.3
RemoteAddr: 10.0.1.54:52364
GET / HTTP/1.1
Host: dt-whoami.redacted.org
User-Agent: curl/7.81.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.18.0.1
X-Forwarded-Host: dt-whoami.redacted.org
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 34e1ad25eb86
X-Real-Ip: 172.18.0.1

What's odd is that AWS EC2 tells me that the private IP of my manager instance is
172.30.1.75, but the output above gives both 172.18.0.3 and 172.18.0.1.

If I tell traefik to bind to 172.18.0.3 as follows:

      - --entrypoints.web.address=172.18.0.3:80
      - --entrypoints.web-secured.address=172.18.0.3:443

it works! The other IP (172.18.0.1) does not.

So this is telling me (I guess) that Traefik still can't see the physical IP address of my swarm manager host.

I'm not sure where that "working" IP (172.18.0.3) comes from; I don't see if in the output of ip address show on the host, but I do in the container.
It's not in the output of docker network inspect proxy (the network I'm using) or docker network inspect host).

So, while I have got something to work, I don't think this solves my problem. I need to be able to bind to different physical IP addresses.

Thanks for reading through all this. Please let me know if you have any suggestions.

5

I'm trying to do the same thing..this forum is kinda dead, tbh; the only think I can think of is to use two traefik services in the swarm, and have them publish ports to different IPs at the docker level...but I don't know how you'd get a service to be proxied by one or other of the two traefik services.

6

You probably didn’t notice, but you responded to a two year old post.

I recommend to create your own post, clearly state your goals and issues.

Share your Traefik static and dynamic config, and docker-compose.yml if used.

7

Oh, I noticed...I don't want to duplicate what's already been asked, as it should stay in the same thread. Forum is kinda dead even for new posts, from my experience. But good ideas, otherwise.

EDIT: I suppose the thing to do would indeed be to open a new post, since this is a forum, and link this post in it; I guess the community doesn't see old posts with new activity as strongly as new posts

8

For the record - here's how we solved this problem:

We set up another VM that had a firewall hole to the public internet on port 443 and 80 (just for redirects to 443) and ran a nginx proxy on it that proxies all the apps that need public access.

We pointed our internal DNS to the swarm and the external DNS to the proxy..

Safer and simpler.

10

Hello, so did you use the private internal network IPs on the published ports for traefik, so that every app is accessible internally, and obviously not publicly, from traefik? And then point dns to the nginx vm, and set it up to reverse proxy to the traefik instance, with rules only for the desired sites that you want to be public? Thanks

11

Basically, yes. We have a different DNS setup for outside our network than we do for internal traffic. So a user coming to one of our public sites will have the hostname resolved to the nginx vm, and internal users' DNS will resolve to the swarm service mesh.

The proxy only has configuration for the sites that are meant to be public.

12

Great, thanks, that makes sense