Is it better to share the same network for Trafic container and exposed containers in Docker?

[Alana]

Option one:

# compose 1
services:
  server:
    image: traefik
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./conf:/etc/traefik
      - /var/run/docker.sock:/var/run/docker.sock:ro

networks:
  default:
    name: reverse-proxy

############################################################

# compose 2
services:
  prometheus:
    image: prom/prometheus
    labels:
      - traefik.enable=true
      - traefik.http.routers.prometheus.tls=true
      - traefik.http.routers.prometheus.rule=Host(`prometheus.$DOMAIN`)
      - traefik.http.services.prometheus.loadbalancer.server.port=9090

networks:
  default:
    name: reverse-proxy
    external: true

Option two:

# compose 1
services:
  server:
    image: traefik
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./conf:/etc/traefik
      - /var/run/docker.sock:/var/run/docker.sock:ro

############################################################

# compose 2
services:
  prometheus:
    image: prom/prometheus
    labels:
      - traefik.enable=true
      - traefik.http.routers.prometheus.tls=true
      - traefik.http.routers.prometheus.rule=Host(`prometheus.$DOMAIN`)
      - traefik.http.services.prometheus.loadbalancer.server.port=9090

Is the shared network option 1 better than option 2? Which option is recommended?

If there is not much difference in performance or others, I prefer option 2, with lower coupling.

[tim.bsm]

If I'm not mistaking, option 2 won't work, as the default (when giving no argument for the network) of docker is to create a new and own bridge network for every container. That on the other hand means that both your containers will be in their own network and don't know about each other or their ports.
This is unless you bind the ports to the host (as you did for traefik ports: ...). Now they can talk to each other but are binded to the host which means that everbody connecting to the host can talk directly to them. But as you want to use traefik as a reverse-proxy, you don't want anybody other than traefik to talk directly to the other containers.

So in conclusion, you need one external network in which all your containers that should be proxied and traefik communicate. Moreover, the only container to bind ports to the host will be traefik here. That means, your first example is correct.

[Alana]

option 2 won't work

I used to think so too, but option 2 is working.

NAME              DRIVER    SCOPE
b_default         bridge    local
reverse-proxy     bridge    local

Treafik in reverse-proxy network and prometheus in b_default network. You can have a try.

Traefik config:

entryPoints:
  web:
    address: :80
  websecure:
    address: :443

api:
  dashboard: true

[bluepuma77]

If no network is provided, Docker compose will create a default network, with all containers inside the compose file attached to it.

I personally prefer explicit networks, so you can manage which containers are connected to which network. For example, you can have one proxy network to connect Traefik with web apps and one database network connecting web apps with a database.

For improved security, you could create an own network for every web app with Traefik, so web apps cannot connect to each other.

[Alana]

@bluepuma77

I do the same thing as you said, I connect different kinds of containers to different networks, such as monitoring network, database network. But some containers don't need to share network with others, so I let them use the network that Docker Compose creates by default. It would be great if Traefik could work without connect to these networks, this is helpful in reducing the coupling between the various modules.

Most of the examples I've seen online have Traefik on the same network as the exposed containers and specifically require this or it won't work.

My question is, is it possible to not be on the same network? (I've tested it and it works, unless I'm doing something wrong) And are there any negative side effects?

[bluepuma77]

Traefik with providers.docker or providers.swarm will use Docker service/container IPs from Docker networks to connect to the target service. So Traefik and target service need to be connected to the same Docker network, implicit or explicit.

[Alana]

Traefik with providers.docker or providers.swarm will use Docker service/container IPs from Docker networks to connect to the target service. So Traefik and target service need to be connected to the same Docker network, implicit or explicit.

Yes, I also found this in the official documentation. But it could work if Traefik and exposed containers are not connected to the same network. I don't know why. Here's my test files (Changed from the official example):

.
├── traefik
│   └── docker-compose.yaml
└── whoami
    └── docker-compose.yaml

traefik/docker-compose.yaml:

services:

  traefik:
    image: "traefik:v3.3"
    container_name: "traefik"
    command:
      #- "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entryPoints.web.address=:80"
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

whoami/docker-compose.yaml:

services:

  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
      - "traefik.http.routers.whoami.entrypoints=web"

/etc/hosts:

127.0.0.1 whoami.localhost

After starting both compose, I can access whoami.localhost and see the page.

docker network ls:

NAME              DRIVER    SCOPE
traefik_default   bridge    local
whoami_default    bridge    local

ENV: Use OrbStack on macOS

[Alana]

@tim.bsm @bluepuma77 Hi everyone, does the Traefik in my test files works on your machines?

[tim.bsm]

I can't really test it atm, I'm sorry. But regardless of whether this works or not, I wouldn't ever expose the container to be directly accessible, as this is what you want to avoid using traefik.
On the one hand, it is just bad practice to not use own networks for traefik. On the other hand, it is also pretty bad in terms of security. Because then somebody could still connect to the server without being routed through traefik.

Really, the only exception I would make here - and have done in my home setup - is if some container inevitably needs network_mode: host to be set on the docker container (e.g. HomeAssistant for me).

[Alana]

Hi @tim.bsm, the container whoami does not have an open port to the host. Therefore, it can only be accessed through the Treafik proxy.

[tim.bsm]

Mhm okay. Well then I think I can't really help here, I'm sorry. I would still make a connected network but it's up to you.

[bluepuma77]

A little testing later:

You can use two separate compose files in the same project folder without an explicit Docker network, and they will be connected to a default created Docker network. Then Traefik will work with whoami.

If you move the second compose file to a different folder, the project will change, the default network will change, and Traefik can not connect to the target service anymore.

[Alana]

Thanks for the testing, but yours is a little different from mine.

In my tests, the two compose files are located in two different folders. These two compose files are launched as two different compose projects, and using different Docker networks. My test details are in this comment.

My test results don't match what's described in the official documentation, that's where I'm confused and I'm trying to figure out what's wrong.

[bluepuma77]

I can just state how regular Docker CE on Linux with docker compose handles the networking. Different compose files in the same directory get the same default network. Compose files in different folders/projects can not connect via their separate default networks - as expected.

I don't know what OrbStack is and how it handles container networking.

[Alana]

Yes, what you describe is also how I remember Docker‘s behavior. Probably something OrbStack did, that's also my guess. Thanks for your patience in replying.