Hi, I've been trying to build a concrete portfolio project to learn about industry tools and new concepts in programming and I decided I was going to do it in a micro-services architecture. I'm trying to use Traefik as somewhat of an API Gateway/Load balancer so that my services can be spun up dynamically and can communicate with each other without me having to hard code any IP addresses or container names in my program. I'm using docker as a provider and docker compose to orchestrate my entire setup. I defined a host rule for an endpoint to hit a cockroachdb cluster which would ideally be load balanced by traefik. My issue is that when I try to hit the cluster's http port from a container running on the same network through traefik and using the fqdns I defined in the docker labels, I get connection refused issues. When I try to hit the endpoint from my host desktop, I can hit it without a hitch. When I try to hit it with the container's name as defined in my docker compose file while still being connected on the same docker network, I can hit it successfully. I tried putting the two services on different network thinking it would force the container to somehow resolve the dns through traefik, but that didn't work.
Here's a diagram I tried drawing to explain what I'm trying to do:
my docker-compose file (imagine there's three of the roachx service:
roach3:
image: cockroachdb/cockroach:v23.2.4
volumes:
- "roach3:/cockroach/cockroach-data"
command: start --cluster-name=crabby-roach --logtostderr=WARNING --log-file-verbosity=INFO --insecure --http-addr=roach3:8081 --cache=.25 --listen-addr=roach3:36257 --sql-addr=roach3:26257 --accept-sql-without-tls --advertise-addr=roach3:36257 --join=roach1:36257,roach2:36257,roach3:36257
container_name: roach3
labels:
- "traefik.enable=true"
- "traefik.docker.network=roachnet"
# admin interface
- "traefik.http.routers.roach_admin.rule=Host(`admin.roachdb.localhost`)"
- "traefik.http.routers.roach_admin.service=roach_admin_svc"
- "traefik.http.services.roach_admin_svc.loadbalancer.server.port=8081"
- "traefik.http.routers.roach_admin.entrypoints=roach_admin"
# intercommunication channel (optional)
- "traefik.tcp.routers.roach_inter.rule=HostSNI(`*`)"
- "traefik.tcp.routers.roach_inter.service=roach_inter_svc"
- "traefik.tcp.services.roach_inter_svc.loadbalancer.server.port=36257"
- "traefik.tcp.routers.roach_inter.entrypoints=roach_inter"
# Sql command interface
- "traefik.tcp.routers.roach_sql.rule=HostSNI(`*`)"
- "traefik.tcp.routers.roach_sql.service=roach_sql_svc"
- "traefik.tcp.services.roach_sql_svc.loadbalancer.server.port=26257"
- "traefik.tcp.routers.roach_sql.entrypoints=roach_sql"
# depends_on:
# - traefik
networks:
- roachnet
db-init:
image: baxydocker/db-init:latest
container_name: init
hostname: roach-init
command: "/tool"
environment:
- COCKROACH_HOST=roach1
- COCKROACH_PORT=26257
- COCKROACH_USER=root
- COCKROACH_INSECURE=true
# - COCKROACH_INIT=
- DB_NAME=crabby_userdb
- DB_PASSWORD=test
- DB_USER=crabby
- CLUSTER_DNS=admin.roachdb.localhost
- CLUSTER_PORT=8081
depends_on:
- traefik
networks:
- init-db
traefik:
image: traefik:v3.0
container_name: traefik
command:
- "--api=true"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.roach_admin=true"
- "--entrypoints.roach_sql=true"
- "--entrypoints.roach_inter=true"
- "--entrypoints.web=true"
- "--entrypoints.web.address=:80"
- "--entrypoints.roach_admin.address=:8081"
- "--entrypoints.roach_inter.address=:36257"
- "--entrypoints.roachdb_sql.address=:26257"
ports:
- "69:80"
- "8080:8080"
- "8081:8081"
- "36527:36527"
- "26257:26257"
labels:
- "traefik.http.routers.dashboard.rule=Host(`traefik.localhost`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
- "traefik.http.routers.dashboard.service=api@internal"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
depends_on:
- roach1
- roach2
- roach3
networks:
- roachnet
- init-db
volumes:
roach1:
roach2:
roach3:
networks:
roachnet:
external: true
init-db:
external: true
