Dual proxy setup

Hello. I'm deploying Immich behind Traefik with implementation of Keycloak for authentication. I was
somewhat successfull until now. The issue is Immich has it's internal proxy that i want to keep as it is pre-configured to work with each immich's container. Now, as docs tells, i can add a custom reverse proxy on top of the setup. So i did exactly that.

The docker compose of immich looks like:

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ["start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - stack.env
    depends_on:
      - redis
      - database
      - typesense
    networks:

      - immich
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ["start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - stack.env
    depends_on:
      - redis
      - database
      - typesense
    networks:
      - immich
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /home/docker/docker_immich/model-cache:/cache
    env_file:
      - stack.env
    networks:
      - immich
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - stack.env
    networks:
      - immich
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - /home/docker/docker_immich/tsdata:/data
    networks:
      - immich
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    networks:
      - immich
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - stack.env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    networks:
      - immich
    volumes:
      - /home/docker/docker_immich/pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.immich-local.rule=Host(`immich.local.dashrave.eu`)"
      - "traefik.http.routers.immich-local.entrypoints=web, websecure"
      - "traefik.http.routers.immich-local.tls=true"
      - "traefik.http.routers.immich-local.tls.certresolver=cloudflare"
      - "traefik.http.routers.immich-local.tls.domains[0].main=local.dashrave.eu"
      - "traefik.http.routers.immich-local.tls.domains[0].sans=*.local.dashrave.eu"
      - "traefik.docker.network=proxy"
    networks:
      - proxy
      - immich
    ports:
      - 2283:8080
    depends_on:
      - immich-server
    restart: always

networks:
  immich:
  proxy:
    external: true

You can see the labels i add to immich-proxy container in order to access it through traefik and have secured connection. That's nice but now i need immich-server container to communicate with Keycloak container.

version: "3"
services:

  traefik:
    image: "traefik:latest"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    ports:
      - "80:80"
      - "443:443"
      - "4130:8080"
    networks:
      - proxy
    environment:
      - CF_API_EMAIL=redacted
      - CF_API_KEY=redacted

    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - /home/docker/docker_traefik:/etc/traefik
      - traefik_ssl_certs:/etc/traefik/certs
      - /home/docker/docker_traefik/logs:/var/log/traefik

  crowdsec:
    container_name: crowdsec
    image: crowdsecurity/crowdsec:latest
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    environment:
      GID: "1001"
      COLLECTIONS: "crowdsecurity/linux crowdsecurity/traefik crowdsecurity/http-cve crowdsecurity/whitelist-good-actors crowdsecurity/sshd"
    networks:
      - proxy
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /home/docker/docker_crowdsec/crowdsec:/etc/crowdsec
      - /var/log/auth.log:/var/log/auth.log:ro
      - /var/log/crowdsec:/var/log/crowdsec:ro
      - /home/docker/docker_crowdsec/database:/var/lib/crowdsec/data
    restart: unless-stopped
    security_opt:
      - no-new-privileges=true
    hostname: crowdsec
    
  bouncer-traefik:
    container_name: crowdsec-bouncer-traefik
    image: fbonalair/traefik-crowdsec-bouncer:latest
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    environment:
      CROWDSEC_BOUNCER_API_KEY: redacted # the api key needs to be created of the crowdsec container with `docker compose exec -t crowdsec cscli bouncers add bouncer-traefik`
      CROWDSEC_AGENT_HOST: crowdsec:8080
    networks:
      - proxy
    restart: unless-stopped
    depends_on:
      - crowdsec
    hostname: crowdsec-bouncer-traefik

networks:
  proxy:
    name: proxy

volumes:
  traefik_ssl_certs:

Here you can see the configuration of Traefik itself. I have CrowdSec middleware to block any unwanted connections as i'm planning to open Immich outside my local network so my family can use the app too. You can also see that i'm defining the proxy network which is grouping together all the necessary containers that i want to have secure connection.

Currently i have in proxy network - traefik, crowdsec, keycloak and immich-proxy container all together. Then i configured Keycloak's new realm (understand it as an environment), and created new client for Immich. I configured it as well and copied the necessary fields into Immich webapp. Applied changes and tried it. It throw error:

[Nest] 1  - 06/11/2023, 2:19:46 PM   ERROR [ExceptionsHandler] getaddrinfo ENOTFOUND keycloak.local.dashrave.eu
Error: getaddrinfo ENOTFOUND keycloak.local.dashrave.eu
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26)

It looks like the immich-server container doesn't have access (hostname not resolved) to keycloak container.

So i tried to directly attach immich-container to the proxy network and set the keycloak container's hostname to keycloak. Then i changed the Issuer URL in Immich webapp to keycloak:8080 (hostname:internal_port). With this configuration i was able to make Immich and Keycloak to communicate.

Now to the question finally. If i directly attach immich-server to the proxy network and so immich-container can resolve keycloak:8080, how should i configure Traefik / immich-proxy container to achieve the same behind those two proxies? It apparently has something to do with those two proxies (and most with the immich-proxy container) because as i said, if i attach the immich-server container directly to proxy docker network, the hostname can be resolved. Otherwise it can't.