Multiple port bindings for the same host V3

070hm · July 12, 2024, 9:25pm

I have a Centrifugo image that exposes port 8000 for HTTP APIs and port 10000 for gRPC APIs. I have configured Centrifugo with Traefik, and it works fine for HTTP APIs. However, I don't know how to configure it for gRPC.

My configuration for the Traefik service is

version: '3.3'

services:

  traefik:
    # Use the latest v3.0.x Traefik image available
    image: traefik:v3.0
    ports:
      # Listen on port 80, default for HTTP, necessary to redirect to HTTPS
      - 80:80
      # Listen on port 443, default for HTTPS
      - 443:443
      # Listen on port 10000, for grpc
      - 10000:10000
    deploy:
      placement:
        constraints:
          # Make the traefik service run only on the node with this label
          # as the node with it has the volume for the certificates
          - node.labels.traefik-public.traefik-public-certificates == true
      labels:
        # Enable Traefik for this service, to make it available in the public network
        - traefik.enable=true
        # Use the traefik-public network (declared below)
        - traefik.docker.network=traefik-public
        # Use the custom label "traefik.constraint-label=traefik-public"
        # This public Traefik will only use services with this label
        # That way you can add other internal Traefik instances per stack if needed
        - traefik.constraint-label=traefik-public
        # admin-auth middleware with HTTP Basic auth
        # Using the environment variables USERNAME and HASHED_PASSWORD
        - traefik.http.middlewares.admin-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
        # https-redirect middleware to redirect HTTP to HTTPS
        # It can be re-used by other stacks in other Docker Compose files
        - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
        - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
        # traefik-http set up only to use the middleware to redirect to https
        # Uses the environment variable DOMAIN
        - traefik.http.routers.traefik-public-http.rule=Host(`${DOMAIN?Variable not set}`)
        - traefik.http.routers.traefik-public-http.entrypoints=http
        - traefik.http.routers.traefik-public-http.middlewares=https-redirect
        # traefik-https the actual router using HTTPS
        # Uses the environment variable DOMAIN
        - traefik.http.routers.traefik-public-https.rule=Host(`${DOMAIN?Variable not set}`)
        - traefik.http.routers.traefik-public-https.entrypoints=https
        - traefik.http.routers.traefik-public-https.tls=true
        # Use the special Traefik service api@internal with the web UI/Dashboard
        - traefik.http.routers.traefik-public-https.service=api@internal
        # Use the "le" (Let's Encrypt) resolver created below
        - traefik.http.routers.traefik-public-https.tls.certresolver=le
        # Enable HTTP Basic auth, using the middleware created above
        - traefik.http.routers.traefik-public-https.middlewares=admin-auth
        # Define the port inside of the Docker service to use
        - traefik.http.services.traefik-public.loadbalancer.server.port=8080
    volumes:
      # Add Docker as a mounted volume, so that Traefik can read the labels of other services
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # Mount the volume to store the certificates
      - traefik-public-certificates:/certificates
    command:
      # Enable Docker in Traefik, so that it reads labels from Docker services
      - --providers.docker
      # Add a constraint to only use services with the label "traefik.constraint-label=traefik-public"
      - --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
      # Do not expose all Docker services, only the ones explicitly exposed
      - --providers.docker.exposedbydefault=false
      # Enable Docker Swarm mode
      - --providers.swarm.endpoint=unix:///var/run/docker.sock
      # Create an entrypoint "http" listening on port 80
      - --entrypoints.http.address=:80
      # Create an entrypoint "https" listening on port 443
      - --entrypoints.https.address=:443
      # create an entrypoint "grpc" listening on port 10000
      - --entrypoints.grpc.address=:10000
      # Create the certificate resolver "le" for Let's Encrypt, uses the environment variable EMAIL
      - --certificatesresolvers.le.acme.email=${EMAIL?Variable not set}
      # Store the Let's Encrypt certificates in the mounted volume
      - --certificatesresolvers.le.acme.storage=/certificates/acme.json
      # Use the TLS Challenge for Let's Encrypt
      - --certificatesresolvers.le.acme.tlschallenge=true
      # Enable the access log, with HTTP requests
      - --accesslog
      # Enable the Traefik log, for configurations and errors
      - --log
      # Enable the Dashboard and API
      - --api
    networks:
      # Use the public network created to be shared between Traefik and
      # any other service that needs to be publicly available with HTTPS
      - traefik-public

volumes:
  # Create a volume to store the certificates, there is a constraint to make sure
  # Traefik is always deployed to the same Docker node with the same volume containing
  # the HTTPS certificates
  traefik-public-certificates:

networks:
  # Use the previously created public network "traefik-public", shared with other
  # services that need to be publicly available via this Traefik
  traefik-public:
    external: true

My configuration for the centrifugo service is

  centrifugo-service:
    image: centrifugo/centrifugo:v5
    container_name: centrifugo-service
    ports:
      - 8000:8000
      - 20000:10000
    volumes:
      - centrifugo_data:/centrifugo
    command: centrifugo -c config.json  
    ulimits:
      nofile:
        soft: 65535
        hard: 65535
    networks:
      - "traefik-public" 
    deploy:
      replicas: 1
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.routers.centrifugo_service_http.rule=Host(`my.domain.com`)
        - traefik.http.routers.centrifugo_service_http.entrypoints=http
        - traefik.http.routers.centrifugo_service_http.middlewares=https-redirect
        - traefik.http.routers.centrifugo_service_https.rule=Host(`my.domain.com`)
        - traefik.http.routers.centrifugo_service_https.entrypoints=https
        - traefik.http.routers.centrifugo_service_https.tls=true
        - traefik.http.routers.centrifugo_service_https.tls.certresolver=le
        - traefik.http.services.centrifugo_service.loadbalancer.server.port=8000

I tried to change the Traefik configuration for the Centrifugo service, but it is not working for me

      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.routers.centrifugo_service_http.rule=Host(`my.domain.com`)
        - traefik.http.routers.centrifugo_service_http.entrypoints=http
        - traefik.http.routers.centrifugo_service_http.middlewares=https-redirect
        - traefik.http.routers.centrifugo_service_https.rule=Host(`my.domain.com`)
        - traefik.http.routers.centrifugo_service_https.entrypoints=https
        - traefik.http.routers.centrifugo_service_https.tls=true
        - traefik.http.routers.centrifugo_service_https.tls.certresolver=le
        - traefik.http.services.centrifugo_service.loadbalancer.server.port=8000
           ...
        - traefik.http.routers.centrifugo_service_grpc.rule=Host(`my.domain.com`)
        - traefik.http.routers.centrifugo_service_grpc.entrypoints=grpc
        - traefik.http.services.centrifugo_service_grpc.loadbalancer.server.port=10000

Any thoughts on how to map two exposed ports?
Note : I'm new to traefik

bluepuma77 · July 13, 2024, 6:50am

gRPC seems not that simple, different protocol, usually forced encryption.

Did you check the gRPC guide?

070hm · July 13, 2024, 7:19am

Yes, I checked the gRPC guide and I added For providers with labels, you will have to specify the 'traefik.http.services.<my-service-name>.loadbalancer.server.scheme=h2c'. It's not working for me. I think I missed something but I don't know what

bluepuma77 · July 13, 2024, 7:38am

Try setting the service for each router:

http.routers.routerTest.service=srv-grpc

That might help when using multiple routers/services on one service/container.

Note that a central http-to-https redirect on entrypoint can reduce labels.

070hm · July 13, 2024, 8:15am

Thanks bro for the feedback and reply
I added these lines to the Centrifugo service labels:

        - traefik.http.services.centrifugo_service.loadbalancer.server.port=8000
         ....
        - traefik.http.routers.routerTest.service=centrifugo
        - traefik.http.routers.routerTest.rule=Host(`my.domain.com`)
        - traefik.http.routers.routerTest.entrypoints=grpc
        - traefik.http.services.routerTest.loadbalancer.server.port=10000
        - traefik.http.services.routerTest.loadbalancer.server.scheme=h2c

but it didn't work. I also deleted the prefix of Traefik, but the problem remains. It is still not working, and not even showing the host name in the Traefik dashboard

070hm · July 13, 2024, 8:31am

I try to change my configuration to

  centrifugo:
    image: centrifugo/centrifugo:v5
    container_name: centrifugo
    ports:
      - 8000:8000
      - 10000:10000
    volumes:
      - centrifugo_data:/centrifugo
    command: centrifugo -c config.json  
    ulimits:
      nofile:
        soft: 65535
        hard: 65535
    networks:
      - "traefik-public" 
    deploy:
      replicas: 1
      labels:
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.routers.centrifugo_https.service=centrifugo_https
        - traefik.http.routers.centrifugo_https.rule=Host(`my.domain.com`)
        - traefik.http.routers.centrifugo_https.entrypoints=https
        - traefik.http.routers.centrifugo_https.tls=true
        - traefik.http.routers.centrifugo_https.tls.certresolver=le
        - traefik.http.services.centrifugo_https.loadbalancer.server.port=8000
        - traefik.http.routers.centrifugo_grpc.service=centrifugo_grpc
        - traefik.http.routers.centrifugo_grpc.rule=Host(`my.grpc.domain.com`)
        - traefik.http.routers.centrifugo_grpc.entrypoints=grpc
        - traefik.http.services.centrifugo_grpc.loadbalancer.server.port=10000
        - traefik.http.services.centrifugo_grpc.loadbalancer.server.scheme=h2c

And the port 8000 is working fine, but when I try to call gRPC services at my.grpc.domain.com, it is not working. I think I have some issue with the gRPC mapping

bluepuma77 · July 13, 2024, 9:31am

Enable and check Traefik dashboard, Traefik debug log and Traefik access log in JSON format.

070hm · July 13, 2024, 10:26am

Nothing works. I will skip the process of exposing gRPC. I will just use it internally within the cluster between services. Thanks @bluepuma77 for your help and feedback .

Topic		Replies	Views
Central Traefik to multiple vm's with dockers Traefik v3 (latest) docker	7	1911	December 28, 2024
2 services in 2 ports of the same container. One works, the identical other doesn't Traefik v2 docker	1	595	August 24, 2020
HTTP and TCP lodabalancer Traefik v2 docker , docker-swarm , tcp	2	701	October 26, 2020
Router for http and websocket on different port Traefik v2 kubernetes-crd	0	514	February 25, 2022
Cannot proxy services using multiple domains/sub domains Traefik v2 docker-swarm	4	389	April 17, 2023

Multiple port bindings for the same host V3

Related topics