Cannot connect to Websockets server behind traefik V2

I am trying to setup a websocket server behind traefik. I want my ws server t be accessable from example.com:6001 but i have been struggling getting this to work even with online research. I get a "Host unreachable" error from the websocket client. I know DNS and all that is working correctly as this project also has a web app component that I can access, I am just trying to add a websocket server to the mix

I have created an endpoint in my traefik config

...
- --endpoints.ws.address=:6001
...

I have tried various different traefik configs but none of them have worked. I know the server is working because the server is accessible from within the stack.

websockets:
    image: php:8.0-fpm-alpine
    volumes:
      - ./laravel:/var/www/html:delegated
    working_dir: /var/www/html
    ports:
      - 6001:6001
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.imm_websocket.rule=Host(`example.com`)"
      - "traefik.http.routers.imm_websocket.entrypoints=ws"
      - "traefik.http.services.imm_websocket.loadbalancer.server.port=6001"
    entrypoint: [ 'php', '/var/www/html/artisan', 'websockets:serve' ]
    networks:
      - imm
      - web

I used a similar config for my gitlab servers :22 routing

- "traefik.enable=true"
      - "traefik.tcp.routers.ws-imm-server.rule=HostSNI(`example.com`)"
      - "traefik.tcp.routers.ws-imm-server.entrypoints=ws"
      - "traefik.tcp.routers.ws-imm-server.service=ws-imm-server"
      - "traefik.tcp.services.ws-imm-server.loadbalancer.server.port=6001"

Here is my traefik config

traefik:
        image: traefik:v2.5.4
        restart: always
        container_name: traefik
        ports:
            - "80:80"
            - "8080:8080"
        command:
            - --api.dashboard=true # <= Enabling the dashboard
            - --api.debug=true # <= Enable additional endoints for debugging and profiling
            - --log.level=DEBUG # <= Setting level of logging
            - --providers.docker=true # <= Enable docker as a provider
            - --providers.docker.exposedbydefault=false # <= Don't expose every container
            - --providers.docker.network=web # <= Operate on the 'web' network
            - --entrypoints.web.address=:80 # <= Define entrypoint as port :80
            - --entrypoints.ssh.address=:2222
            - --entrypoints.ws.address=:6001
            - --accesslog=true
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock # <= Volume for docker admin
        networks:
            - web
        labels:
            - "traefik.enable=true"
            - "traefik.http.routers.api.rule=Host(`traefik.example.com`)"
            - "traefik.http.routers.api.service=api@internal"

From the dashboard I can see that the endpoint exists, the service is good, and the router is good

What am I missing here?

You'll have to move your published port from websockets over to traefik too.

@shaqaruden Have you managed to get this working? If so, could you please be so kind and describe the relevant parts? I have an almost identical setup as yours and I've been struggling with this for a while now.

3 hints for the original config:

  • use latest Traefik release
  • open port in docker-compose.yml
  • set docker.network in labels when using multiple

Share you Traefik static and dynamic config, and docker-compose.yml if used.

This is the service block in my docker-compose.yml where the websockets server is running:

  workspace:
    build:
      context: ./workspace
    volumes:
      - ${APP_CODE_PATH_HOST}:${APP_CODE_PATH_CONTAINER}${APP_CODE_CONTAINER_FLAG}
      - docker-in-docker:/certs/client
      - ./php-worker/supervisord.d:/etc/supervisord.d
      - ~/.config/composer/auth.json:/home/laradock/.composer/auth.json
    extra_hosts:
      - "dockerhost:${DOCKER_HOST_IP}"
    ports:
      - "${WORKSPACE_SSH_PORT}:22"
    tty: true
    environment:
      - PHP_IDE_CONFIG=${PHP_IDE_CONFIG}
      - DOCKER_HOST=tcp://docker-in-docker:2376
      - DOCKER_TLS_VERIFY=1
      - DOCKER_TLS_CERTDIR=/certs
      - DOCKER_CERT_PATH=/certs/client
      - CHOKIDAR_USEPOLLING=true
    networks:
      - frontend
      - backend
    links:
      - docker-in-docker
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.workspace_websocket.rule=Host(`${FRONTEND_SUBDOMAIN}.${BASE_DOMAIN}`, `${BASE_DOMAIN}`)"
      - "traefik.http.routers.workspace_websocket.entrypoints=websocket"
      - "traefik.http.routers.workspace_websocket.tls.certresolver=mydnschallenge"
      - "traefik.http.services.workspace_websocket.loadbalancer.server.port=6001"

This is traefik's service block inside the same docker-compose.yml

  traefik:
    image: "traefik:v2.8"
    command:
      - "--api=true"
      - "--api.insecure=true"
      - "--api.dashboard=true"
      - "--api.debug=true"
      - "--log.level=DEBUG"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.file.directory=/etc/traefik/traefik.d"
      - "--providers.docker.network=backend"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.websocket.address=:6001"
      - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--certificatesresolvers.mydnschallenge.acme.email=*****@**********"
      - "--certificatesresolvers.mydnschallenge.acme.storage=/var/acme.json"
      - "--certificatesresolvers.mydnschallenge.acme.dnschallenge=true"
      - "--certificatesresolvers.mydnschallenge.acme.dnschallenge.provider=digitalocean"
      - "--certificatesresolvers.mydnschallenge.acme.dnschallenge.delayBeforeCheck=180"
      - "--certificatesresolvers.mydnschallenge.acme.dnschallenge.resolvers=1.1.1.1:53,8.8.8.8:53"
    ports:
      - "${TRAEFIK_HOST_HTTP_PORT}:80"
      - "${TRAEFIK_HOST_HTTPS_PORT}:443"
    networks:
      frontend:
        aliases:
          - ${BASE_DOMAIN}
          - ${FRONTEND_SUBDOMAIN}.${BASE_DOMAIN}
          - mailpit-ui.${BASE_DOMAIN}
          - mailpit-smtp.${BASE_DOMAIN}
      backend:
        aliases:
          - ${BASE_DOMAIN}
          - ${FRONTEND_SUBDOMAIN}.${BASE_DOMAIN}
          - mailpit-ui.${BASE_DOMAIN}
          - mailpit-smtp.${BASE_DOMAIN}
    environment:
      DO_AUTH_TOKEN: "*******************************"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - ${DATA_CONF_PATH_HOST}/traefik/certs/acme.json:/var/acme.json
      - ${DATA_CONF_PATH_HOST}/traefik/dynamic:/etc/traefik/traefik.d
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.http-catchall.rule=PathPrefix(`/`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${BASE_DOMAIN}`)"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.tls.certresolver=mydnschallenge"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.services.api@internal.loadbalancer.server.port=8080"

And this is the only dynamic configuration being used with this setup (a redirection middleware definition):

http:
  middlewares:
    redirect-to-non-www:
      redirectRegex:
        regex: "^^https?://www\\.(.+)"
        replacement: "https://${1}"
        permanent: true
    compress:
      compress: {}

And I believe this is the version of Traefik that I'm currently using (retrieved via a docker inspect [image-id]):

 "Labels": {
    "org.opencontainers.image.description": "A modern reverse-proxy",
    "org.opencontainers.image.documentation": "https://docs.traefik.io",
    "org.opencontainers.image.title": "Traefik",
    "org.opencontainers.image.url": "https://traefik.io",
    "org.opencontainers.image.vendor": "Traefik Labs",
    "org.opencontainers.image.version": "v2.8.8"
}

This is Traefik's dashboard page for the corresponding router:

And this is what I see in the browser's Dev Tools' Network tab for a ws client dashboard when it tries to connect to the server:

I would say if you declare a Traefik entrypoint on an additional port:

You need to open the additional port for the Traefik container:

Why not use current/latest Traefik release?

Thanks. Let me look into what you pointed out.

Up 'til now, v2.8 has been working without issues for my use cases. Perhaps now will be the time to make the move to a more recent version.

Opened the port 6001 under Traefik's container:

ports:
  - "${TRAEFIK_HOST_HTTP_PORT}:80"
  - "${TRAEFIK_HOST_HTTPS_PORT}:443"
  - "6001:6001"

But now I'm getting a bind: already in use error when running the containers:

Error response from daemon: driver failed programming external connectivity on endpoint projects094-traefik-1 (31fef635b5a1bd8de4d953eee9bc0e9df81c8e08ea014e66f9e2b4aa16629c3a): Error starting userland proxy: listen tcp4 0.0.0.0:6001: bind: address already in use

Well, then some process on your machine is already listening on that port.

You can try netstat -tulpn on CLI to find the process.

I'm using the laravel-websockets package in order to have a local/open-source ws server to work with. When issuing the command at VS Code's integrated terminal to start this server while remotely connected to the container, the VS Code plugin that allows for this dev scenario/setup was auto-forwarding the port 6001 to my local host. Once I configured this plugin to stop forwarding the port, everything started to work as expected.

Thanks for the help and pointers!