Expose an application with two ports: one with https and another for ws?

mdfc · June 25, 2019, 7:47pm

I have the following docker-compose.yml file:

version: '2'

services:
  traefik:
    image: traefik:1.7.12
    restart: always
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    networks:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.toml:/traefik.toml
      - ./acme.json:/acme.json
    container_name: traefik

networks:
  web:
    external: true

The traefik.toml:

debug = false

logLevel = "DEBUG"
defaultEntryPoints = ["https","http"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]

[retry]

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "traefik.mdfc.tech"
watch = true
exposedByDefault = false

[acme]
email = "mdfc@my-email-here.com"
storage = "acme.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"

And I deployed a Java app called Phantombot which exposes two ports: one for the website on port 25000, and another for a websocket on port 25003… So I configured so the websocket port is 443 and left the website port at 25000, then I defined the following in another docker-compose.yml:

version: '2'

services:
  phantombot:
    container_name: phantombot
    build: .
    ports:
      - 25000
      - 443
    restart: always
    volumes:
      - ./phantombot-data:/home/bot/config
    networks:
      - web
    environment:
      - CONF1=demo
    labels:
      - "traefik.enable=true"
      - "traefik.backend=bot"
      - "traefik.docker.network=web"
      - "traefik.ui.frontend.rule=Host:bot.traefik.mdfc.tech"
      - "traefik.ui.port=25000"
      - "traefik.ui.protocol=https"
      - "traefik.ws.frontend.rule=Host:bot.traefik.mdfc.tech"
      - "traefik.ws.port=443"
      - "traefik.ws.protocol=ws"

networks:
  web:
    external: true

The website successfully works if I refresh multiple times, however ~90% of the time one of the assets of the website won’t load and it will return a 500 error because the protocol “ws” isn’t supported.

Is there any way to expose a service with two different ports? As a side note, the web UI is under /panel so I don’t know if there’s a way to proxy all requests to /panel via the 25000 port, and leave everything else on the websocket port since websocket is just an upgrade from the http request.

ldez · June 27, 2019, 9:33pm

Hello,

You have to remove this line when your are using segment labels.

The Websocket protocol is HTTP(S), in your case you have to use https

patrick · July 1, 2019, 7:07am

Awesome, we’ll try that.

As a last question, if I change the protocol to https, how will traefik know which https request goes to what protocol? It seems to me some of the websocket requests may go to the ui.frontend segment while others will go to the websocket one… And yeah, making the ws an https request makes sense – it starts that way anyways then gets upgraded, right?

ldez · July 1, 2019, 7:16am

It's not possible to have 2 segments with the same rule expect if the segments use different entry point (i.e. different ports).
Then you need to change your rules or your entry points.

mdfc · July 1, 2019, 10:56pm

Hello,

Tried that, changed the config to…

    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=web"
      - "traefik.ui.frontend.rule=Host:bot.traefik.mdfc.tech"
      - "traefik.ui.port=25000"
      - "traefik.ui.protocol=https"
      - "traefik.ws.frontend.rule=Host:bot.traefik.mdfc.tech;Headers:Connection,upgrade" 
      - "traefik.ws.port=443" 
      - "traefik.ws.protocol=https"

Unfortunately we’re getting 504 Gateway Timeouts now. Per your docs in Basic - Matchers we see the semicolon as the rule separator for AND and Headers:Connection,upgrade being the matcher, as well as the host that’s responding on, but now the service doesn’t even run. Before at least it will work but some routes may return a 500.

ldez · July 1, 2019, 11:37pm

The connection is upgraded after the routing, then you cannot route on that.

Remember that your routing must be based on something related to the client's call, because WS uses HTTP, it's impossible to tell if a call is for the UI or WS.

Then you have to create a difference:

2 separate entry points (2 ports)
2 sub-domains or 2 domains
a path for one segment
...

mdfc · July 2, 2019, 3:47am

Sorry but I don’t understand… On a websocket connection, the first request IS an HTTP GET which looks like this:

GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13

Note the headers. This connection gets resolved by the server with…

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Protocol: chat

So the client call does indeed have a Connection: Upgrade header, just THEN the route gets changed to a full websocket connection.

Will that not work with Traefik?

ldez · July 2, 2019, 8:12am

Yes I’m sorry, the routing on the Header in your case should be possible.

As you are using Header matcher you have to take care of the case of the value and the header.

I will create a small project to reproduce your use case, I will give you a simple working example.

mdfc · July 2, 2019, 3:36pm

Hello Idez,

Thanks for the answer! Adding to your comment though, we are using the Header matcher but we thought since Traefik is a Go application, it should somehow follow the same rules as the Go standard library, unless you guys wrote your own parser for it… In the Go std library, case sensitivity doesn’t really matter on header names. Or are you referring to the fact it could be (u|U)pgrade?

I’ll let mdfc.tech try using the matcher that supports regexps but judging by the speed of regexps in Go, maybe an OR matcher can somehow be used here, so it supports both Connection: Upgrade and Connection: upgrade.

ldez · July 2, 2019, 4:34pm

I created a small reproducible case with a whoami application.

whoami:

on port 80: return some request information
on port 80 and path /echo: create a ws connection (a simple echo)

The whoami doesn’t use several ports but several paths, but it’s pretty similar to your case.

version: '3'

services:
  reverse_proxy:
    image: traefik:v1.7.12
    command: --api --docker --loglevel=DEBUG
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  whoami:
    image: containous/whoami
    labels:
      - "traefik.ui.frontend.rule=Host:whoami.docker.localhost"
      - "traefik.ws.frontend.rule=Host:whoami.docker.localhost;Headers:Connection,Upgrade;ReplacePathRegex:^(.*) /echo"
      - "traefik.port=80"

$ wscat -c http://whoami.docker.localhost
connected (press CTRL+C to quit)
> test
< test

$ curl http://whoami.docker.localhost
Hostname: 7fd55c87bbe8
IP: 127.0.0.1
IP: 172.19.0.3
GET / HTTP/1.1
Host: whoami.docker.localhost
User-Agent: curl/7.65.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.19.0.1
X-Forwarded-Host: whoami.docker.localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: e376a7f22ca0
X-Real-Ip: 172.19.0.1

mdfc · July 2, 2019, 9:21pm

Thanks, but that solution didn’t work. It seems we’re hitting this issue now since we’re getting Gateway Timeouts.

This is the last set of labels:

    labels: 
      - "traefik.enable=true"
      - "traefik.docker.network=web"
      - "traefik.ui.frontend.rule=Host:bot.traefik.mdfc.tech"
      - "traefik.ui.port=25000"
      - "traefik.ui.protocol=https"
      - "traefik.ws.frontend.rule=Host:bot.traefik.mdfc.tech;Headers:Connection,Upgrade"
      - "traefik.ws.port=443"
      - "traefik.ws.protocol=https"

And when we do this, it tries routing correctly, but we see this:

traefik              | time="2019-07-02T19:44:28Z" level=debug msg="'504 Gateway Timeout' caused by: read tcp 172.18.0.3:32888->172.18.0.6:25000: i/o timeout"

Now, if we do…

$ curl -vvv 172.18.0.6:25000
* Rebuilt URL to: 172.18.0.6:25000/
*   Trying 172.18.0.6...
* TCP_NODELAY set
* Connected to 172.18.0.6 (172.18.0.6) port 25000 (#0)
> GET / HTTP/1.1
> Host: 172.18.0.6:25000
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 02 Jul 2019 21:17:22 GMT
< Content-type: text/html; charset=UTF-8
< Content-length: 6689
<
<!--
  # ommitted for brevity

Note the routing works properly, but for some reason traefik fails to connect to it. Yet if we connect via cURL to the dialed endpoint, it works. Do note, however, we have the traefik.docker.network=web label.

patrick · July 2, 2019, 9:53pm

Hello all!

I think I found the issue because now it properly works. Since the network is external, reapplying docker-compose up -d doesn’t really let traefik find the appropriate ports and internal IP address, so it was pointing to the initial IP address / port from the first time the container was brought to life, rather than the new one when it was respawned. Doing docker-compose down --rmi=all && docker-compose up -d fixed the issue with the configuration @mdfc put above.

Additionally, I enabled the Web UI. It clearly showed what traefik was reading from the previous configuration, and it was showing the same backend with two different addresses (using our initial configuration from the first post) which was roundrobin’d randomly, that’s why we were seeing some of them hit one, and some of them hit the wrong backend port. In the second case it showed that the IP address detected by traefik didn’t match the new one given by docker (visible by doing docker ps).

Thanks for your help, @ldez. The docs weren’t exactly that clear regarding segments so I’m glad we got your help on that.

JuanT · March 31, 2022, 11:07am

Hi Idez,

With traefik:v1.7.12

I tried using the 2 segments with different paths targeting the same port:

wss://cluster06.gbetlds.mgt/autotest01/echo
https://cluster06.gbetlds.mgt/autotest01/index.html


traefik.enable = "true"
traefik.ui.frontend.rule = "Host:cluster06.gbetlds.mgt; PathPrefixStrip: /autotest01"
traefik.ui.port = "8001"
traefik.wss.frontend.rule = "Host:cluster06.gbetlds.mgt; PathPrefixStrip: /autotest01/echo"
traefik.wss.port = "8001"

but keep getting a handshake error on when invoking wss://cluster06.gbetlds.mgt/autotest01/echo

time="2022-03-29T16:19:11Z" level=error msg="vulcand/oxy/forward/websocket: Error dialing "cluster06.gbetlds.mgt": websocket: bad handshake with resp: 404 404 Not Found"



Traefik.toml

################################################################
# Core Entrypoints
################################################################
defaultEntryPoints = ["http", "https"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      certFile = "/certs/traefik.crt"
      keyFile = "/certs/traefik.key"
	  
	  
################################################################
# Traefik dashboard
################################################################
[api]
entrypoint="http"

Any thoughts?

Thanks!

kamelromdhani · June 17, 2022, 8:43pm

why are you using http and not ws?
I tried

wscat -c ws://whoami.docker.localhost

and it didn't work
I got this error:

error: getaddrinfo EAI_AGAIN c c:80

Topic		Replies	Views
Traefik as reverse-proxy in front of 2 docker containers exposing one port each Traefik v2 docker	9	9686	January 5, 2021
Multiple docker-compose Treafik instances Traefik v2 docker	4	9401	August 10, 2021
Simple configuration for container with two open ports Traefik v2 docker	1	940	January 23, 2020
Making WebSockets accessible via Traefik v2 additionally on a non-standard port Traefik v2 docker	8	1345	May 27, 2024
Communication between two services Traefik v2 docker	3	1410	January 5, 2024

Expose an application with two ports: one with https and another for ws?

Related topics