HTTP -> HTTPS for a specific URL, only?

TrafikJam · March 11, 2020, 9:16am

@ldez The way you separated things is starting to make more sense. I've copied your example, but I'm still getting a 404 unfortunately:

"POST /path/123 HTTP/1.1" 404 165 "-" "-" 644 "services_redirect@docker"

That's the result for posting to "http://www.URL.com/path/123" - while if I post to "https://services.URL.com/456" it does work:

"POST /456 HTTP/1.1" 200 2 "-" "-" 645 "services@docker"

I still don't see how the HTTPS service will be able to send back as HTTP.

ldez · March 11, 2020, 9:34am

Traefik dashboard: https://traefik.localhost
https://services.example.localhost -> OK
http://www.example.localhost/path/123 -> https://services.example.localhost/456 -> ok

Note to redirect you have to use the redirectregex middleware.

https://docs.traefik.io/v2.1/middlewares/redirectregex/

version: '3.7'

services:

  traefik:
    image: traefik:v2.1.6
    command:
      - --log.level=INFO
      - --api
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      traefik.enable: true

      # Dashboard
      traefik.http.routers.traefik.rule: Host(`traefik.localhost`)
      traefik.http.routers.traefik.entrypoints: websecure
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls: true

  whoami:
    image: containous/whoami:v1.4.0
    command:
      # It tells whoami to start listening on 2001 instead of 80
      - --port=2001
    labels:
     traefik.enable: true

      # router for services.example.localhost on the entrypoint websecure
      # https://services.example.localhost
     traefik.http.routers.app.rule: Host(`services.example.localhost`)
     traefik.http.routers.app.entrypoints: websecure
     traefik.http.routers.app.middlewares: comp
     traefik.http.routers.app.tls: true

     traefik.http.services.app_svc.loadbalancer.server.port: 2001

      # router for http://www.example.localhost on the entrypoint web, always redirect.
      # http://www.example.localhost/path/123 -> https://services.example.localhost/456
     traefik.http.routers.app_redirect.rule: Host(`www.example.localhost`) && PathPrefix(`/path/123`)
     traefik.http.routers.app_redirect.entrypoints: web
     traefik.http.routers.app_redirect.middlewares: redirect

      # Middlewares defintions

     traefik.http.middlewares.comp.compress: true
      
     traefik.http.middlewares.redirect.redirectregex.regex: ^http://www\.example\.localhost/path/123
     traefik.http.middlewares.redirect.redirectregex.replacement: https://services.example.localhost/456

This example uses self-signed certificates only to be simple, it also works with Let's Encrypt.

Also I used different syntax for labels (without -, and with : instead of =), but this don't change the behavior.

$ curl -k https://services.example.localhost

Hostname: 865d693c538a
IP: 127.0.0.1
IP: 172.23.0.3
RemoteAddr: 172.23.0.2:36172
GET / HTTP/1.1
Host: services.example.localhost
User-Agent: curl/7.68.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.23.0.1
X-Forwarded-Host: services.example.localhost
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: ee94c8d56a29
X-Real-Ip: 172.23.0.1

$ curl -I http://www.example.localhost/path/123

HTTP/1.1 307 Temporary Redirect
Location: https://services.example.localhost/456
Date: Wed, 11 Mar 2020 09:39:00 GMT
Content-Length: 18
Content-Type: text/plain; charset=utf-8

$ curl -Lk http://www.example.localhost/path/123

Hostname: 865d693c538a
IP: 127.0.0.1
IP: 172.23.0.3
RemoteAddr: 172.23.0.2:36172
GET /456 HTTP/1.1
Host: services.example.localhost
User-Agent: curl/7.68.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.23.0.1
X-Forwarded-Host: services.example.localhost
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: ee94c8d56a29
X-Real-Ip: 172.23.0.1

TrafikJam · March 11, 2020, 9:41am

I asked a few times and I don't think there's a straight answer. I'm using redirectregex.replacement, like you are... But if this is a redirect as in 301, then it will not work for me. What works is what nginx calls a "rewrite." A 307 could also potentially work for me. The API going to the replacement URL does not allow redirection.

This is what is coming through:
https://developer.paypal.com/docs/ipn/
So I cannot test with Curl, but I do test with the IPN simulator. PayPal does not allow anyone to change their old IPN URLs.

ldez · March 11, 2020, 9:47am

You can add traefik.http.middlewares.redirect.redirectregex.permanent: true to get a permanent redirect.

You will have a 308 on GET and a 301 with the other HTTP methods (POST, HEAD, ...)

https://docs.traefik.io/v2.1/middlewares/redirectscheme/#permanent

version: '3.7'

services:

  traefik:
    image: traefik:v2.1.6
    command:
      - --log.level=INFO
      - --api
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      traefik.enable: true

      # Dashboard
      traefik.http.routers.traefik.rule: Host(`traefik.localhost`)
      traefik.http.routers.traefik.entrypoints: websecure
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls: true

  whoami:
    image: containous/whoami:v1.4.0
    command:
      # It tells whoami to start listening on 2001 instead of 80
      - --port=2001
    labels:
     traefik.enable: true

     # router for services.example.localhost on the entrypoint websecure
     # https://services.example.localhost
     traefik.http.routers.app.rule: Host(`services.example.localhost`)
     traefik.http.routers.app.entrypoints: websecure
     traefik.http.routers.app.middlewares: comp
     traefik.http.routers.app.tls: true

     traefik.http.services.app_svc.loadbalancer.server.port: 2001

     # router for http://www.example.localhost on the entrypoint web, always redirect.
     # http://www.example.localhost/path/123 -> https://services.example.localhost/456
     traefik.http.routers.app_redirect.rule: Host(`www.example.localhost`) && PathPrefix(`/path/123`)
     traefik.http.routers.app_redirect.entrypoints: web
     traefik.http.routers.app_redirect.middlewares: redirect

     # Middlewares defintions

     traefik.http.middlewares.comp.compress: true

     traefik.http.middlewares.redirect.redirectregex.regex: ^http://www\.example\.localhost/path/123
     traefik.http.middlewares.redirect.redirectregex.replacement: https://services.example.localhost/456
     traefik.http.middlewares.redirect.redirectregex.permanent: true

$ curl -I http://www.example.localhost/path/123

HTTP/1.1 308 Permanent Redirect
Location: https://services.example.localhost/456
Date: Wed, 11 Mar 2020 09:44:58 GMT
Content-Length: 18
Content-Type: text/plain; charset=utf-8

TrafikJam · March 11, 2020, 9:58am

Yes, so as I said, a 301 will break it. Do you understand my question about terminology? Is a rewrite the same in Traefik-speak as it is in Nginx-speak? It either has to be a 307 or a rewrite as far as I know... This solution seems so much simpler using other methods around the web:

ldez · March 11, 2020, 10:04am

A redirect in Traefik is a kind of rewrite in Nginx.

The 2 products have different approaches so there is no one by one conversion.

You must try to understand how Traefik works instead of trying to do a one by one conversion.

TrafikJam · March 11, 2020, 10:07am

I think Traefik needs to convert their special language into the lingua franca... Perhaps it has and I'm not sufficiently educated. I've obviously put A LOT of effort into this... Can you please see my above link and let me know if it is possible in Traefik? If so, then what would it be called?

ldez · March 11, 2020, 10:39am

I don't know Paypal IPN, from the link you sent, I think you're just looking for a temporary redirect on POST.

The redirect middleware will send a 307 on POST and a 301 on GET.

version: '3.7'

services:

  traefik:
    image: traefik:v2.1.6
    command:
      - --log.level=INFO
      - --api
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      traefik.enable: true

      # Dashboard
      traefik.http.routers.traefik.rule: Host(`traefik.localhost`)
      traefik.http.routers.traefik.entrypoints: websecure
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls: true

  whoami:
    image: containous/whoami:v1.4.0
    command:
      # It tells whoami to start listening on 2001 instead of 80
      - --port=2001
    labels:
      traefik.enable: true

      # router for services.example.localhost on the entrypoint websecure
      # https://services.example.localhost
      traefik.http.routers.app.rule: Host(`services.example.localhost`)
      traefik.http.routers.app.entrypoints: websecure
      traefik.http.routers.app.middlewares: comp
      traefik.http.routers.app.tls: true

      traefik.http.services.app_svc.loadbalancer.server.port: 2001

      # router for http://www.example.localhost on the entrypoint web, always redirect.
      traefik.http.routers.app_redirect.rule: Host(`www.example.localhost`) && PathPrefix(`/path`)
      traefik.http.routers.app_redirect.entrypoints: web
      traefik.http.routers.app_redirect.middlewares: redirect

      # Middlewares defintions

      traefik.http.middlewares.comp.compress: true

      traefik.http.middlewares.redirect.redirectregex.regex: ^http://[^/]+/path/(.+)$$
      traefik.http.middlewares.redirect.redirectregex.replacement: https://services.example.localhost/$${1}

$ curl -v -X POST -d "test"  http://www.example.localhost/path/123

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying ::1:80...
* TCP_NODELAY set
* Connected to www.example.localhost (::1) port 80 (#0)
> POST /path/123 HTTP/1.1
> Host: www.example.localhost
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 4
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 4 out of 4 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 307 Temporary Redirect
< Location: https://services.example.localhost/123
< Date: Wed, 11 Mar 2020 10:37:53 GMT
< Content-Length: 18
< Content-Type: text/plain; charset=utf-8
< 
* Connection #0 to host www.example.localhost left intact

$ curl -L -k -X POST -d "test"  http://www.example.localhost/path/123

Hostname: 38159b0aa900
IP: 127.0.0.1
IP: 172.23.0.3
RemoteAddr: 172.23.0.2:40620
POST /123 HTTP/1.1
Host: services.example.localhost
User-Agent: curl/7.68.0
Content-Length: 4
Accept: */*
Accept-Encoding: gzip
Content-Type: application/x-www-form-urlencoded
X-Forwarded-For: 172.23.0.1
X-Forwarded-Host: services.example.localhost
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: ee94c8d56a29
X-Real-Ip: 172.23.0.1

TrafikJam · March 11, 2020, 12:07pm

@ldez
OK, now I see a 307 or 308 depending upon if I use "permanent" or not. But it still does not work... If I use replacepathregex middleware, then I get a 404.

So, I'm thinking with redirectregex: how does it get the reply? Isn't it replying to the HTTP remote with SSL? Because on the way in it gets translated into an HTTP URL by redirectregex, but then on the way out the server is running on HTTPS - so doesn't it reply using SSL? Does that make sense? The remote is HTTP only and my server is replying with HTTPS because of the above config.

So: how would I make redirectregex.replacement reply without SSL?

ldez · March 11, 2020, 12:15pm

you have to use the redirectregex: https://docs.traefik.io/v2.1/middlewares/redirectregex/

The replacepathregex is only to handle path: https://docs.traefik.io/v2.1/middlewares/replacepathregex/

TrafikJam · March 11, 2020, 12:16pm

OK, I am... And the message seems to get in, but not out. My hypothesis is because I need to send back as HTTP, not HTTPS. Hopefully I described the thought process in my last reply.

ldez · March 11, 2020, 12:22pm

Your goals change after each answer...

Please describe in detail your goals (be clear and explicit) and I will try to answer.

So: how would I make redirectregex.replacement reply without SSL?

A redirect is just a redirect, so if you want to handle no TLS connection you have to handle a router without TLS.

TrafikJam · March 11, 2020, 12:24pm

I described it several times including here:

As I said, "traefik needs to remove SSL somehow" (on the reply).

Would you mind helping me using your above example?
The only criteria I've left out is that I'd like the global SSL redirect (your standard one is what I used) to work again.

ldez · March 11, 2020, 12:28pm

version: '3.7'

services:

  traefik:
    image: traefik:v2.1.6
    command:
      - --log.level=INFO
      - --api
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      traefik.enable: true

      # Dashboard
      traefik.http.routers.traefik.rule: Host(`traefik.localhost`)
      traefik.http.routers.traefik.entrypoints: websecure
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls: true

  whoami:
    image: containous/whoami:v1.4.0
    command:
      # It tells whoami to start listening on 2001 instead of 80
      - --port=2001
    labels:
      traefik.enable: true

      # https://services.example.localhost
      traefik.http.routers.app_secure.rule: Host(`services.example.localhost`)
      traefik.http.routers.app_secure.entrypoints: websecure
      traefik.http.routers.app_secure.middlewares: comp
      traefik.http.routers.app_secure.tls: true

      # http://services.example.localhost
      traefik.http.routers.app.rule: Host(`services.example.localhost`)
      traefik.http.routers.app.entrypoints: web
      traefik.http.routers.app.middlewares: comp
      traefik.http.routers.app.tls: true

      traefik.http.services.app_svc.loadbalancer.server.port: 2001

      # router for http://www.example.localhost on the entrypoint web, always redirect.
      traefik.http.routers.app_redirect.rule: Host(`www.example.localhost`) && PathPrefix(`/path`)
      traefik.http.routers.app_redirect.entrypoints: web
      traefik.http.routers.app_redirect.middlewares: redirect

      # Middlewares defintions

      traefik.http.middlewares.comp.compress: true

      traefik.http.middlewares.redirect.redirectregex.regex: ^http://[^/]+/path/(.+)$$
      traefik.http.middlewares.redirect.redirectregex.replacement: http://services.example.localhost/$${1}

ldez · March 11, 2020, 12:32pm

With the global redirect:

version: '3.7'

services:

  traefik:
    image: traefik:v2.1.6
    command:
      - --log.level=INFO
      - --api
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      traefik.enable: true

      # global HTTP to HTTPS redirection
      traefik.http.routers.http_catchall.rule: hostregexp(`{host:.+}`)
      traefik.http.routers.http_catchall.entrypoints: web
      traefik.http.routers.http_catchall.middlewares: redirect_https
      traefik.http.routers.http_catchall.priority: 1

      # Dashboard
      traefik.http.routers.traefik.rule: Host(`traefik.localhost`)
      traefik.http.routers.traefik.entrypoints: websecure
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls: true

      # Middlewares defintions
      traefik.http.middlewares.comp.compress: true
      traefik.http.middlewares.redirect_https.redirectscheme.scheme: https

  whoami:
    image: containous/whoami:v1.4.0
    command:
      # It tells whoami to start listening on 2001 instead of 80
      - --port=2001
    labels:
      traefik.enable: true

      # https://services.example.localhost
      traefik.http.routers.app_secure.rule: Host(`services.example.localhost`)
      traefik.http.routers.app_secure.entrypoints: websecure
      traefik.http.routers.app_secure.middlewares: comp
      traefik.http.routers.app_secure.tls: true

      # http://services.example.localhost
      traefik.http.routers.app.rule: Host(`services.example.localhost`)
      traefik.http.routers.app.entrypoints: web
      traefik.http.routers.app.middlewares: comp

      traefik.http.services.app_svc.loadbalancer.server.port: 2001

      # router for http://www.example.localhost on the entrypoint web, always redirect.
      traefik.http.routers.app_redirect.rule: Host(`www.example.localhost`) && PathPrefix(`/path`)
      traefik.http.routers.app_redirect.entrypoints: web
      traefik.http.routers.app_redirect.middlewares: redirect

      # Middlewares defintions

      traefik.http.middlewares.redirect.redirectregex.regex: ^http://[^/]+/path/(.+)$$
      traefik.http.middlewares.redirect.redirectregex.replacement: http://services.example.localhost/$${1}

  whoami2:
    image: containous/whoami:v1.4.0
    labels:
      traefik.enable: true

      traefik.http.routers.app2.rule: Host(`app2.example.localhost`)
      traefik.http.routers.app2.entrypoints: websecure
      traefik.http.routers.app2.middlewares: comp
      traefik.http.routers.app2.tls: true

TrafikJam · March 11, 2020, 12:32pm

Mine has this difference (https, not http, for the replacement):

-"traefik.http.middlewares.redirect.redirectregex.regex=^http://www.URL.com/path/123"
-"traefik.http.middlewares.redirect.redirectregex.replacement=https://services.URL.com/456"

I see... So what in your example causes the reply to be over HTTP instead of HTTPS?
OK, I've implemented your solution and it has not helped. I get a 307 from Traefik, but my internal service is not sending anything back.

ldez · March 11, 2020, 12:56pm

A redirect is done by the client (browser, curl, ...), not by Traefik (it's the standard HTTP behavior, see RFC):
When a request comes to an endpoint to redirect, Traefik send a response to the client to inform him to do a request on another location
So the client (browser, curl, ...) calls the new location by it-self.

If have a global redirect (or redirect only for a domain) from HTTP to HTTPS

ex: http://services.example.localhost -> https://services.example.localhost

So it's impossible to call http://services.example.localhost directly: you will be redirected.

To do that you have to create a router with a rule on the entrypoint web that only match your excluded request.

version: '3.7'

services:

  traefik:
    image: traefik:v2.1.6
    command:
      - --log.level=INFO
      - --api
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      traefik.enable: true

      # global HTTP to HTTPS redirection
      traefik.http.routers.http_catchall.rule: hostregexp(`{host:.+}`)
      traefik.http.routers.http_catchall.entrypoints: web
      traefik.http.routers.http_catchall.middlewares: redirect_https
      traefik.http.routers.http_catchall.priority: 1

      # Dashboard
      traefik.http.routers.traefik.rule: Host(`traefik.localhost`)
      traefik.http.routers.traefik.entrypoints: websecure
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls: true

      # Middlewares defintions
      traefik.http.middlewares.comp.compress: true
      traefik.http.middlewares.redirect_https.redirectscheme.scheme: https

  whoami:
    image: containous/whoami:v1.5.0
    command:
      # It tells whoami to start listening on 2001 instead of 80
      - --port=2001
    labels:
      traefik.enable: true

      # https://services.example.localhost
      traefik.http.routers.app_secure.rule: Host(`services.example.localhost`)
      traefik.http.routers.app_secure.entrypoints: websecure
      traefik.http.routers.app_secure.middlewares: comp
      traefik.http.routers.app_secure.tls: true

      # http://services.example.localhost/foo
      traefik.http.routers.app.rule: Host(`services.example.localhost`) && PathPrefix(`/foo`)
      traefik.http.routers.app.entrypoints: web
      traefik.http.routers.app.middlewares: strip,comp

      traefik.http.services.app_svc.loadbalancer.server.port: 2001

      # router for http://www.example.localhost on the entrypoint web, always redirect.
      traefik.http.routers.app_redirect.rule: Host(`www.example.localhost`) && PathPrefix(`/path`)
      traefik.http.routers.app_redirect.entrypoints: web
      traefik.http.routers.app_redirect.middlewares: redirect

      # Middlewares defintions

      traefik.http.middlewares.strip.stripprefix.prefixes: /foo
      traefik.http.middlewares.redirect.redirectregex.regex: ^http://[^/]+/path/(.+)$$
      traefik.http.middlewares.redirect.redirectregex.replacement: http://services.example.localhost/foo/$${1}

  whoami2:
    image: containous/whoami:v1.4.0
    labels:
      traefik.enable: true

      traefik.http.routers.app2.rule: Host(`app2.example.localhost`)
      traefik.http.routers.app2.entrypoints: websecure
      traefik.http.routers.app2.middlewares: comp
      traefik.http.routers.app2.tls: true

TrafikJam · March 11, 2020, 12:59pm

I've implemented your solution and it didn't help. Same as before: I see a POST with a 307 on the regex URL. But the communication can't happen - and I have the same hypothesis as before: how is your latest example causing SSL to be removed on the reply?

ldez · March 11, 2020, 1:07pm

Could you create simple and detailed (with all your cases) graph to explain your goal?

ex: (the graph related to my latest example)

http://services.example.localhost
|_ (redirect)-> https://services.example.localhost
   |_ your app

http://services.example.localhost/foo
|_ your app

http://www.example.localhost/path/123 [POST]
|_ (redirect 307)->  http://services.example.localhost/foo/123 [POST]
  |_ (strip)-> http://services.example.localhost/123 [POST]
     |_ your app

https://www.example.localhost/path/123
|_ 404

TrafikJam · March 11, 2020, 1:11pm

@ldez

I'm unsure of the correct formatting, but after this is "redirected," then it needs to be able to answer. Obviously it will answer using SSL. However, the remote service is expecting HTTP because it was talking to an HTTP service (that was redirected to https://services.example.localhost/123). Does that make sense?

Request comes in to http://www.URL.com/path/123
Traefik (silently) replaces it with https://services.URL.com/456
https://services.URL.com/456 receives the requests
https://services.URL.com/456 answers, but the remote doesn't understand because it is answering with SSL, which expecting none.

Topic		Replies	Views
Traefik global redirect to https except one container Traefik v2 docker , letsencrypt-acme	6	4291	March 3, 2020
Traefik Routers - SSL Redirect + Port 80 for specific Sites Traefik v2 docker , middleware	2	2392	May 22, 2022
Catch-all for all https traffic into one container? Traefik v2 docker	3	1225	June 16, 2023
Quick question about https redirect Traefik v2 docker , file , letsencrypt-acme	2	615	October 28, 2019
I can't set up redirect http to https Traefik v2 docker , middleware	2	190	July 1, 2024

HTTP -> HTTPS for a specific URL, only?

Related topics