Problem with headers, probably forwarding

igromanru · September 23, 2019, 2:26pm

I'm trying to replace nginx with Treafik 2 in my docker-compose, but my Frontend can't communicate with the Backend. And I'm not sure why and what I'm missing. It looks like that headers defined in backbend doesn't got forwarded. But I'm not a HTTP expert, I'm not sure how it works in detail.
If my frontend trys to communicate to the Backend, I got this Warning in the Firefox console and 404 Not Found in request.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://my-backend.foobar.com/apiCall. (Reason: CORS header 'Access-Control-Allow-Origin' missing).

Here are my latest configs that I've tried.
traefik_dynamic.toml

[tls]
  [[tls.certificates]]
    certFile = "/etc/ssl/certs/foobar.com.crt"
    keyFile = "/etc/ssl/certs/foobar.com.key"

[http.middlewares]
  [http.middlewares.redirect.redirectScheme]
    scheme = "https"
  [http.middlewares.backend-cors]
    [http.middlewares.backend-cors.headers]
      accessControlAllowCredentials = true
      accessControlAllowMethods= ["GET", "OPTIONS", "PUT", "DELETE"]
      accessControlAllowOrigin = "*"
      accessControlMaxAge = 86400
      addVaryHeader = true

My docker-compose.yml looks like this:

version: '3'
services:

  traefik:
    image: traefik:2.0
    container_name: traefik
    command:
      - "--api=true"
      - "--providers.docker=true"
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /etc/ssl/certs:/etc/ssl/certs
      - ./config/traefik:/etc/traefik
    networks:
      - mynetwork

  keycloak:
    image: jboss/keycloak:latest
    container_name: keycloak
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.keycloak.loadbalancer.server.port=8080"
      - "traefik.http.services.keycloak.loadbalancer.passhostheader=true"
      - "traefik.http.routers.keycloak.rule=Host(`keycloak.foobar.com`)"
      - "traefik.http.routers.keycloak.entrypoints=web"
      - "traefik.http.routers.keycloak.middlewares=redirect@file"
      - "traefik.http.routers.keycloak-secured.rule=Host(`keycloak.foobar.com`)"
      - "traefik.http.routers.keycloak-secured.entrypoints=websecure"
      - "traefik.http.routers.keycloak-secured.tls=true"
      - "traefik.http.routers.keycloak-secured.tls.domains[0].main=foobar.com"
      - "traefik.http.routers.keycloak-secured.tls.domains[0].sans=*.foobar.com"
    expose:
      - 8080
      - 9990
    networks:
      - mynetwork

  my-frontend:
    image: my-frontend:latest
    container_name: my-frontend
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.my-frontend.loadbalancer.server.port=8080"
      - "traefik.http.services.my-frontend.loadbalancer.passhostheader=true"
      - "traefik.http.routers.my-frontend.rule=Host(`my-frontend.foobar.com`)"
      - "traefik.http.routers.my-frontend.entrypoints=web"
      - "traefik.http.routers.my-frontend.middlewares=redirect@file"
      - "traefik.http.routers.my-frontend-secured.rule=Host(`my-frontend.foobar.com`)"
      - "traefik.http.routers.my-frontend-secured.entrypoints=websecure"
      - "traefik.http.routers.my-frontend-secured.tls=true"
      - "traefik.http.routers.my-frontend-secured.tls.domains[0].main=foobar.com"
      - "traefik.http.routers.my-frontend-secured.tls.domains[0].sans=*.foobar.com"
    expose:
      - 8080
    networks:
      - mynetwork

  my-backend:
    image: my-backend:latest
    container_name: my-backend
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.my-backend.loadbalancer.server.port=8080"
      - "traefik.http.services.my-backend.loadbalancer.passhostheader=true"
      - "traefik.http.routers.my-backend.rule=Host(`my-backend.foobar.com`)"
      - "traefik.http.routers.my-backend.entrypoints=web"
      - "traefik.http.routers.my-backend.middlewares=redirect@file"
      - "traefik.http.routers.my-backend-secured.rule=Host(`my-backend.foobar.com`)"
      - "traefik.http.routers.my-backend-secured.entrypoints=websecure"
      - "traefik.http.routers.my-backend-secured.tls=true"
      - "traefik.http.routers.my-backend-secured.tls.domains[0].main=foobar.com"
      - "traefik.http.routers.my-backend-secured.tls.domains[0].sans=*.foobar.com"
      - "traefik.http.routers.my-backend-secured.middlewares=backend-cors@file"
    expose:
      - 8080
    networks:
      - mynetwork

networks:
  mynetwork:
    external: true

I've tried both, to do it with backend-cors middleware and without it, but it has no effect.
With nginx the CORS header are set to "*" by the Backend.

ldez · September 23, 2019, 2:40pm

Hello,

If I understand well: Browser -> frontend (my-frontend) -> backend (my-backend)

So CORS must be define on the frontend.

igromanru · September 23, 2019, 9:39pm

Yes, the frontend is an Angular application and backend is an Spring Boot application. What I know is, that we have explicit enabled CORS with "*" as origin in our Rest Controllers for responses. And it looks like that exactly this part doesn't comes through traefik.

ldez · September 23, 2019, 10:01pm

Have you try to do that:

  my-frontend:
    image: my-frontend:latest
    container_name: my-frontend
    labels:
      # ...
      - "traefik.http.routers.my-frontend-secured.middlewares=backend-cors@file"

igromanru · September 24, 2019, 6:54am

Ah yes, now my OPTIONS requests return 200 but following GET requests are still blocked with this messages:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://my-backend.foobar.com/apiCall. (Reason: missing token 'authorization' in CORS header 'Access-Control-Allow-Headers' from CORS preflight channel).

followed by

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://my-backend.foobar.com/apiCall. (Reason: CORS request did not succeed).

I just wonder, what it the different between nginx and traefik in this part.
The only settings that I made in nginx are:
Frontend:

server {
  listen       443;
  server_name  my-frontend.foorbar.com;

  location / {
    proxy_pass http://my-frontend-container:8080;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
  }

  #ssl
  ssl on;
  ssl_certificate (placeholder);
  ssl_certificate_key (placeholder);
}

Backend:

server {
  listen       443;
  server_name  my-backend.foorbar.com;

  location / {
    proxy_pass http://my-backend-container:8080;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
  }
  
  ssl on;
  ssl_certificate (placeholder);
  ssl_certificate_key (placeholder)
}

And for keycloak (backend connects to keycloak with a bearer token):

server {
  listen       443;
  server_name  keycloak.foorbar.com;

  location / {
    proxy_pass http://keycloak-container:8080;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
  ssl on;
  ssl_certificate (placeholder);
  ssl_certificate_key (placeholder);
}

igromanru · September 30, 2019, 7:08am

Does nobody has a solution for me? Am I really the only one with such a problem?
It's really sad, I was very excited about traefik and wanted to use it in the company. But this issue is the deal breaker.

daniel.tomcej · October 1, 2019, 7:58pm

Hello @igromanru

If your backend is handling the cors responses, then you probably don't want Traefik interfering with them.

If you configure cors headers within Traefik, the preflights will be intercepted, and Traefik will overwrite headers from the backend, which is probably not what you want.

The error you are getting:

Reason: missing token 'authorization' in CORS header 'Access-Control-Allow-Headers' from CORS preflight channel

Is because you have not configured any 'Access-Control-Allow-Headers' in your configuration, and therefore, the authorization header is not specified in the preflight.

igromanru · October 2, 2019, 6:36am

Hello @daniel.tomcej,

thank you for your reply. I would like to keep Traefik out of the way and let him to just pass anything through as is it. But like I tried to describe my problem in the first post, but for some reason if I use traefik with pretty basic settings, I got the problem that you can see here:

daniel.tomcej · October 2, 2019, 1:35pm

Hello @igromanru

Can you please provide a

curl -v my-backend.foobar.com and curl -v my-frontend.foobar.com?

Curious to see what headers are being sent.

igromanru · October 2, 2019, 2:01pm

Thank you for the answer @daniel.tomcej.
Well, with curl I got another problem. At least I think that it's another problem.

curl -v https://my-frontend.foobar.com
* Rebuilt URL to: https://my-frontend.foobar.com/
*   Trying X.X.X.48...
* TCP_NODELAY set
* Connected to my-frontend.foobar.com (X.X.X.48) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, Server hello (2):
* SSL certificate problem: unable to get local issuer certificate
* stopped the pause stream!
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

The same message appears for backend.

zespri · October 2, 2019, 7:39pm

curl -k to bypass the cert check.

daniel.tomcej · October 2, 2019, 7:52pm

@igromanru,

The reason I was trying to see what headers are being sent by your frontend without dealing with the browser overhead.

Forwarded to where?

Is your frontend container making a request to your backend container?
Is it your application in the browser that is making a request to your backend container via Traefik?
You mention that your frontend is setting CORS headers, but in your Traefik configuration above, you are setting them on the backend. Can you give us a little bit more information?

If I can't see what the request without Traefik and the request with Traefik look like, it will be very difficult to debug where the issue stems from.

igromanru · October 7, 2019, 6:53am

@daniel.tomcej thank you for your replay.
It's the other way around. The Backend should set his own headers (response headers).
Actually I can't really work with curl, because it needs a keycloak authentication.
But here I got headers out of firefox network tool.
Here is a working OPTIONS request to Backend out of Browser by the frontend (ajax):
Request headers:

OPTIONS /backend-api HTTP/1.1
Host: my-backend.foobar.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate, br
Access-Control-Request-Method: GET
Access-Control-Request-Headers: authorization
Referer: https://my-frontend.foobar.com/
Origin: https://my-frontend.foobar.com
Connection: keep-alive

Response headers:

HTTP/1.1 200 
Server: nginx/1.15.12
Date: Mon, 07 Oct 2019 06:38:07 GMT
Content-Length: 0
Connection: keep-alive
Access-Control-Allow-Origin: https://my-frontend.foobar.com
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET
Access-Control-Allow-Headers: authorization

Here is what I get with Treafik:
Request headers:

OPTIONS /backend-api HTTP/1.1
Host: my-backend.foobar.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0
Accept: */*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate, br
Access-Control-Request-Method: GET
Access-Control-Request-Headers: authorization
Referer: https://my-frontend.foobar.com/articles
Origin: https://my-frontend.foobar.com
Connection: keep-alive

Response headers:

HTTP/2.0 404 Not Found
content-type: text/plain; charset=utf-8
x-content-type-options: nosniff
content-length: 19
date: Mon, 07 Oct 2019 06:48:37 GMT
X-Firefox-Spdy: h2

And here is again the Error:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://my-backend.foorbar.com/backend-api. (Reason: CORS header 'Access-Control-Allow-Origin' missing).

numToStr · April 26, 2020, 6:50am

I am also facing the same issue with CORS

Here is the github issue
https://github.com/containous/traefik/issues/6676#issuecomment-619494992

zespri · May 1, 2020, 12:36pm

@numToStr In this case the analysis above should also apply to you. 404 coming from the back end generally means that the server application did not recognise resource requested.

Topic		Replies	Views
Traefik 2.5 CORS issue with 3 services (fe and api, mysql) Traefik v2 (latest) docker	1	1152	November 28, 2021
Issues with CORS Traefik v2 (latest) middleware	4	6627	July 15, 2020
Traefik not setting Access-Control-Allow-Origin header Traefik v2 (latest) docker , middleware	2	9029	September 29, 2023
Issue with CORS headers Traefik v2 (latest) docker	0	835	June 8, 2022
Wide open CORS not working? Traefik v2 (latest) docker	1	1286	July 13, 2020

Problem with headers, probably forwarding

Related Topics