Traefik is not using my Cloudflare origin cert but rather Letsencrypt?

[Auxtal]

Heya, I have recently purchased my VPS and it's currently running portainer and traefik. This is also working through cloudflare. So as shown in the title traefik is currently displaying letsencrypt certificates instead of my cloudflare origin certificate. This is my first time trying this so please forgive me if I'm making some silly mistake.

The expected outcome here for me would to have the cloudflare origin certificate show in the certificate viewer as opposed to the letsencrypt one. Not to sure what it is I'm doing wrong here any help is greatly appreciated!

This is my docker-compose.yml for portainer and traefik:

version: '3.8'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: always
    command:
      - --api
      - --api.insecure=true
      - --log.level=INFO
      - --providers.docker
      - --providers.file.filename=/dynamic.yaml
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --certificatesresolvers.leresolver.acme.email=ethan@auxtal.xyz
      - --certificatesresolvers.leresolver.acme.storage=/letsencrypt/acme.json
      - --certificatesresolvers.leresolver.acme.httpchallenge=true
      - --certificatesresolvers.leresolver.acme.httpchallenge.entrypoint=web
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik:/traefik:ro
      - ./traefik/letsencrypt:/letsencrypt:ro
      - ./traefik/cloudflare:/cloudflare:ro
      - ./traefik/dynamic.yaml:/dynamic.yaml:ro
    labels:
      - traefik.enable=true
      - traefik.http.routers.http-catchall.rule=hostregexp(`{host:[a-z-.]+}`)
      - traefik.http.routers.http-catchall.entrypoints=web
      - traefik.http.routers.http-catchall.middlewares=redirect-to-https
      - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
      - traefik.http.routers.traefik.rule=Host(`traefik.auxtal.xyz`)
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.routers.traefik.entrypoints=websecure
      - traefik.http.routers.traefik.tls=true

  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    command: -H unix:///var/run/docker.sock
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    labels:
      - traefik.enable=true
      - traefik.http.routers.portainer.rule=Host(`portainer.auxtal.xyz`)
      - traefik.http.routers.portainer.service=portainer
      - traefik.http.routers.portainer.entrypoints=websecure
      - traefik.http.services.portainer.loadbalancer.server.port=9000
      - traefik.http.routers.portainer.tls=true

volumes:
  portainer_data:

This is my dynamic config file:

# Dynamic configuration

tls:
  stores:
    default:
      defaultCertificate:
        certFile: /cloudflare/auxtal.xyz.pem
        keyFile: /cloudflare/auxtal.xyz.key

Here are some screenshots to better show what I'm talking about:

[bluepuma77]

Interesting, are you sure the Cloudflare certificate should not be from LetsEncrypt? Maybe they use the free service, too.

You define a LetsEncrypt certificatesresolvers but it's not assigned to any service, so it should not be used. Have you tried removing the 4 lines?

Furthermore, what happens when you access your service directly, without Cloudflare? To test, just create a second DNS entry with the direct public IP of your VPS and a second router:

      - traefik.http.routers.traefik2.rule=Host(`traefik2.auxtal.xyz`)
      - traefik.http.routers.traefik2.service=api@internal
      - traefik.http.routers.traefik2.entrypoints=websecure
      - traefik.http.routers.traefik2.tls=true

I checked your domain, it seems the DNS includes public and private IPs, you should remove the private IPs.

[Auxtal]

Hey thanks for your swift reply, I have since been bashing my head against a brick wall and have come to the following conclusion. I removed literally everything about letsencrypt from my server. Yet it was still using it. So with that being said I changed my log level to DEBUG to see what was going on. Turns out traefik is not finding my default certificate therefore uses the traefik default generated one.

This raised my eye brows significantly due to the fact that in my traefik dashboard my file provider is displayed meaning it's there

I have since taken on your advice, and I have recreated my certificate and tried .crt instead of .pem
but no cigar on that one.

Also a cloudflare origin certificate should look like this, This is an example:

So I moved my config stuff around I am now using a traefik.yaml file for my static config:

log:
  level: DEBUG

api:
  insecure: true
  dashboard: true

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    filename: certificates.yaml

And this is my dynamic config, certificates.yaml:

tls:
  stores:
    default:
      defaultCertificate:
        certFile: /cloudflare/auxtal.crt
        keyFile: /cloudflare/auxtal.key

  certificates:
    - certFile: /cloudflare/auxtal.crt
      keyFile: /cloudflare/auxtal.key

And my docker-compose.yml:

version: '3.8'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: always
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./cloudflare:/cloudflare:ro
      - ./certificates.yaml:/certificates.yaml:ro
      - ./traefik.yaml:/traefik.yaml:ro
    labels:
      - traefik.enable=true
      - traefik.http.routers.http-catchall.rule=hostregexp(`{host:[a-z-.]+}`)
      - traefik.http.routers.http-catchall.entrypoints=web
      - traefik.http.routers.http-catchall.middlewares=redirect-to-https
      - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
      - traefik.http.routers.traefik.rule=Host(`traefik.auxtal.xyz`)
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.routers.traefik.entrypoints=websecure
      - traefik.http.routers.traefik.tls=true

  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    command: -H unix:///var/run/docker.sock
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    labels:
      - traefik.enable=true
      - traefik.http.routers.portainer.rule=Host(`portainer.auxtal.xyz`)
      - traefik.http.routers.portainer.service=portainer
      - traefik.http.routers.portainer.entrypoints=websecure
      - traefik.http.services.portainer.loadbalancer.server.port=9000
      - traefik.http.routers.portainer.tls=true

volumes:
  portainer_data:

Here are the logs: VmaJn7T9Tj | SourceBin

[bluepuma77]

Config looks really clean now, you can use redirect at entrypoint 80, which looks even cleaner

Is is working now? I see this in the logs:

Adding certificate for domain(s) *.auxtal.xyz,auxtal.xyz,cloudflare origin certificate

[Auxtal]

As mentioned above no it's still not working. Still displaying the same behaviour as before.

[bluepuma77]

Just check if a basic service works, add it to your docker-compose.yml

  whoami:
    image: traefik/whoami:v1.8.1
    labels:
      - 'traefik.enable=true'
      - 'traefik.http.routers.whoami.tls=true'
      - 'traefik.http.routers.whoami.entrypoints=websecure'
      - 'traefik.http.routers.whoami.rule=Host(`whoami.auxtal.xyz`)'
      - 'traefik.http.services.whoami.loadbalancer.server.port=80'

[Auxtal]

Already tried that, sorry I should've said that. I have tried everything I can think of. When I created a whoami service or any other service as a matter of fact it still used the le cert, and cloudflare returns a timeout error.

[youngt2]

I have a very similar setup and it seems to work fine.
Are you using the orange cloud option in the dns settings?
You used a certificate generated from the 'origin server' option on cloudflares SSL/TLS option right? I used and .pem and .key file. I didn't see a .crt file option there.

  certificates:
    - certFile: /traefik/mysite.pem
      keyFile: /traefik/mysite.key

Do you have a univeral certificate for your domain and wildcards in the 'Edge Certificates' section of the same cloudflare menu? This is the certificate that CF would serve when visiting your site.
Have you tried clearing the site cache and putting your domain into 'development' mode on cloudflare?

These are things that gave me trouble at the beginning

[Auxtal]

I have tried every single one of these things prior to creating this thread, the file extension for example .pem, .crt doesn't matter, either way I tried both none worked.

[bluepuma77]

Have you checked the contents of your certificate files?

.pem usually has 3 times:

-----BEGIN CERTIFICATE-----

.key usually has 1:

-----BEGIN PRIVATE KEY-----

[youngt2]

What certificate did it display when you removed all let's encrypt info?

[Auxtal]

They key is fine, but the certificate file only has -----BEGIN CERTIFICATE----- once. The contents was directly copied and pasted from cloudflare sooo.

[Auxtal]

Default traefik lets encrypt certificate, It says it in the logs.

[bluepuma77]

Usually the cert file has 3 entries. Your individual one, some issuer and a root one. Maybe Traefik doesn’t use it because parts are missing. Check for how to create a full Cloudflare certificate.

[youngt2]

I feel like this is a cloudflare issue and not a traefik issue. I think the cloudflare certificate will be served regardless of what traefik is doing. The origin cert is for traefik -> cloudflare traffic (needed for strict ssl mode).

What is your edge certificate on cloudflare showing?

[Auxtal]

I'm aware of what a cloudflare origin certificate is for, the problem here is that's not displaying in the browser, here's what my edge certs on cloudflare look like atm:

[Auxtal]

That makes no sense at all? Why tf would cloudflare give me half a certificate?? I have followed instructions to the letter to create a cloudflare origin certificate, also what I got is the same as what's shown in tutorials online.

[youngt2]

Very strange. I am out of ideas my friend. I hope someone comes along that can help you further

[Auxtal]

No worries, hopefully lol, thank you for your help I appreciate it!

[youngt2]

One other thing I just thought of - what is the CA shown if you click the dropdown arrows on your cloudflare edge certificates?

My universal one is from Google Trust, while my backup is from Let's Encrypt.

Perhaps your primary is a Let's Encrypt?