Yes, they use ip and not the hostname to call the back end.
I understand, the "official" proposed solution is to use TCP router instead if http one for this scenario.
@ldez prometheus had a similar problem which was solved by adding the correct host name (which can be taken from the host header in the traefik case) to ServerName field of tls.Config, would it be possible for traefik too?
Just for future reference, here is what worked for me for TLS paththrough. I used devd for testing to simulate a backend application. devd accepts the cert and the key in the same file, so there was a double up:
version: "3.7"
services:
reverse-proxy:
image: "traefik:v2.0.4"
container_name: "traefik"
command:
- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.websecure.address=:443"
- "--providers.file.filename=/dyn.toml"
ports:
- "443:443"
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./dyn.toml:/dyn.toml"
- "./cert.key:/cert.key"
- "./cert.pem:/cert.pem"
devd:
image: "tahurt/docker-devd"
container_name: "devd"
command: ["-H", "/static", "-c","/cp.pem"]
volumes:
- "./cp.pem:/cp.pem"
ports:
- "8000:8000"
labels:
- "traefik.enable=true"
- "traefik.tcp.routers.devd.rule=HostSNI(`devd.domain.tld`)"
- "traefik.tcp.routers.devd.entrypoints=websecure"
- "traefik.tcp.routers.devd.tls=true"
- "traefik.tcp.routers.devd.tls.passthrough=true"
- "traefik.tcp.routers.devd.service=devd"
- "traefik.tcp.services.devd.loadbalancer.server.port=8000"
[[tls.certificates]]
certFile = "cert.pem"
keyFile = "cert.key"
You can access insecure dashboard on 8080, to see how the services and routers are created. You can access devd straight to 8000 (https) to make sure that the cert is working. You can access devd on 443 via traefik with tls passthrough.
Hope this helps.
@zespri I understand that I can use TCP router and TLS passthrough, but we need session stickiness which is not available for TCP routers. Am I correct? Is there any way to use session stickiness with TCP routers?
I don't beleive it is. I'm sure a traefik team member will correct me if I'm wrong.
I'm not a golang or traefik code expert, but it appears to me that this particular change would not be straight forward to fix. "Transport" (that is where you set up insecureSkipVerify and RootCA) is configured separately from all the services, and it appears that the same configuration used for all of them. TLS Config belong to this "Transport" configuration. So there is no straightforward way, without some re-architecutre, to solve this. If/when this is solved I would not expect this to happen overnight.
Okay, thank you for all your help. Let's wait for @ldez to answer the question you asked above.
I create my self signed certificates on startup of the container with the IP's as SNI.
This should fix:
The public key of the rootCA is configured in Traefik and used to create the self signed certificate in the container so this should work but I have the same problem as @aleksvujic
Shouldn't this work?
Kind regards,
Marc Gerritsen
@marcgerritsen It's probably a different problem. I suggest creating new thread and posting your full configuration and relevant debug logs. From your description it's not possible to say if it should work as it is unclear what you are doing.
Hey @aleksvujic
I think there’s a confusion;
The fact that you want traffic between Traefik and your backends to be encrypted does not mean the certificates being used need to be signed by a trusted CA.
If you use a self signed certificate with insecureSkipVerify=true, the traffic will be encrypted at the same level.
When to insist on using insecureSkipVerify=false, then? If you have a concern where someone can deploy more backends to your VNET and impose as your backend, then this concern could be mitigated by making Traefik trust the CA which signed your certificate.
If you go with insecureSkipVerify=false, then you need to configure Traefik to trust the relevant CAs. Which once done, every backend presenting a certificate not signed by these CAs, will not be recognized by Traefik.
Hopefully I’ve helped.
It would be nice if it was possible to specify insecureSkipVerify=true on the router level - If I have a single backend with a certificate that cannot be verified it does not mean I want to disable verification across the board.
Is this possible?
So the problem here is the IP's as a few people are saying. The issue is that the IP that the container is presenting from is different than the IP which resolves to the DNS (because of the swarm loadbalancer) so the certificate is being labeled as invalid. There is no easy way to specify what the IP is prior to the containers deployment so we are stuck.
Now I tested this using both a File provider and a Docker label provider in v2.2 and here are the results:
| | TLS v1.2 | TLS v1.3 |
| File Provider | works | not work |
| Docker | not work. |. not work |
So in TLS v1.3 even if I specify the "url" as a dns within the file provider or using Docker it still sees the wrong IP and throws an error. However if I use the filep rovider and downgrade to v1.2 it works without using the insecureSkipVerify. So I think the feature/bug is that we should be able to specify that the Docker service use DNS for the TLS v1.2 case and then we will have to come up with a technical solution for the IP addresses later
If I recall right I got this working by adding CA or Interim certificate to Traefik certificates that points to certificates used in app in internal network. You have to add the certificates with dynamic configuration. Might have been insecure-skip-verify it is some year or so ago I had it with keycloak.