TLS encryption missing on https redirect with self signed certificates

Traefik Traefik v2 (latest)
1

Hi everyone,
I need help on a subject which is bothering me for a good deal of time.
Basically I've got a stack file with Traefik as ingress server and Apache server (no control on this last service).

I need Traefik (v.2.3.2) to redirect HTTP requests to HTTPS.
The redirection and HTTP exchange between Traefik and Apache works fine but it seems that no TLS session is opened on the Traefik side and the whole exchange terminates with a "ERR_TOO_MANY_REDIRECTS".

Follows the truncated HTTP exchange captured in Apache server:
traefik_traefik.3h08jqqzfo1wy39au32lxzfle.xavvr3mgesj2tmr53hs0xoran.mcs_overlay_network.59684 > ed62f7fa6d45.80: Flags [P.], cksum 0x6f33 (incorrect -> 0xf44f), seq 1:717, ack 1, win 221, options [nop,nop,TS val 11918813 ecr 11918813], length 716: HTTP, length: 716
GET / HTTP/1.1
Host: 172.23.17.226
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7
Dnt: 1
Purpose: prefetch
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.0.0.2
X-Forwarded-Host: 172.23.17.226
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: d3afa3e2ad69
X-Real-Ip: 10.0.0.2

10:17:22.504838 IP (tos 0x0, ttl 64, id 44541, offset 0, flags [DF], proto TCP (6), length 52)
ed62f7fa6d45.80 > traefik_traefik.3h08jqqzfo1wy39au32lxzfle.xavvr3mgesj2tmr53hs0xoran.mcs_overlay_network.59684: Flags [.], cksum 0x6c67 (incorrect -> 0x0adc), seq 1, ack 717, win 230, options [nop,nop,TS val 11918813 ecr 11918813], length 0
10:17:22.543833 IP (tos 0x0, ttl 64, id 8206, offset 0, flags [DF], proto TCP (6), length 769)
traefik_traefik.3h08jqqzfo1wy39au32lxzfle.xavvr3mgesj2tmr53hs0xoran.mcs_overlay_network.59684 > ed62f7fa6d45.80: Flags [P.], cksum 0x6f34 (incorrect -> 0x1424), seq 717:1434, ack 475, win 229, options [nop,nop,TS val 11918852 ecr 11918815], length 717: HTTP, length: 717
GET // HTTP/1.1
Host: 172.23.17.226
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7
Dnt: 1
Purpose: prefetch
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.0.0.2
X-Forwarded-Host: 172.23.17.226
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: d3afa3e2ad69
X-Real-Ip: 10.0.0.2

10:17:22.544451 IP (tos 0x0, ttl 64, id 44543, offset 0, flags [DF], proto TCP (6), length 526)
ed62f7fa6d45.80 > traefik_traefik.3h08jqqzfo1wy39au32lxzfle.xavvr3mgesj2tmr53hs0xoran.mcs_overlay_network.59684: Flags [P.], cksum 0x6e41 (incorrect -> 0x4d48), seq 475:949, ack 1434, win 241, options [nop,nop,TS val 11918853 ecr 11918852], length 474: HTTP, length: 474
HTTP/1.1 302 Found
Date: Fri, 23 Oct 2020 10:17:22 GMT
Server: Apache/2.4.43 (Unix)
Location: https://172.23.17.226//
Content-Length: 284
Content-Type: text/html; charset=iso-8859-1

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>302 Found</title>
    </head><body>
    <h1>Found</h1>
    <p>The document has moved <a href="https://172.23.17.226//">here</a>.</p>
    <hr>
    <address>Apache/2.4.43 (Unix) Server at 172.23.17.226 Port 80</address>
    </body></html>

...

For TLS encryption I need to use self signed certificates provided by the Apache server container developer and I cannot use my own certificates or an external provider for them.

Since, the TLS initiation is missing on Traefik side I suspect that either my stack file configuration is wrong or the self signed certificates are not correctly addressed inside the Traefik container and the default generated are used in their place.

I've been trying on and on with different configuration with no success.
This is my current stack file (excerpt):

version: '3.8'

services:
traefik:
image: traefik:v2.3
command:
- "--providers.docker.endpoint=unix:///var/run/docker.sock"
- "--api.insecure=true"
- "--api.dashboard=true"
- "--metrics.prometheus=true"
- "--metrics.prometheus.buckets=0.1,0.3,1.2,5.0"
- "--providers.docker=true"
- "--providers.docker.swarmmode=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.watch"
- "--providers.file.directory=/etc/traefik/dynamic"
- "--providers.file.watch=true"
- "--log.filePath=/etc/traefik/traefik.log"
- "--accesslog=true"
- "--accesslog.filepath=/etc/traefik/traefik_access.log"
- "--accesslog.bufferingsize=100"
- "--log"
- "--log.level=DEBUG"
- "--api=true"
- "--entrypoints.web.address=:80"
- "--entrypoints.web-secured.address=:443"
networks:
- ingress
- my_overlay_network
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /mnt/traefik/traefik.toml:/etc/traefik/dynamic/conf.toml
- /mnt/traefik/certs:/etc/traefik/dynamic/certs
ports:
- "80:80"
- "8080:8080"
- "443:443"
deploy:
mode: global
labels:
- "traefik.enable=true"
- "traefik.http.services.dummy-svc.loadbalancer.server.port=9999"
...

apache_server:
image: apache_server_image
...
networks:
- my_overlay_network
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.middlewares.mcxptt_configurator-redirect-web-secure.redirectscheme.scheme=https"
- "traefik.http.middlewares.mcxptt_configurator-redirect-web-secure.redirectscheme.permanent=true"
- "traefik.http.routers.mcxptt_configurator.middlewares=mcxptt_configurator-redirect-web-secure"
- "traefik.http.routers.mcxptt_configurator.rule=Host(172.23.17.226)"
- "traefik.http.routers.mcxptt_configurator.service=apache_server"
- "traefik.http.routers.mcxptt_configurator.entrypoints=web"
- "traefik.http.services.mcxptt_configurator.loadbalancer.server.port=80"
- "traefik.http.routers.mcxptt_configurator-auth.rule=Host(172.23.17.226)"
- "traefik.http.routers.mcxptt_configurator-auth.service=apache_server"
- "traefik.http.routers.mcxptt_configurator-auth.entrypoints=web-secured"
- "traefik.http.routers.mcxptt_configurator-auth.tls=true"
- "traefik.http.services.mcxptt_configurator-auth.loadbalancer.server.port=443"
- "traefik.docker.network=my_overlay_network"
...
The dynamic configuration file traefik.toml is loaded correctly in the Traefik container together with the .pem and .key certificates files.

The treafik.toml file content is:
[tls.stores]
[tls.stores.default]
[tls.stores.default.defaultCertificate]
certFile = "/etc/traefik/dynamic/certs/server.pem"
keyFile = "/etc/traefik/dynamic/certs/server.key"

PIease be patient with this request of mine because I know that it seems quite similar to other topics already closed but I can find no resolutive clue on that.

I'd like to thank you in advance for any help on this topic.
Luca.

2

Hello @luca.spinacci,

It seems that the traefik.toml file is not read/applied since the generated default cert is served only when a default cert has not been explicitly defined in configuration.
Could you provide your logs ?
Also, probably just a typo when writing here, but your file is named "treafik.toml" instead of "traefik.toml".

3

Hello @rtribotte,
I really appreciate your help.
Follows the link to the log file hoping that it can help:
https://easyupload.io/rzol0o
You can download the file using password "luca"
I used a link due to the limitiations imposed for message and attachement size.

Actually the file name you mentioned was my fault, i.e. a typo.

Thank you!
Luca.

4

Hello @luca.spinacci,

I checked your log file, but i have no clue.
Still, Traefik cannot serve the generated certificate once the default certificate has been explicitly set through the dynamic conf.
In your logs, i cannot see a request handled (logs from oxy) can you produce logs of Traefik when the request is handled ?
Also did you checked your certificates ?

5

Hello @rtribotte,
thank you for supporting me!
Certificates, according to my understanding, should work fine: https requests from my browser are correctly processed by my Apache server while bypassing Traefik, i.e. whenever Traefik is not exposing port 443 but Apache is doing instead.
I'm quite confused on Traefik's behavior because reading other posts I'm afraid that my configuration is valid only for terminating SSL on Traefik but not on the Apache backend. That may explain while the capture on the backend is only "plain" http.
Apart from this, I didn't find any clue from the logs too.
I've configured the debug level. Any suggestions on tracing with deeper info?
Thank you!
Luca.

6

Did you tried the passthrough option ? Traefik Routers Documentation - Traefik

7

Yes, I'm sure I did that config too.
Anyway, I'll share you the dynamic config as per my understanding the passthrough option is valid only with the toml or yaml file.
Thx!
Luca

8

It's not only available with the file provider, you can use the docker references to see how to use it through labels :wink: Traefik Docker Configuration Documentation - Traefik

9

Hi @rtribotte,
your suggestions, i.e. passthrough option, seems working but now I'm stuck on another problem as the tcp router works only in case I provide a "catch all" HostSNI(*) and that put me in some trouble due to the need to address two different service via tcp router with the same 443 entry point.
In simpler words, apache_server1 needs to be reached via port 443 at e.g. HostSNI(10.0.0.1/server1) and apache_server2 needs to be reached via port 443 at e.g. HostSNI(10.0.0.1/server2). Now only the catch all HostSNI(*) rule is working and I-m unable to differentiate beetween server1 and server2.
any clues on that?
Thx

10

Hello @luca.spinacci,

I have no clue, can your share your configuration ?
The documentation state that you'll need a TLS router:

It is important to note that the Server Name Indication is an extension of the TLS protocol. Hence, only TLS routers will be able to specify a domain name with that rule. However, non-TLS routers will have to explicitly use that rule with * (every domain) to state that every non-TLS request will be handled by the router.

11

Hello,
this is the, simplified, working configuration with HostSNI(*):

version: '3.8'

services:
traefik:
image: traefik:v2.3
command:
- "--providers.docker.endpoint=unix:///var/run/docker.sock"
- "--api.insecure=true"
- "--api.dashboard=true"
- "--metrics.prometheus=true"
- "--metrics.prometheus.buckets=0.1,0.3,1.2,5.0"
- "--providers.docker=true"
- "--providers.docker.swarmmode=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.watch"
- "--providers.file.directory=/etc/traefik/dynamic"
- "--providers.file.watch=true"
- "--log.filePath=/etc/traefik/traefik.log"
- "--log.format=json"
- "--accesslog=true"
- "--accesslog.filepath=/etc/traefik/traefik_access.log"
- "--accesslog.bufferingsize=100"
- "--log"
- "--log.level=DEBUG"
- "--api=true"
- "--entrypoints.web.address=:80"
- "--entrypoints.web-secured.address=:443"
- "--serversTransport.insecureSkipVerify=true"
- "--serversTransport.rootCAs=/etc/traefik/dynamic/certs/server.pem, /etc/traefik/dynamic/certs/server.key"
networks:
- traefik
- my_overlay_network
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- type: bind
source: /mnt/traefik/traefik.toml
target: /etc/traefik/dynamic/traefik.toml
- type: bind
source: /mnt/traefik/certs
target: /etc/traefik/dynamic/certs
ports:
- "80:80"
- "8080:8080"
- "443:443"
...

...

apache_server1:
image: myrepo/apache_server   
  ...
networks:
  - my_overlay_network
deploy:
  labels:
   - "traefik.enable=true"
   - "traefik.tcp.routers.apache_server1.entrypoints=web-secured"
   - "traefik.tcp.routers.apache_server1.rule=HostSNI(`*`)"
   - "traefik.tcp.routers.apache_server1.service=apache_server1"
   - "traefik.tcp.routers.apache_server1.tls=true"
   - "traefik.tcp.routers.apache_server1.tls.passthrough=true"
   - "traefik.tcp.services.apache_server1.loadbalancer.server.port=443"       
   - "traefik.docker.network=my_overlay_network"    
...  
  
apache_server2:
  ...

The "mistaken" config is obtained by writing in the HostSNI() field your domain address.

Thank you,
Luca

12

Hello @luca.spinacci,

The hostSNI rule is probably wrong here, did you tried with the servername only ?

HostSNI(`server1`)
13

Yes, I tried it, for instance HostSNI(10.0.0.1) as well as HostSNI(server1.local)...

Luca

14

Ok thanks for the clarification.
To be honest, i cannot troubleshoot further your problem without more inputs.
Can you share the output of the endpoint /api/rawdata ?
Also can you provide the logs showing how the request with the servername is handled ?
Also can you share the command you use to make the request ?

15

Hi,
I split the information in two sections: the former is for the working rule catch-all HostSNI('*'); the latter for a not working rule HostSNI('https://172.23.17.226:443') but believe me other combination produce the same output.
The command I use is a simple CLI "curl" from the host running the swarm with my Apache server.

//////////////////////// Rule HostSNI('*') ////////////////////////////
File traefik.log:

...
{"entryPointName":"web-secured","level":"debug","msg":"Creating TCP server 0 at 172.0.10.34:443","routerName":"apache_server@docker","serverName":0,"serviceName":"apache_server","time":"2020-11-06T14
{"entryPointName":"web-secured","level":"debug","msg":"Adding route * on TCP","routerName":"apache_server@docker","time":"2020-11-06T14:50:37Z"}
...

api/rawdata:

{"routers":{"api@internal":{"entryPoints":["traefik"],"service":"api@internal","rule":"PathPrefix(/api)","priority":2147483646,"status":"enabled","using":["traefik"]},"dashboard@internal":{"entryPoints":["traefik"],"middlewares":["dashboard_redirect@internal","dashboard_stripprefix@internal"],"service":"dashboard@internal","rule":"PathPrefix(/)","priority":2147483645,"status":"enabled","using":["traefik"]},"grafana@docker":{"entryPoints":["web"],"service":"grafana","rule":"Host(traefik.grafana.com)","status":"enabled","using":["web"]},"kibana@docker":{"entryPoints":["web"],"service":"kibana","rule":"Host(traefik.kibana.com)","status":"enabled","using":["web"]},"portainer@docker":{"entryPoints":["portainer"],"service":"portainer","rule":"Host(traefik.portainer.com)","status":"enabled","using":["portainer"]},"prometheus@docker":{"entryPoints":["web"],"service":"prometheus","rule":"Host(traefik.prometheus.com)","status":"enabled","using":["web"]},"prometheus@internal":{"entryPoints":["traefik"],"service":"prometheus@internal","rule":"PathPrefix(/metrics)","priority":2147483647,"status":"enabled","using":["traefik"]},"traefik-traefik@docker":{"entryPoints":["kibana","portainer","web","web-secured"],"service":"dummy-svc","rule":"Host(traefik-traefik)","status":"enabled","using":["kibana","portainer","web","web-secured"]}},"middlewares":{"dashboard_redirect@internal":{"redirectRegex":{"regex":"^(http:\/\/(\[[\w:.]+\]|[\w\._-]+)(:\d+)?)\/$","replacement":"${1}/dashboard/","permanent":true},"status":"enabled","usedBy":["dashboard@internal"]},"dashboard_stripprefix@internal":{"stripPrefix":{"prefixes":["/dashboard/","/dashboard"]},"status":"enabled","usedBy":["dashboard@internal"]}},"services":{"api@internal":{"status":"enabled","usedBy":["api@internal"]},"dashboard@internal":{"status":"enabled","usedBy":["dashboard@internal"]},"dummy-svc@docker":{"loadBalancer":{"servers":[{"url":"http://10.0.0.9:9999"},{"url":"http://172.0.10.13:9999"},{"url":"http://10.0.1.5:9999"}],"passHostHeader":true},"status":"enabled","usedBy":["traefik-traefik@docker"],"serverStatus":{"http://10.0.0.9:9999":"UP","http://10.0.1.5:9999":"UP","http://172.0.10.13:9999":"UP"}},"grafana@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.42:3000"}],"passHostHeader":true},"status":"enabled","usedBy":["grafana@docker"],"serverStatus":{"http://172.0.10.42:3000":"UP"}},"kibana@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.40:5601"}],"passHostHeader":true},"status":"enabled","usedBy":["kibana@docker"],"serverStatus":{"http://172.0.10.40:5601":"UP"}},"noop@internal":{"status":"enabled"},"portainer@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.24:9000"}],"passHostHeader":true},"status":"enabled","usedBy":["portainer@docker"],"serverStatus":{"http://172.0.10.24:9000":"UP"}},"prometheus@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.38:9090"}],"passHostHeader":true},"status":"enabled","usedBy":["prometheus@docker"],"serverStatus":{"http://172.0.10.38:9090":"UP"}},"prometheus@internal":{"status":"enabled","usedBy":["prometheus@internal"]}},"tcpRouters":{"apache_server@docker":{"entryPoints":["web-secured"],"service":"apache_server","rule":"HostSNI(*)","tls":{"passthrough":true},"status":"enabled","using":["web-secured"]}},"tcpServices":{"apache_server@docker":{"loadBalancer":{"terminationDelay":100,"servers":[{"address":"172.0.10.34:443"}]},"status":"enabled","usedBy":["apache_server@docker"]}}}

curl command client Side (172.23.17.226):

[root@my docker-traefik-prometheus]# curl -k -v https://172.23.17.226:443

  • About to connect() to 172.23.17.226 port 443 (#0)
  • Trying 172.23.17.226...
  • Connected to 172.23.17.226 (172.23.17.226) port 443 (#0)
  • Initializing NSS with certpath: sql:/etc/pki/nssdb
  • skipping SSL peer certificate verification
  • SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • Server certificate:
  •   subject: E=root@b89efdb12d33,CN=b89efdb12d33,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--

  •   start date: Apr 20 10:35:39 2020 GMT

  •   expire date: Apr 20 10:35:39 2021 GMT

  •   common name: b89efdb12d33

  •   issuer: E=root@b89efdb12d33,CN=b89efdb12d33,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--

GET / HTTP/1.1
User-Agent: curl/7.29.0
Host: 172.23.17.226
Accept: /

< HTTP/1.1 302 Found
< Date: Fri, 06 Nov 2020 14:44:03 GMT
< Server: Apache/2.4.43 (Unix)
< Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< X-Powered-By: PHP/5.6.40
< Set-Cookie: PHPSESSID=fd20fkbesc37gqfkbosqkh4t50; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Location: https://172.23.17.226/authentication/login
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
<

  • Connection #0 to host 172.23.17.226 left intact
    [root@my docker-traefik-prometheus]# curl -k -v https://172.23.17.226/authentication/login:443
  • About to connect() to 172.23.17.226 port 443 (#0)
  • Trying 172.23.17.226...
  • Connected to 172.23.17.226 (172.23.17.226) port 443 (#0)
  • Initializing NSS with certpath: sql:/etc/pki/nssdb
  • skipping SSL peer certificate verification
  • SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • Server certificate:
  •   subject: E=root@b89efdb12d33,CN=b89efdb12d33,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--

  •   start date: Apr 20 10:35:39 2020 GMT

  •   expire date: Apr 20 10:35:39 2021 GMT

  •   common name: b89efdb12d33

  •   issuer: E=root@b89efdb12d33,CN=b89efdb12d33,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--

GET /authentication/login:443 HTTP/1.1
User-Agent: curl/7.29.0
Host: 172.23.17.226
Accept: /

< HTTP/1.1 302 Found
< Date: Fri, 06 Nov 2020 14:44:54 GMT
< Server: Apache/2.4.43 (Unix)
< Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< X-Powered-By: PHP/5.6.40
< Set-Cookie: PHPSESSID=6jgqmfebjjd4g2ferphvdoqe25; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Location: https://172.23.17.226/authentication/login
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
<

  • Connection #0 to host 172.23.17.226 left intact

tcpdump Apache server Side:
14:44:54.125042 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [S.], cksum 0x6c5e (incorrect -> 0x6a92), seq 1167419855, ack 1605348140, win 27960, options [mss 1410,sackOK,TS val 5185789 ecr 5150919,nop,wscale 7], length 0
14:44:54.125513 IP (tos 0x0, ttl 64, id 13293, offset 0, flags [DF], proto TCP (6), length 52)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [.], cksum 0x0587 (correct), seq 1, ack 1, win 221, options [nop,nop,TS val 5150920 ecr 5185789], length 0
14:44:54.125777 IP (tos 0x0, ttl 64, id 41459, offset 0, flags [DF], proto UDP (17), length 70)
localhost.52971 > 127.0.0.11.34439: [bad udp cksum 0xfe4f -> 0x1192!] UDP, length 42
14:44:54.163110 IP (tos 0x0, ttl 64, id 18642, offset 0, flags [DF], proto TCP (6), length 1446)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [P.], cksum 0x71c8 (incorrect -> 0x9c00), seq 1:1395, ack 174, win 227, options [nop,nop,TS val 5185827 ecr 5150923], length 1394
14:44:54.163746 IP (tos 0x0, ttl 64, id 13295, offset 0, flags [DF], proto TCP (6), length 52)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [.], cksum 0xff05 (correct), seq 174, ack 1395, win 243, options [nop,nop,TS val 5150958 ecr 5185827], length 0
14:44:54.171930 IP (tos 0x0, ttl 64, id 13296, offset 0, flags [DF], proto TCP (6), length 145)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [P.], cksum 0x26d5 (correct), seq 174:267, ack 1395, win 243, options [nop,nop,TS val 5150966 ecr 5185827], length 93
14:44:54.179692 IP (tos 0x0, ttl 64, id 18643, offset 0, flags [DF], proto TCP (6), length 103)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [P.], cksum 0x6c89 (incorrect -> 0xba16), seq 1395:1446, ack 267, win 227, options [nop,nop,TS val 5185844 ecr 5150966], length 51
14:44:54.202122 IP (tos 0x0, ttl 64, id 13297, offset 0, flags [DF], proto TCP (6), length 182)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [P.], cksum 0xe0f6 (correct), seq 267:397, ack 1446, win 243, options [nop,nop,TS val 5150997 ecr 5185844], length 130
14:44:54.241547 IP (tos 0x0, ttl 64, id 18644, offset 0, flags [DF], proto TCP (6), length 52)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [.], cksum 0x6c56 (incorrect -> 0xfd84), seq 1446, ack 397, win 236, options [nop,nop,TS val 5185906 ecr 5150997], length 0
14:44:54.683793 IP (tos 0x0, ttl 64, id 18645, offset 0, flags [DF], proto TCP (6), length 633)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [P.], cksum 0x6e9b (incorrect -> 0x0ac2), seq 1446:2027, ack 397, win 236, options [nop,nop,TS val 5186348 ecr 5150997], length 581
14:44:54.688351 IP (tos 0x0, ttl 64, id 13298, offset 0, flags [DF], proto TCP (6), length 83)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [P.], cksum 0x0f9b (correct), seq 397:428, ack 2027, win 264, options [nop,nop,TS val 5151483 ecr 5186348], length 31
14:44:54.688472 IP (tos 0x0, ttl 64, id 18646, offset 0, flags [DF], proto TCP (6), length 52)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [.], cksum 0x6c56 (incorrect -> 0xf77b), seq 2027, ack 428, win 236, options [nop,nop,TS val 5186353 ecr 5151483], length 0
14:44:54.688498 IP (tos 0x0, ttl 64, id 13299, offset 0, flags [DF], proto TCP (6), length 52)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [F.], cksum 0xf763 (correct), seq 428, ack 2027, win 264, options [nop,nop,TS val 5151483 ecr 5186348], length 0
14:44:54.691004 IP (tos 0x0, ttl 64, id 18647, offset 0, flags [DF], proto TCP (6), length 83)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [P.], cksum 0x6c75 (incorrect -> 0x17c8), seq 2027:2058, ack 429, win 236, options [nop,nop,TS val 5186355 ecr 5151483], length 31
14:44:54.691220 IP (tos 0x0, ttl 64, id 18648, offset 0, flags [DF], proto TCP (6), length 52)
424087a53a03.443 > traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298: Flags [F.], cksum 0x6c56 (incorrect -> 0xf758), seq 2058, ack 429, win 236, options [nop,nop,TS val 5186355 ecr 5151483], length 0
14:44:54.691663 IP (tos 0x0, ttl 64, id 13300, offset 0, flags [DF], proto TCP (6), length 52)
traefik_traefik.wi8z2041jn4venzsdgjfeivnd.bcww66bzc64vnh7shla2zy8lz.mcs_overlay_network.36298 > 424087a53a03.443: Flags [.], cksum 0xf739 (correct), seq 429, ack 2059, win 264, options [nop,nop,TS val 5151486 ecr 5186355], length 0

//////////////////////// Rule HostSNI('https://172.23.17.226:443') ////////////////////////////
File traefik.log:

...
{"entryPointName":"web-secured","level":"debug","msg":"Creating TCP server 0 at 172.0.10.3:443","routerName":"apache_server@docker","serverName":0,"serviceName":"apache_server","time":"2020-11-06T15:
{"entryPointName":"web-secured","level":"debug","msg":"Adding route https://172.23.17.226:443 on TCP","routerName":"apache_server@docker","time":"2020-11-06T15:10:52Z"}
...

api/rawdata:

{"routers":{"api@internal":{"entryPoints":["traefik"],"service":"api@internal","rule":"PathPrefix(/api)","priority":2147483646,"status":"enabled","using":["traefik"]},"dashboard@internal":{"entryPoints":["traefik"],"middlewares":["dashboard_redirect@internal","dashboard_stripprefix@internal"],"service":"dashboard@internal","rule":"PathPrefix(/)","priority":2147483645,"status":"enabled","using":["traefik"]},"grafana@docker":{"entryPoints":["web"],"service":"grafana","rule":"Host(traefik.grafana.com)","status":"enabled","using":["web"]},"kibana@docker":{"entryPoints":["web"],"service":"kibana","rule":"Host(traefik.kibana.com)","status":"enabled","using":["web"]},"portainer@docker":{"entryPoints":["portainer"],"service":"portainer","rule":"Host(traefik.portainer.com)","status":"enabled","using":["portainer"]},"prometheus@docker":{"entryPoints":["web"],"service":"prometheus","rule":"Host(traefik.prometheus.com)","status":"enabled","using":["web"]},"prometheus@internal":{"entryPoints":["traefik"],"service":"prometheus@internal","rule":"PathPrefix(/metrics)","priority":2147483647,"status":"enabled","using":["traefik"]},"traefik-traefik@docker":{"entryPoints":["kibana","portainer","web","web-secured"],"service":"dummy-svc","rule":"Host(traefik-traefik)","status":"enabled","using":["kibana","portainer","web","web-secured"]}},"middlewares":{"dashboard_redirect@internal":{"redirectRegex":{"regex":"^(http:\/\/(\[[\w:.]+\]|[\w\._-]+)(:\d+)?)\/$","replacement":"${1}/dashboard/","permanent":true},"status":"enabled","usedBy":["dashboard@internal"]},"dashboard_stripprefix@internal":{"stripPrefix":{"prefixes":["/dashboard/","/dashboard"]},"status":"enabled","usedBy":["dashboard@internal"]}},"services":{"api@internal":{"status":"enabled","usedBy":["api@internal"]},"dashboard@internal":{"status":"enabled","usedBy":["dashboard@internal"]},"dummy-svc@docker":{"loadBalancer":{"servers":[{"url":"http://10.0.0.27:9999"},{"url":"http://10.0.0.28:9999"},{"url":"http://10.0.0.29:9999"}],"passHostHeader":true},"status":"enabled","usedBy":["traefik-traefik@docker"],"serverStatus":{"http://10.0.0.27:9999":"UP","http://10.0.0.28:9999":"UP","http://10.0.0.29:9999":"UP"}},"grafana@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.6:3000"}],"passHostHeader":true},"status":"enabled","usedBy":["grafana@docker"],"serverStatus":{"http://172.0.10.6:3000":"UP"}},"kibana@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.44:5601"}],"passHostHeader":true},"status":"enabled","usedBy":["kibana@docker"],"serverStatus":{"http://172.0.10.44:5601":"UP"}},"noop@internal":{"status":"enabled"},"portainer@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.30:9000"}],"passHostHeader":true},"status":"enabled","usedBy":["portainer@docker"],"serverStatus":{"http://172.0.10.30:9000":"UP"}},"prometheus@docker":{"loadBalancer":{"servers":[{"url":"http://172.0.10.48:9090"}],"passHostHeader":true},"status":"enabled","usedBy":["prometheus@docker"],"serverStatus":{"http://172.0.10.48:9090":"UP"}},"prometheus@internal":{"status":"enabled","usedBy":["prometheus@internal"]}},"tcpRouters":{"apache_server@docker":{"entryPoints":["web-secured"],"service":"apache_server","rule":"HostSNI(https://172.23.17.226:443)","tls":{"passthrough":true},"status":"enabled","using":["web-secured"]}},"tcpServices":{"apache_server@docker":{"loadBalancer":{"terminationDelay":100,"servers":[{"address":"172.0.10.3:443"}]},"status":"enabled","usedBy":["apache_server@docker"]}}}

curl command client Side (172.23.17.226):

[root@my docker-traefik-prometheus]# curl -k -v https://172.23.17.226:443

  • About to connect() to 172.23.17.226 port 443 (#0)
  • Trying 172.23.17.226...
  • Connected to 172.23.17.226 (172.23.17.226) port 443 (#0)
  • Initializing NSS with certpath: sql:/etc/pki/nssdb
  • skipping SSL peer certificate verification
  • SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • Server certificate:
  •   subject: E=root@b89efdb12d33,CN=b89efdb12d33,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--

  •   start date: Apr 20 10:35:39 2020 GMT

  •   expire date: Apr 20 10:35:39 2021 GMT

  •   common name: b89efdb12d33

  •   issuer: E=root@b89efdb12d33,CN=b89efdb12d33,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--

GET / HTTP/1.1
User-Agent: curl/7.29.0
Host: 172.23.17.226
Accept: /

< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Fri, 06 Nov 2020 15:06:55 GMT
< Content-Length: 19
<
404 page not found

  • Connection #0 to host 172.23.17.226 left intact

tcpdump Apache server Side:

Thank you,
Luca