Trouble connecting remote services to local ZNC service behind Traefik

(Using Traefik v3.1.6)

Hello,

Am having trouble connecting services on a remote Hetzner dedicated server to my local ZNC docker container that is behind Traefik. This includes apps such as TheLounge and autobrr. Typically, I never have an issue with (http) services on my remote server connecting to my local services behind both Traefik and Authelia (for example - local radarr/remote unpackerr). All my services use cloudflare as the DNS and they all have valid A/CNAME records.

I did some Googling and found this page: Reverse Proxy - ZNC

As such I gave ZNC two sets of ports - host/cont for web (6501:6501) and irc (6502:6502). I then made an 'irc' entrypoint for Traefik and gave it a host/cont irc port of: '0.0.0.0:7001:7001/tcp'. I also port forwarded 7001 to 7001 in my local router. I then gave ZNC tcp Traefik labels:

traefik.docker.network: traefik_proxy
traefik.enable: true
traefik.tcp.routers.znc-irc-rtr.entryPoints: irc
traefik.tcp.routers.znc-irc-rtr.priority: 20
traefik.tcp.routers.znc-irc-rtr.rule: HostSNI(`znc.mydomain.com`)
traefik.tcp.routers.znc-irc-rtr.service: znc-irc-rtr-svc
traefik.tcp.routers.znc-irc-rtr.tls: true
traefik.tcp.routers.znc-irc-rtr.tls.certresolver:: dns-cloudflare
traefik.tcp.routers.znc-irc-rtr.tls.options: tls-opts@file
traefik.tcp.services.znc-irc-rtr-svc.loadbalancer.server.port: 6502
traefik.tcp.services.znc-irc-rtr-svc.loadbalancer.terminationdelay: 500

Once I spun everything back up, the Traefik dashboard is showing the irc entrypoint:

It's also showing the ZNC tcp router and service:


However, if I try to connect to ZNC via the remote 'TheLounge', all I get is 'Connection closed unexpectedly: AggregateError'. If I try to connect to my ZNC via the remote autobrr, I get:

'04:55:06 ERR client encountered connection error error="dial tcp 104.21.91.176:7001: i/o timeout"= module=irc network=irc.filelist.io
09:13:06 ERR client encountered connection error error="dial tcp 172.67.177.27:7001: i/o timeout"= module=irc network=irc.filelist.io '

Note: I have my local TheLounge and autobrr connecting just fine to ZNC on the new IRC port using the same password, etc, that I'm using on the remote services. The only thing I've changed is going from the local address (znc:6502) to the remote one: znc.mydomain.com:7001

Relevant info:

Port forward for IRC in router:

Traefik TLS options:

tls:
  options:
    tls-opts:
      minVersion: VersionTLS12
      sniStrict : true
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305

Docker container commands (not in order, just as they appeared in Portainer):

--global.checkNewVersion=true 
--global.sendAnonymousUsage=false 
--api=true --api.dashboard=true 
--api.insecure=true 
--entrypoints.http.address=:80 
--entrypoints.https.address=:443 
--entrypoints.traefik.address=:8080 
--entrypoints.ping.address=:8081 
--entrypoints.metrics.address=:8082 
--entrypoints.irc.address=:7001 
--entrypoints.https.http.tls=true 
--entrypoints.https.http.tls.options=tls-opts@file 
--entrypoints.https.http.tls.certresolver=dns-cloudflare 
--entrypoints.https.http.tls.domains[0].main=mydomain.com 
--entrypoints.https.http.tls.domains[0].sans=*.mydomain.com 
--serversTransport.insecureSkipVerify=true 
--entrypoints.http.forwardedheaders.trustedIPs= (relevant local and cloudflare ips for these ones)
--entrypoints.http.proxyprotocol.trustedIPs= 
--entrypoints.https.forwardedheaders.trustedIPs=0: 
--entrypoints.https.proxyprotocol.trustedIPs= 
--entrypoints.http.transport.respondingTimeouts.readTimeout=600 
--entrypoints.http.transport.respondingTimeouts.writeTimeout=0 
--entrypoints.http.transport.respondingTimeouts.idleTimeout=180 
--entrypoints.https.transport.respondingTimeouts.readTimeout=600 
--entrypoints.https.transport.respondingTimeouts.writeTimeout=0 
--entrypoints.https.transport.respondingTimeouts.idleTimeout=180 
--certificatesResolvers.dns-cloudflare.acme.caServer=https://acme-v02.api.letsencrypt.org/directory 
--certificatesResolvers.dns-cloudflare.acme.storage=/acme.json 
--certificatesResolvers.dns-cloudflare.acme.dnsChallenge.provider=cloudflare 
--certificatesResolvers.dns-cloudflare.acme.dnsChallenge.resolvers=1.1.1.1:53,1.0.0.1:53 
--providers.docker=true 
--providers.docker.endpoint=tcp://socket-proxy:2375 
--providers.docker.exposedByDefault=false 
--providers.docker.network=traefik_proxy 
--providers.file.filename=/config.yml 
--providers.file.watch=true 
--log.level=ERROR 
--log=True 
--log.filepath=/logs/traefik.log 
--log.format=json 
--log.maxsize=10 
--log.maxbackups=3 
--log.maxage=3 
--log.compress=true 
--log.nocolor=true 
--accesslog=True 
--accesslog.filepath=/logs/access.log 
--accesslog.bufferingsize=100 
--accesslog.format=json 
--accesslog.fields.defaultmode=keep 
--accesslog.fields.names.StartUTC=drop 
--accesslog.fields.headers.defaultmode=keep 
--accesslog.fields.headers.names.User-Agent=keep 
--accesslog.fields.headers.names.Referer=keep 
--metrics.prometheus=true 
--metrics.prometheus.buckets=0.1,0.3,1.2,5.0 
--metrics.prometheus.addEntryPointsLabels=true 
--metrics.prometheus.addrouterslabels=true 
--metrics.prometheus.addServicesLabels=true 
--metrics.prometheus.manualrouting=true 
--experimental.plugins.cloudflarewarp.modulename=github.com/BetterCorp/cloudflarewarp 
--experimental.plugins.cloudflarewarp.version=v1.3.3 
--experimental.plugins.themepark.modulename=github.com/packruler/traefik-themepark 
--experimental.plugins.themepark.version=v1.4.0 
--ping=true 
--ping.entryPoint=ping

Traefik docker labels (not in order, just as they appeared in Portainer):

traefik.docker.network: traefik_proxy
traefik.enable: true
traefik.http.middlewares.authelia-basic.forwardauth.address: http://authelia:9091/api/verify?auth=basic
traefik.http.middlewares.authelia-basic.forwardauth.authResponseHeaders: Remote-User, Remote-Groups, Remote-Name, Remote-Email
traefik.http.middlewares.authelia-basic.forwardauth.trustForwardHeader: true
traefik.http.middlewares.authelia.forwardauth.address: http://authelia:9091/api/verify?rd=https://authelia.mydomain.com/
traefik.http.middlewares.authelia.forwardauth.authResponseHeaders: Remote-User, Remote-Groups, Remote-Name, Remote-Email
traefik.http.middlewares.authelia.forwardauth.trustForwardHeader: true
traefik.http.middlewares.autodetect.contenttype: true
traefik.http.middlewares.cloudflarewarp.plugin.cloudflarewarp.disableDefault: false
traefik.http.middlewares.gzip.compress: true
traefik.http.middlewares.redirect-to-https.redirectscheme.permanent: true
traefik.http.middlewares.redirect-to-https.redirectscheme.scheme: https
traefik.http.routers.metrics-rtr-secure.entrypoints: https
traefik.http.routers.metrics-rtr-secure.middlewares: globalHeaders@file,secureHeaders@file,autodetect@docker,gzip@docker,robotHeaders@file,hsts@file,cloudflarewarp@docker
traefik.http.routers.metrics-rtr-secure.priority: 20
traefik.http.routers.metrics-rtr-secure.rule: Host(`traefik.mydomain.com`) && (Path(`/prometheus`))
traefik.http.routers.metrics-rtr-secure.service: prometheus@internal
traefik.http.routers.metrics-rtr-secure.tls: true
traefik.http.routers.metrics-rtr-secure.tls.options: tls-opts@file
traefik.http.routers.metrics-rtr.entrypoints: http
traefik.http.routers.metrics-rtr.middlewares: globalHeaders@file,redirect-to-https@docker,autodetect@docker,gzip@docker,robotHeaders@file,cloudflarewarp@docker
traefik.http.routers.metrics-rtr.priority: 20
traefik.http.routers.metrics-rtr.rule: Host(`traefik.mydomain.com`) && (Path(`/prometheus`))
traefik.http.routers.metrics-rtr.service: prometheus@internal
traefik.http.routers.traefik-rtr-secure.entrypoints: https
traefik.http.routers.traefik-rtr-secure.middlewares: globalHeaders@file,secureHeaders@file,autodetect@docker,gzip@docker,robotHeaders@file,hsts@file,cloudflarewarp@docker,authelia@docker
traefik.http.routers.traefik-rtr-secure.priority: 20
traefik.http.routers.traefik-rtr-secure.rule: Host(`traefik.mydomain.com`)
traefik.http.routers.traefik-rtr-secure.service: api@internal
traefik.http.routers.traefik-rtr-secure.tls: true
traefik.http.routers.traefik-rtr-secure.tls.options: tls-opts@file
traefik.http.routers.traefik-rtr.entrypoints: http
traefik.http.routers.traefik-rtr.middlewares: globalHeaders@file,redirect-to-https@docker,autodetect@docker,gzip@docker,robotHeaders@file,cloudflarewarp@docker,authelia@docker
traefik.http.routers.traefik-rtr.priority: 20
traefik.http.routers.traefik-rtr.rule: Host(`traefik.mydomain.com`)
traefik.http.routers.traefik-rtr.service: api@internal

Am I missing something here? Seem to have done everything in that ZNC reverse proxy page. Am I missing something within Traefik's commands/labels/config related to TCP?

Not really the same Problem (I am trying to use traefik to Reverse Proxy the Web GUI), but I think you have the same Issue.

You might need to Configure a Port in ZNC_DATA_FOLDER/configs/znc.conf and DISABLE SSL for that (and of course DO NOT open/expose that Port in the Port Mapping Section - it needs to be an Internal Port to your Switch Only !)

cat /znc-data/configs/znc.conf

# ...

# Default (Exposed to Outside)
<Listener listener0>
	AllowIRC = true
	AllowWeb = true
	IPv4 = true
	IPv6 = true
	Port = 6543
	SSL = true
	URIPrefix = /
</Listener>

# Added for Traefik Proxy Network Only !
<Listener listener1>
	AllowIRC = false
	AllowWeb = true
	IPv4 = true
	IPv6 = true
	Port = 8123
	SSL = false
	URIPrefix = /
</Listener>

# ....

Not sure if it's the same Issue, but maybe worth for you to give it a Try :slight_smile:.