I have gitea setup behind traefik and it's working nicely, HTTPS clones are working, but I cannot seem to setup SSH clones. I have read through quite a number of guides and topics and attempted to apply what they say which is how I've gotten to this point, which I feel is mostly right, but something isn't quite working. Would someone be able to review my config please?
SSH is running on port 22 in the Gitea container, I'm attempting to expose this as port 222 through traefik.
When I try to clone I get this error:
GIT_SSH_COMMAND="ssh -v" git clone ssh://git@gitea.domain.xyz:222/user/TestRepo.git
Cloning into 'TestRepo'...
OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to gitea.domain.xyz [ip] port 222.
debug1: connect to address [ip] port 222: Connection timed out
ssh: connect to host [domain] port 222: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I have ensured that UFW is allowing port 222 on my server
So in the static config traefik.yml I have an entrypoint setup for port 222:
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: websecure
scheme: https
websecure:
address: ":443"
http:
tls: {}
gitea_ssh:
address: ":222"
Then in the traefik docker I have port 222 forwarded to 222:
version: "3.4"
services:
traefik:
image: "traefik:latest"
ports:
- "80:80"
- "443:443"
- "222:222"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./acme.json:/acme.json"
- "./traefik.yml:/traefik.yml"
- "./dynamic-conf:/etc/traefik/dynamic/"
networks:
- web
networks:
web:
external: true
Then I have my gitea docker setup like so:
version: "3.8"
networks:
gitea:
external: false
web:
external: true
services:
server:
image: gitea/gitea:latest
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
- DB_TYPE=mysql
- DB_HOST=db:3306
- DB_NAME=gitea
- DB_USER=gitea
- DB_PASSWD=password
- RUN_MODE=prod
- DOMAIN=gitea.domain.xyz
- HTTP_PORT=3000
- ROOT_URL=https://gitea.domain.xyz
# SSH port displayed in clone URL.
- SSH_DOMAIN=gitea.domain.xyz
- SSH_PORT=222
# Port for the built-in SSH server
- SSH_LISTEN_PORT=22
restart: always
networks:
- gitea
- web
volumes:
- /srv/gitea:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
depends_on:
- db
labels:
- "traefik.enable=true"
- "traefik.http.routers.gitea.rule=Host(`gitea.domain.xyz`)"
- "traefik.http.routers.gitea.entrypoints=web"
- "traefik.http.routers.gitea.entrypoints=websecure"
- 'traefik.http.services.gitea.loadbalancer.server.port=3000'
- "traefik.backend=gitea"
- "traefik.docker.network=web"
- "traefik.default.protocol=http"
- "traefik.port=3000"
- "traefik.http.routers.gitea.tls=true"
- "traefik.http.routers.gitea.tls.certresolver=letsEncrypt"
- "traefik.http.routers.gitea.tls.domains[0].main=gitea.domain.xyz"
# SSH routing, can't route based on host so anything to port 222 will come to this container
- "traefik.tcp.routers.gitea-ssh.rule=HostSNI(`*`)"
- "traefik.tcp.routers.gitea-ssh.entrypoints=gitea_ssh"
- "traefik.tcp.routers.gitea-ssh.service=gitea-ssh-svc"
- "traefik.tcp.services.gitea-ssh-svc.loadbalancer.server.port=22"
db:
image: mariadb:latest
container_name: gitea_db
restart: always
environment:
- MYSQL_ROOT_PASSWORD=some_root_password
- MYSQL_USER=gitea
- MYSQL_PASSWORD=password
- MYSQL_DATABASE=gitea
networks:
- gitea
volumes:
- /srv/gitea/db:/var/lib/mysql
ports:
- 9090:8080
If it’s only TCP, not HTTP, use the right router:
traefik.tcp.routers.gitea
Sorry I don't quite follow what you mean I've done wrong, in my docker compose file I am using the tcp router for the ssh specific setup, the http router is used for the web front end.
You are right, misread your config.
Do you need this line?
No worries.
I commented it out and the web front end still works but no change to the error I get with SSH sadly.
Have you checked if gitea really has an open port 22 within the container?
Have you tried to replace gitea image with something like a debian:stable-slim container to see if port 22 works?
I believe the ssh server in the gitea container is working just fine, I did try this before posting here, I probably should have mentioned it in my first post!
bash-5.1# ssh localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ED25519 key fingerprint is *
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
root@localhost: Permission denied (publickey).
if it wasn't running then it would have straight out refused the connection right?
I may try another container like you suggest, but I think I have something wrong in the traefik config so I'm sure it'll break in the same way.
So, your question got my thinking about have I checked ssh working within the container.
This led me to try connecting locally on the host to the container through port 222, so:
ssh -v -p 222 localhost
This then works! With this outpit:
ssh -p 222 localhost
The authenticity of host '[localhost]:222 ([127.0.0.1]:222)' can't be established.
ED25519 key fingerprint is *
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Failed to add the host to the list of known hosts (/home/pi/.ssh/known_hosts).
pi@localhost: Permission denied (publickey).
With the gitea container outputting this:
gitea | Invalid user pi from 172.18.0.2 port 34692
gitea | Connection closed by invalid user pi 172.18.0.2 port 34692 [preauth]
So this means it is working, traefik is working and is setup correctly. It's the connection from outside my pi to port 222 that isn't working for some reason, I've made sure that port 222 is allowed through UFW:
222/tcp ALLOW Anywhere
222/tcp (v6) ALLOW Anywhere (v6)
There doesn't seem anything in /var/log/ufw.log to show that a connection has been blocked from outside.
Is a simple ssh working with Traefik and just git is failing?
it's not git specifically but ssh from another system to my pi (running traefik and gitea).
So this doesn't work either:
ssh -v -p 222 gitea.domain.xyz
I fear this may be either a networking issue now or a gitea issue, I think my traefik setup is working just fine at forwarding the SSH traffic, as shown by doing ssh -v -p 222 localhost on the pi and it working (well denying access, but that shows connectivity).
yea this is some sort of networking thing that I don't understand, and I now realise this as I ssh to my pi by doing ssh pi@hostname.local rather than ssh pi@domain.xyz so it's going locally.
I found this:
nmap -p 22,222 domain.xyz
PORT STATE SERVICE
22/tcp filtered ssh
222/tcp filtered rsh-spx
nmap -p 22,222 hostname.local
PORT STATE SERVICE
22/tcp open ssh
222/tcp open rsh-spx
So doing:
git clone ssh://git@hostname.local:222/user/TestRepo.git
Works just fine! I don't really understand why hostname.local works though. I guess it's blocking non-local traffic, I may ask on the gitea forum, that is probably a better place for this now that I've found that my traefik config works just fine.
Example of a working SSH server behind Traefik proxy server.
Not very elegant, the SSH-server updates and installs on every restart.
docker-compose.yml with Traefik, web-server and ssh-server:
version: '3.9'
services:
traefik:
image: traefik:v2.9.6
ports:
- target: 80
published: 80
protocol: tcp
mode: host
- target: 443
published: 443
protocol: tcp
mode: host
- target: 9000
published: 9000
protocol: tcp
mode: host
networks:
- proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- traefik-certificates:/certificates
command:
- --providers.docker=true
- --providers.docker.network=proxy
- --providers.docker.exposedByDefault=false
- --entryPoints.web.address=:80
- --entryPoints.web.http.redirections.entryPoint.to=websecure
- --entryPoints.web.http.redirections.entryPoint.scheme=https
- --entryPoints.websecure.address=:443
- --entryPoints.websecure.http.tls=true
- --entryPoints.websecure.http.tls.certResolver=myresolver
- --entryPoints.tcp9000.address=:9000
- --api.debug=true
- --api.dashboard=true
- --log.level=DEBUG
- --accesslog=true
- --certificatesResolvers.myresolver.acme.email=mail@example.com
- --certificatesResolvers.myresolver.acme.storage=/certificates/acme.json
- --certificatesresolvers.myresolver.acme.tlschallenge=true
labels:
- traefik.enable=true
- traefik.http.routers.dashboard.entrypoints=websecure
- traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)
- traefik.http.routers.dashboard.tls.certresolver=myresolver
- traefik.http.routers.dashboard.service=api@internal
- traefik.http.routers.dashboard.middlewares=myauth
- 'traefik.http.middlewares.myauth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/'
whoami:
image: traefik/whoami:v1.8
networks:
- proxy
labels:
- traefik.enable=true
- traefik.http.routers.whoami.entrypoints=websecure
- traefik.http.routers.whoami.rule=Host(`whoami.example.com`)
- traefik.http.services.whoami.loadbalancer.server.port=80
debian:
image: debian:stable
networks:
- proxy
command: bash -c "apt update && apt install -y openssh-server sudo net-tools && service ssh start && while true; do sleep 1; done"
labels:
- traefik.enable=true
- traefik.tcp.routers.debian.entrypoints=tcp9000
- traefik.tcp.routers.debian.rule=HostSNI(`*`)
- traefik.tcp.services.debian.loadbalancer.server.port=22
networks:
proxy:
name: proxy
volumes:
traefik-certificates:
SSH command:
ssh -v -p 9000 root@<IP-or-FQDN>
Thanks for your very comprehensive example.
I realised I've been a dummy and I didn't forward port 222 through my router to my pi, which is why HTTPS checkouts on the domain name were working but SSH on the domain wasn't working (but was working while using hostname.local).
At least I learnt a bunch of stuff while bumbling through this. Thankyou for all your input here, your questioning really helped me along the way 