Routing Bytebase gets me 404 Error Page Not Found

Traefik Traefik v2
,
1

Hey guys, I just wanted some help (if possible) or any directions on what I may be doing wrong. I'm testing Traefik as a reverse proxy with Docker. So far, I haven't had any trouble with any container that I want to expose to x.mydomain.com. However, for some reason, with a specific container (Bytebase), I can't seem to get it to be exposed. I've thought that it may have something to do with the tcp6 port that's exposed inside the container or maybe the way that Bytebase creates the webserver that Traefik may not be picking up. I have checked the docs on Traefik, Docker, and Bytebase and can't seem to find any solution or explanation for why this could be happening.

So, in short, I'd like (if possible) for you to check the configs and logs that I'm getting so far. Maybe I'm not seeing something that is clear to other people. Thanks in advance, and sorry for the bother.

My traefik docker-compose.yml:

version: "3"
networks:
  default:
    name: production
    external: true
services:
  traefik:
    image: "traefik"
    container_name: "traefik"
    security_opt:
      - no-new-privileges:true
    ports:
      - "80:80"
      - "443:443"
    environment:
      - CF_API_EMAIL=${CF_API_EMAIL}
      - CF_API_KEY=${CF_API_EMAIL}
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik-data/traefik.yml:/traefik.yml:ro
      - ./traefik-data/acme.json:/acme.json
      - ./traefik-data/config.yml:/config.yml:ro
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.mydomain.com`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.users=user:password"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.mydomain.com`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik-secure.tls.domains[0].main=mydomain.com"
      - "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.mydomain.com"
      - "traefik.http.routers.traefik-secure.service=api@internal"
      - "traefik.http.routers.traefik-secure.middlewares=authelia@docker"
    restart: unless-stopped

My traefik.yml:

api:
  dashboard: true
  debug: true
entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https
  https:
    address: ":443"
serversTransport:
  insecureSkipVerify: true
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    useBindPortIP: true
  file:
    filename: /config.yml
certificatesResolvers:
  cloudflare:
    acme:
      email: user@domaincom
      storage: acme.json
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

My bytebase docker-compose.yml:

version: '3'
networks:
  default:
    name: production
    external: true
services:
  bytebase:
    container_name: bytebase
    image: bytebase/bytebase:2.7.0
    restart: unless-stopped
    volumes:
      - ./bytebase-data:/var/opt/bytebase
    command: --data /var/opt/bytebase --pg ${BB_PG} --external-url ${BB_EXTERNAL_URL} --port 8087 --disable-sample --disable-metric --debug
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
      - "traefik.enable=true"
      - "traefik.http.routers.bytebase.entrypoints=http"
      - "traefik.http.routers.bytebase.rule=Host(`bytebase.mydomain.com`)"
      - "traefik.http.middlewares.bytebase-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.bytebase.middlewares=bytebase-https-redirect"
      - "traefik.http.routers.bytebase-secure.entrypoints=https"
      - "traefik.http.routers.bytebase-secure.rule=Host(`bytebase.mydomain.com`)"
      - "traefik.http.routers.bytebase-secure.tls=true"
      - "traefik.http.routers.bytebase-secure.service=bytebase"
      - "traefik.http.services.bytebase.loadbalancer.server.port=8087"
      - "traefik.docker.network=production@docker"
      - "traefik.http.routers.bytebase-secure.middlewares=authelia@docker"

In traefik logs, all seems to be fine:

time="2023-09-05T17:15:49+02:00" level=info msg="Configuration loaded from file: /traefik.yml"

Also, in bytebase logs, there's nothing failing:

___________________________________________________________________________________________
██████╗ ██╗   ██╗████████╗███████╗██████╗  █████╗ ███████╗███████╗
██╔══██╗╚██╗ ██╔╝╚══██╔══╝██╔════╝██╔══██╗██╔══██╗██╔════╝██╔════╝
██████╔╝ ╚████╔╝    ██║   █████╗  ██████╔╝███████║███████╗█████╗
██╔══██╗  ╚██╔╝     ██║   ██╔══╝  ██╔══██╗██╔══██║╚════██║██╔══╝
██████╔╝   ██║      ██║   ███████╗██████╔╝██║  ██║███████║███████╗
╚═════╝    ╚═╝      ╚═╝   ╚══════╝╚═════╝ ╚═╝  ╚═╝╚══════╝╚══════╝
Version 2.7.0 (schema version 2.7.8) has started on port 8087 🚀
___________________________________________________________________________________________
2023-09-05T15:20:25.222Z	DEBUG	backuprun/runner.go:63	Auto backup runner started	{"interval": "10m0s"}
2023-09-05T15:20:25.222Z	DEBUG	taskrun/schedulerv2.go:60	Task scheduler V2 started and will run every 1s
2023-09-05T15:20:25.222Z	INFO	taskrun/schedulerv2.go:507	TaskSkippedOrDoneListener started
2023-09-05T15:20:25.222Z	DEBUG	anomaly/scanner.go:50	Anomaly scanner started and will run every 10m0s
2023-09-05T15:20:25.222Z	DEBUG	taskcheck/scheduler.go:49	Task check scheduler started and will run every 1s
2023-09-05T15:20:25.222Z	DEBUG	apprun/runner.go:56	Application runner started and will run every 30s
2023-09-05T15:20:25.222Z	DEBUG	schemasync/syncer.go:55	Schema syncer started and will run every 30m0s
2023-09-05T15:20:25.222Z	DEBUG	slowquerysync/syncer.go:55	Slow query syncer started and will run every 12h0m0s
2023-09-05T15:20:25.222Z	DEBUG	mail/slow_query_weekly_mail.go:66	Slow query weekly mail sender started
2023-09-05T15:20:25.222Z	DEBUG	relay/runner.go:69	Relay runner started and will run every 10m0s
2023-09-05T15:20:25.222Z	DEBUG	approval/runner.go:70	Approval runner started and will run every 1s
2023-09-05T15:20:25.222Z	DEBUG	metricreport/reporter.go:82	Metrics reporter started and will run every 1h0m0s
2023-09-05T15:20:25.222Z	DEBUG	plancheck/scheduler.go:48	Plan check scheduler started and will run every 1s

Doing curl inside the container i get a response:

root@fb60d7f7d8cc:/# curl 127.0.0.1:8087/healthz
OK!

Also checking ports exposed inside the container i get this:

root@fb60d7f7d8cc:/# netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.11:43639        0.0.0.0:*               LISTEN     
tcp6       0      0 :::8088                 :::*                    LISTEN     
tcp6       0      0 :::8087                 :::*                    LISTEN     
udp        0      0 127.0.0.11:42281        0.0.0.0:*
2

Try without "@docker“.

You have a http->https redirect on entrypoint, don’t need it again on labels.

What is Traefik dashboard and Traefik debug log telling you?

Check simple Traefik example.

3

Hey! Thx for the response.

  1. I already tried without the @docker, i get the same issue
  2. Thx for the correction, here's the new labels:
labels:
      - "com.centurylinklabs.watchtower.enable=true"
      - "traefik.enable=true"
      # - "traefik.http.routers.bytebase.entrypoints=http"
      - "traefik.http.routers.bytebase.rule=Host(`bytebase.mydomain.com`)"
      - "traefik.http.middlewares.bytebase-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.bytebase.middlewares=bytebase-https-redirect"
      - "traefik.http.routers.bytebase-secure.entrypoints=https"
      - "traefik.http.routers.bytebase-secure.rule=Host(`bytebase.mydomain.com`)"
      - "traefik.http.routers.bytebase-secure.tls=true"
      - "traefik.http.routers.bytebase-secure.service=bytebase"
      - "traefik.http.services.bytebase.loadbalancer.server.port=8087"
      - "traefik.docker.network=production"
      - "traefik.http.routers.bytebase-secure.middlewares=authelia@docker"
  1. Traefik debug logs only shows this:
time="2023-09-05T17:15:49+02:00" level=info msg="Configuration loaded from file: /traefik.yml"

Traefik dashboard doesn't show the router for some reason:

imagen
imagen1453×642 36.1 KB

That's why i think that for some reason traefik is not picking up the info about Bytebase container.

4

Suggestion:

Enable TLS on https entrypoint.

5

Tried that too, same result :confused:

6

Re-read your post, it seems bytebase is only using IPv6, not sure if that works with Traefik Docker Configuration Discovery.

Did you enable IPv6 in Docker itself?

Can you enable IPv4 with bytebase?

7

yup, I thought that too. I was hoping to see if someone here has had any similar issue with a container that uses TCP6 ports internally.

No, I didn't enable IPv6 because I intend to expose the service through IPv4. From what I read about enabling IPv6 in Docker, I would need to enable IPv6 if I intend to expose or use an IPv6 host. However, I'm not sure about that. Also i don't how to manage IPv6 networks...

Also, when using Bytebase normally in Docker (without Traefik), the ports get exposed normally through 0.0.0.0.

Regarding enabling IPv4, their documentation doesn't have any references about that so far, so nope...

I made this post to see if anybody else had a similar problem with a container that has similarities with the one of Bytebase (ports on TCP6, uses a Go web server).

8

Hey, how about this:

What if i expose the trafic of my docker container to localhost like so:

version: '3'
networks:
  default:
    name: production
    external: true
services:
  bytebase:
    container_name: bytebase
    image: bytebase/bytebase:2.7.0
    restart: unless-stopped
    ports:
      - 127.0.0.1:8087:8087
    volumes:
      - ./bytebase-data:/var/opt/bytebase
    command: --data /var/opt/bytebase --pg ${BB_PG} --external-url ${BB_EXTERNAL_URL} --port 8087 --disable-sample --disable-metric --debug
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
      # - "traefik.enable=true"
      # - "traefik.http.routers.bytebase.entrypoints=http"
      # - "traefik.http.routers.bytebase.rule=Host(`bytebase.mydomain.com`)"
      # - "traefik.http.middlewares.bytebase-https-redirect.redirectscheme.scheme=https"
      # - "traefik.http.routers.bytebase.middlewares=bytebase-https-redirect"
      # - "traefik.http.routers.bytebase-secure.entrypoints=https"
      # - "traefik.http.routers.bytebase-secure.rule=Host(`bytebase.mydomain.com`)"
      # - "traefik.http.routers.bytebase-secure.tls=true"
      # - "traefik.http.routers.bytebase-secure.service=bytebase"
      # - "traefik.http.services.bytebase.loadbalancer.server.port=8087"
      # - "traefik.docker.network=production"
      # - "traefik.http.routers.bytebase-secure.middlewares=authelia@docker"

Then in my config.yml for traefik i use this:

http:
  routers:
    bytebase:
      entryPoints:
        - "https"
      rule: "Host(`bytebase.mydomain.com`)"
      middlewares:
        - authelia
        - default-headers
      tls: {}
      service: bytebase
  services:
    bytebase:
      loadBalancer:
        servers:
          - url: "http://127.0.0.1:8087"
        passHostHeader: true

  middlewares:
    https-redirect:
      redirectScheme:
        scheme: https
    authelia:
      forwardAuth:
        address: "http://authelia:9091/api/verify?rd=https://auth.mydomain.com"
    default-headers:
      headers:
        frameDeny: true
        sslRedirect: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https

    default-whitelist:
      ipWhiteList:
        sourceRange:
          - "10.0.0.0/8"
          - "192.168.0.0/16"
          - "172.0.0.0/12"

    secured:
      chain:
        middlewares:
          - default-whitelist
          - default-headers

Tho, so far trying this i get 403 Forbidden, dunno if this could be a workaround, what do you think?

9

It’s strange, in the bytebase doc seems to be no hint about IPv6.

The Dockerfile has no expose, so you definitely need to tell Traefik which port to use, it seems 80 is the default inside the bytebase container.

10

Got plain Bytebase running in Docker Swarm. Of course you can`t run it like that in production because of the bind mount.

Using default port 80, enabling init, disabling healthcheck.

Init might be important for the whole app to start, healthcheck never turned "green", so I had to disable it (its also defined in Dockerfile) for Traefik to pick it up.

version: '3.9'

networks:
  proxy:
    name: proxy
    external: true

services:
  bytebase:
    image: bytebase/bytebase:2.7.0
    init: true
    healthcheck:
      disable: true
      #  test: ["CMD", "curl", "--fail", "http://localhost:80/healthz", "||", "exit", "1"]
      #  interval: 5m
      #  timeout: 60s
    volumes:
      - ~/.bytebase/data:/var/opt/bytebase
    command:
      - "--data"
      - "/var/opt/bytebase"
    networks:
      - proxy
    deploy:
      mode: global
      labels:
        - traefik.enable=true
        - traefik.http.routers.bytebase.entrypoints=websecure
        - traefik.http.routers.bytebase.rule=Host(`bytebase.example.com`)
        - traefik.http.services.bytebase.loadbalancer.server.port=80
1 Like
11

It worked! also i was getting 403 Forbidden, but that was cause i didn't add the access control rule for bytebase on the authelia config.yml

So in short, the healthcheck seems to be the issue.

12

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.