[Plugin CrowdSec 1.3.0] Unable to download error 500

Traefik Traefik v2
,
1

Hi,
I'm trying to get the crowdsec middleware plugin to work on my Traefik 2.11.2 installation.
Already when defining the plugin in my static configuration and restarting Traefik as such:

experimental:
  plugins:        
    crowdsec-bouncer-traefik-plugin:
      moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
      version: "1.3.0"

I get an error in my traefik log:

unable to download plugin github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin: error : 500, Failed to get plugin github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin@v1.3.0

This is even without using the actual plugin in any dynamic configuration. When I try that, the dynamic configuration fails and goes red in the dashboard:

invalid middleware "crowdsec-media@file" configuration: invalid middleware type or middleware does not exist

The dynamic config looks as follows:

http:
  routers:
    media:
      entryPoints:
      - web
      middlewares:
      - crowdsec-media@file
      service: media
      rule: Host(`media.domain.com`)
  middlewares:
    crowdsec-media:
      plugin:
        crowdsec-bouncer-traefik-plugin:
          crowdsecLapiKey: <redacted>
          enabled: false
          logLevel: DEBUG

services:
    media:
      loadBalancer:
        healthCheck:
          path: /
          interval: 10s
          timeout: 3s
        servers:
        - url: http://backend.domain.local:8800/

I understood from previous posts that the plugin download system can getcongested sometimes? Is this the case? I have tried multiple times, but no luck.
The used docker-compose looks as follows:

version: '3'

services:
  traefik:
    # The official v2 Traefik docker image
    image: traefik:v2.11.2
    container_name: traefik
    restart: unless-stopped

    # Enables the web UI and tells Traefik to listen to docker
    command:

    network_mode: "host"
    environment:
      - TZ=<redacted>
      - EXEC=1
    volumes:
      # So that Traefik can listen to the Docker events
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/dockeruser/traefik/data/traefik.yml:/traefik.yml:ro
      - /home/dockeruser/traefik/data/certs/domain.cert:/domain.cert:ro
      - /home/dockeruser/traefik/data/certs/domain.key:/domain.key:ro
      - /home/dockeruser/traefik/data/acme.json:/acme.json
      - /home/dockeruser/traefik/data/config.yml:/config.yml:ro
      - /home/dockeruser/traefik/data/logs:/var/log/traefik
      - /home/dockeruser/traefik/data/dynamic_configs:/dynamic_configs
      - /home/dockeruser/traefik/data/plugins-storage:/plugins-storage
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik.rule=Host(`traefik.de-baay.nl`)"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.service=api@internal"
      - "traefik.http.services.justAdummyService.loadbalancer.server.port=1337"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.de-baay.nl`)"      
      - "traefik.http.middlewares.traefik-auth.basicauth.users=dockeruser:<redacted>"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https" 
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
  crowdsec:
    image: crowdsecurity/crowdsec:v1.6.0
    container_name: crowdsec
    environment:
      PGID: "1000"
      COLLECTIONS: "crowdsecurity/traefik crowdsecurity/http-cve"
    network_mode: "host"
    expose:
      - "8080"
    volumes:
      - /home/dockeruser/crowdsec:/var/log/crowdsec:ro
      - /home/dockeruser/crowdsec/db:/var/lib/crowdsec/data
      - /home/dockeruser/crowdsec/log/auth.log:/var/log/auth.log:ro
      - /home/dockeruser/crowdsec/config:/etc/crowdsec
      - /home/dockeruser/traefik/data/logs:/var/log/traefik
    restart: unless-stopped
    labels:
      - traefik.enable=false

Does anyone have an idea where this is going wrong?
I tried with a number of previous crowdsec plugin versions, but no dice...

2

I don’t think it’s best practice to run (multiple) containers in network_mode: host. Use a Docker network instead.

3

I wonder whether that is the problem. I have actually tried both cases.
The thing is, the least I need to run the Traefik container in host mode, as I need to map multiple physical addresses to Traefik to be able to use TCP Routes (behind which I host a number of external webapps that use Windows authentication, external to Docker, that is).
I have had the crowdsec container in it's own docker container, but that resulted in the exact same error.

In any case, my suspicion is that the error shown has more to do with downloading the plugin from Github than my actual docker configuration. :thinking:

4

Works for me, I followed roughly the doc and now get an error:

level=error msg="CrowdsecLapiKey || (CrowdsecLapiTLSCertificateBouncer && CrowdsecLapiTLSCertificateBouncerKey): cannot be all empty"

So it seems the plugin is loaded and running.

experimental:
  plugins:
    bouncer:
      moduleName: github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
      version: v1.3.0

  whoami:
    image: traefik/whoami:v1.10
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.whoami.entrypoints=websecure
      - traefik.http.routers.whoami.rule=Host(`whoami.example.com`) || PathPrefix(`/whoami`)
      - traefik.http.services.whoami.loadbalancer.server.port=80
      - traefik.http.middlewares.crowdsec.plugin.bouncer.enabled=false
      - traefik.http.middlewares.crowdsec.plugin.bouncer.logLevel=DEBUG
      - traefik.http.routers.whoami.middlewares=crowdsec
1 Like
5

Just to make sure, you are not using the Traefik in a host network configuration?
Would you be willing to test that?

6

My "working" configuration.

./docker-compose.yml:

services:
  traefik:
    image: traefik:v2.11
    command: --configFile=/config/traefik.yml
    ports:
      - published: 80
        target: 80
        protocol: tcp
        mode: host
      - published: 443
        target: 443
        protocol: tcp
        mode: host
    networks:
      - proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /var/log:/var/log
      - ./config:/config

  whoami:
    image: traefik/whoami:v1.10
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.whoami.entrypoints=websecure
      - traefik.http.routers.whoami.rule=Host(`whoami.example.com`) || PathPrefix(`/whoami`)
      - traefik.http.services.whoami.loadbalancer.server.port=80
      - traefik.http.middlewares.crowdsec.plugin.bouncer.enabled=false
      - traefik.http.middlewares.crowdsec.plugin.bouncer.logLevel=DEBUG
      - traefik.http.routers.whoami.middlewares=crowdsec

  crowdsec:
    image: crowdsecurity/crowdsec
    container_name: crowdsec
    environment:
      PGID: "1000"
      COLLECTIONS: "crowdsecurity/traefik crowdsecurity/http-cve"
    expose:
      - "8080"
    volumes:
      - /var/log/crowdsec:/var/log/crowdsec:ro
      - .config/crowdsec-db:/var/lib/crowdsec/data
      - /var/log/auth.log:/var/log/auth.log:ro
      - ./config/crowdsec:/etc/crowdsec
    restart: unless-stopped
    labels:
      - traefik.enable=false
    networks:
      - proxy

networks:
  proxy:
    name: proxy
    #driver: overlay
    #attachable: true
    external: true

./config/traefik.yml:

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    network: proxy
  file:
    directory: /config
    watch: true

entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: :443
    http:
      tls:
        certResolver: myresolver
  traefik:
    address: :8080

api:
  dashboard: true
  debug: false
  insecure: false

log:
  level: DEBUG

accessLog: {}

certificatesResolvers:
  myresolver:
    acme:
      email: mail@example.com
      storage: /config/acme.json
      tlsChallenge: {}

experimental:
  plugins:
    bouncer:
      moduleName: github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
      version: v1.3.0
7

Let me see if I can spin this up on another host. The crappy thing is, that I have configured it all on an Ubuntu host, which turns out to run snap (!) docker... :expressionless:

-EDIT 1 (there will be more :smiley: )-
Set it up on another physical host using non-snap docker and that fails as well, exact same error as described.
Now I do see that you are using a slightly different network config than I am. I'll see whether I can emulate that and then test.

-EDIT 2-
@bluepuma77, many thanks for your help, but even when emulating the configuration that you are proposing, I am still getting the error 500 when trying to download the plugin. For the life of me I cannot imagine why...

time="2024-05-16T21:42:19+02:00" level=debug msg="loading of plugin: bouncer: github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin@1.3.0"
time="2024-05-16T21:42:20+02:00" level=error msg="Plugins are disabled because an error has occurred." error="unable to set up plugins environment: unable to download plugin github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin: error: 500: {\"error\":\"Failed to get plugin github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin@1.3.0\"}\n"
8

Running Debian and installed Docker via official helper script.

9

Well, I'll look into that. Means that I'll have to deploy a whole new vm running Debian. That'll be for the week-end. It sure is finicky.
Are there Traefik staff or mods around to shed some light in this error 500?

10

It's because something blocks the call inside your local network.

11

@ldez, any common issues that I might look into? Can you give a n example of the exact url that is being tried to call? I have Pi-Hole running, might that be an issue?

-EDIT-
Just to be sure, disabled Pi-hole, but no dice...

12

Go into your Traefik container:

docker exec -it <c-id> sh

and try to fetch something manually:

wget https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin

Then start debugging why your network is not enabling that request. Can the IP be resolved (DNS), can the target IP be reached (firewall), etc.

This seems more like a Docker networking problem than a Traefik issue.

1 Like
13

'course... could have thought about that myself... grmbl...

I'll check..
@bluepuma77, works in the Traefik container, no problem.
What is the call used to retrieve the plugin via the traefik plugin directory? Because I believe the call is being directed there and I suspect an error over there. I have been browsing the traefik code to see whether I can find the URL being used, but wasn't able to find it.
-EDIT-
Also tried

wget https://plugins.traefik.io/... etc.

And this works in the traefik container. Tried to wget to several possible folders I would suspect the plugin to be downloaded to, and calls complete fine. I can cat the files being created and there is content. So, no reason why the Traefik application should fail. @ldez , any ideas?

14

I'm thinking about a hidden character, a typo inside your configuration, or a file encoding issue: something that impacts the name or the version of the plugin used to perform the call.

15

Your problem is here: the version should be v1.3.0 and not 1.3.0.

$ wget https://plugins.traefik.io/public/download/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/1.3.0  
--2024-05-17 14:07:27--  https://plugins.traefik.io/public/download/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/1.3.0
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving plugins.traefik.io (plugins.traefik.io)... 2606:4700:20::681a:265, 2606:4700:20::ac43:4b08, 2606:4700:20::681a:365, ...
Connecting to plugins.traefik.io (plugins.traefik.io)|2606:4700:20::681a:265|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2024-05-17 14:07:28 ERROR 500: Internal Server Error.

$ wget https://plugins.traefik.io/public/download/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/v1.3.0                  
--2024-05-17 14:07:18--  https://plugins.traefik.io/public/download/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/v1.3.0
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving plugins.traefik.io (plugins.traefik.io)... 2606:4700:20::ac43:4b08, 2606:4700:20::681a:365, 2606:4700:20::681a:265, ...
Connecting to plugins.traefik.io (plugins.traefik.io)|2606:4700:20::ac43:4b08|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘v1.3.0’

v1.3.0                                                                      [ <=>                                                                                                                                                                         ] 376,80K  --.-KB/s    in 0,05s   

2024-05-17 14:07:18 (6,81 MB/s) - ‘v1.3.0’ saved [385845]

The right configuration:

experimental:
  plugins:        
    crowdsec-bouncer-traefik-plugin:
      moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
      version: "v1.3.0"
1 Like
16

@ldez , @bluepuma77 , that was it! Both a lot of thanks for your time and help. I was about to give up :wink:
I am a notoriously bad typist and my colleague would laugh his ass off... :blush:
Solved for now!

1 Like
17

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.