Traefik is not applying config from http provider

Evolutio · November 26, 2022, 3:25pm

I created an API where traefik should get the configuration from.

Traefik config:

log:
  level: DEBUG

api:
  dashboard: true
  insecure: true

entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"

providers:
  docker: false
  http:
    pollInterval: "5s"
    pollTimeout: "5s"
    endpoint: "http://config/api/v1/server?api_token=cabea4c73c8c883a6a7eeb27193ba27b3d42c2d5905c5680e9149a92568020f1"

The response from my api is:

{
  "http": {
    "routers": {
      "whoami": {
        "rule": "Host(`whoami.local`)",
        "service": "whoami"
      }
    },
    "services": {
      "whoami": {
        "loadbalancer": {
          "servers": [
            {
              "url": "http://whoami/"
            }
          ]
        }
      }
    }
  }
}

I tested it with returning json and the json as download, but both aren't working and traefik don't log anything what could be wrong.
In my API-logs I see that traefik makes every 5 seconds (as configured) an request for the given url. So what could be my mistake?
When I curl the endpoint from the traefik-container I get the json-response, so the endpoint is reachable within the container.

bluepuma77 · November 26, 2022, 3:45pm

Is Traefik reading your traefik config? Did you check the Traefik log for "error"?

The HTTP provider docs state

The HTTP provider uses the same configuration as the File Provider in YAML or JSON format

But the File Provider docs state

The file provider lets you define the dynamic configuration in a YAML or TOML file.

I don't think Traefik will accept a JSON config. Try YAML instead, that worked for me.

Evolutio · November 26, 2022, 4:45pm

thanks for reply!

There is no error from Traefik.

hmm, I tried with YAML too, but it didn't worked. Below generated YAML:

http:
  routers:
    whoami:
      rule: Host(`whoami.local`)
      service: whoami
  services:
    whoami:
      loadbalancer:
        servers:
          - url: 'http://whoami/'

bluepuma77 · November 26, 2022, 5:32pm

Is Traefik reading your traefik config? Do you see any debug level output?

Traefik dashboard in insecure mode should automatically open port 8080. If using Docker, expose 8080. Then check http://ip:8080/dashboard/.

Evolutio · November 27, 2022, 8:59am

yeah. Traefik is reading my config. If I change the log level there are not as much logs as on DEBUG.
If I switch to ERROR, I don't get any error in my logs.

Yeah, the traefik-dashboard is reachable.
As you can see below the dashboard don't list the router/service from the yaml.

As you can see here, it calls my API every 5seconds:

Evolutio · November 27, 2022, 9:19am

shame over me!

@bluepuma77 My mistake was a typo in the url which get's called. But it needs to be yaml as you say.

cakiwi · November 27, 2022, 3:32pm

JSON does work.

docker-compose.yaml:

version: '3'
services:
  traefik:
    image: traefik:2.9
    ports:
      - '80:80'
    command:
      - --entrypoints.http.address=:80
      - --log.level=DEBUG
      - --providers.http.endpoint=http://config/config
  whoami: 
    image: traefik/whoami
  config:
    image: nginx
    volumes:
      - ./web:/usr/share/nginx/html

web/config:

{
  "http": {
    "routers": {
      "whoami": {
        "rule": "PathPrefix(`/`)",
        "service": "whoami"
      }
    },
    "services": {
      "whoami": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://whoami"
            }
          ]
        }
      }
    }
  }
}

bluepuma77 · November 27, 2022, 7:56pm

@cakiwi Wow. Can Traefik and File Provider read JSON?

ldez · November 28, 2022, 3:37am

The JSON syntax "follows" the specifications of YAML.

The file extensions allowed for the file provider are: .toml, .yaml, and .yml.

So Traefik can understand JSON because it's a kind of YAML subset if you write the JSON content inside a YAML file (.yaml or .yml).

It's also why the HTTP provider supports YAML and JSON, it's because we are using a YAML parser to handle payloads.

In summary: JSON is YAML, but YAML is not JSON.

Evolutio · November 28, 2022, 6:49am

@cakiwi
if I use JSON on the endpoint, traefik will log me the following:

traefik_1  | time="2022-11-28T06:44:01Z" level=info msg="Configuration loaded from file: /traefik.yml"
traefik_1  | time="2022-11-28T06:44:01Z" level=error msg="Provider connection error cannot decode configuration data: yaml: found unknown escape character, retrying in 366.364934ms" providerName=http

what get's returned:

{
  "http": {
    "routers": {
      "whoami": {
        "rule": "Host(`whoami.local`)",
        "service": "whoami"
      }
    },
    "services": {
      "whoami": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://whoami/"
            },
            {
              "url": "http://whoami/2"
            }
          ]
        }
      }
    }
  }
}

ldez · November 28, 2022, 8:16am

I tried your configuration:

with HTTP provider:

DEBU[2022-11-28T09:20:23+01:00] Static configuration loaded {"global":{"checkNewVersion":true},"serversTransport":{"maxIdleConnsPerHost":200},"entryPoints":{"web":{"address":":8081","transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s"}},"forwardedHeaders":{},"http":{},"http2":{"maxConcurrentStreams":250},"udp":{"timeout":"3s"}}},"providers":{"providersThrottleDuration":"2s","docker":{"watch":true,"endpoint":"unix:///var/run/docker.sock","defaultRule":"Host(`{{ normalize .Name }}`)","exposedByDefault":true,"swarmModeRefreshSeconds":"15s"},"http":{"endpoint":"http://localhost:9090","pollInterval":"5s","pollTimeout":"5s"}},"log":{"level":"debug","format":"common"}} 
INFO[2022-11-28T09:20:23+01:00] 
Stats collection is disabled.
Help us improve Traefik by turning this feature on :)
More details on: https://doc.traefik.io/traefik/contributing/data-collection/ 
INFO[2022-11-28T09:20:23+01:00] Starting provider aggregator aggregator.ProviderAggregator 
DEBU[2022-11-28T09:20:23+01:00] Starting TCP Server                           entryPointName=web
INFO[2022-11-28T09:20:23+01:00] Starting provider *http.Provider             
DEBU[2022-11-28T09:20:23+01:00] *http.Provider provider configuration: {"endpoint":"http://localhost:9090","pollInterval":"5s","pollTimeout":"5s"} 
INFO[2022-11-28T09:20:23+01:00] Starting provider *docker.Provider           
DEBU[2022-11-28T09:20:23+01:00] *docker.Provider provider configuration: {"watch":true,"endpoint":"unix:///var/run/docker.sock","defaultRule":"Host(`{{ normalize .Name }}`)","exposedByDefault":true,"swarmModeRefreshSeconds":"15s"} 
INFO[2022-11-28T09:20:23+01:00] Starting provider *traefik.Provider          
DEBU[2022-11-28T09:20:23+01:00] *traefik.Provider provider configuration: {} 
INFO[2022-11-28T09:20:23+01:00] Starting provider *acme.ChallengeTLSALPN     
DEBU[2022-11-28T09:20:23+01:00] *acme.ChallengeTLSALPN provider configuration: {} 
DEBU[2022-11-28T09:20:23+01:00] Configuration received: {"http":{"services":{"noop":{}},"serversTransports":{"default":{"maxIdleConnsPerHost":200}}},"tcp":{},"udp":{},"tls":{}}  providerName=internal
DEBU[2022-11-28T09:20:23+01:00] Configuration received: {"http":{"routers":{"whoami":{"service":"whoami","rule":"Host(`whoami.local`)"}},"services":{"whoami":{"loadBalancer":{"servers":[{"url":"http://whoami/"},{"url":"http://whoami/2"}],"passHostHeader":true}}}},"tcp":{},"udp":{},"tls":{}}  providerName=http

With the file provider:

DEBU[2022-11-28T09:12:08+01:00] Static configuration loaded {"global":{"checkNewVersion":true},"serversTransport":{"maxIdleConnsPerHost":200},"entryPoints":{"web":{"address":":8081","transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s"}},"forwardedHeaders":{},"http":{},"http2":{"maxConcurrentStreams":250},"udp":{"timeout":"3s"}}},"providers":{"providersThrottleDuration":"2s","docker":{"watch":true,"endpoint":"unix:///var/run/docker.sock","defaultRule":"Host(`{{ normalize .Name }}`)","exposedByDefault":true,"swarmModeRefreshSeconds":"15s"},"file":{"directory":"./dyn/","watch":true}},"log":{"level":"debug","format":"common"}} 
INFO[2022-11-28T09:12:08+01:00] 
Stats collection is disabled.
Help us improve Traefik by turning this feature on :)
More details on: https://doc.traefik.io/traefik/contributing/data-collection/ 
INFO[2022-11-28T09:12:08+01:00] Starting provider aggregator aggregator.ProviderAggregator 
DEBU[2022-11-28T09:12:08+01:00] Starting TCP Server                           entryPointName=web
INFO[2022-11-28T09:12:08+01:00] Starting provider *file.Provider             
DEBU[2022-11-28T09:12:08+01:00] *file.Provider provider configuration: {"directory":"./dyn/","watch":true} 
INFO[2022-11-28T09:12:08+01:00] Starting provider *traefik.Provider          
DEBU[2022-11-28T09:12:08+01:00] *traefik.Provider provider configuration: {} 
INFO[2022-11-28T09:12:08+01:00] Starting provider *acme.ChallengeTLSALPN     
DEBU[2022-11-28T09:12:08+01:00] *acme.ChallengeTLSALPN provider configuration: {} 
INFO[2022-11-28T09:12:08+01:00] Starting provider *docker.Provider           
DEBU[2022-11-28T09:12:08+01:00] *docker.Provider provider configuration: {"watch":true,"endpoint":"unix:///var/run/docker.sock","defaultRule":"Host(`{{ normalize .Name }}`)","exposedByDefault":true,"swarmModeRefreshSeconds":"15s"} 
DEBU[2022-11-28T09:12:08+01:00] Configuration received: {"http":{"routers":{"whoami":{"service":"whoami","rule":"Host(`whoami.local`)"}},"services":{"whoami":{"loadBalancer":{"servers":[{"url":"http://whoami/"},{"url":"http://whoami/2"}],"passHostHeader":true}}}},"tcp":{},"udp":{},"tls":{}}  providerName=file

there is no parsing problem (take a look at "Configuration received:" lines).

So I think you have an invalid character in your real configuration.

Evolutio · November 29, 2022, 6:17pm

after some testing I figured this out:

it should not have \ in the response to escape /

so this response don't work:

{
  "http": {
    "routers": {
      "whoami": {
        "rule": "PathPrefix(`\/`)",
        "service": "whoami"
      }
    },
    "services": {
      "whoami": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http:\/\/whoami\/"
            },
            {
              "url": "http:\/\/whoami\/2"
            }
          ]
        }
      }
    }
  }
}

sosoba · November 30, 2022, 5:09am

Are you sending content-type=application/json header in your config service?

Evolutio · November 30, 2022, 6:21am

yes: like below:

> GET /api/v1/traefik-config?api_token=cabea4c73c8c883a6a7eeb27193ba27b3d42c2d5905c5680e9149a92568020f1 HTTP/1.1
> Host: localhost
> User-Agent: curl/7.84.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Host: localhost
< Date: Tue, 29 Nov 2022 18:11:20 GMT
< Connection: close
< Cache-Control: no-cache, private
< Date: Tue, 29 Nov 2022 18:11:20 GMT
< Content-Type: application/json
< X-RateLimit-Limit: 60
< X-RateLimit-Remaining: 57
< Access-Control-Allow-Origin: *
<
{
    "http": {
        "routers": {
            "whoami": {
                "rule": "PathPrefix(`\/`)",
                "service": "whoami"
            }
        },
        "services": {
            "whoami": {
                "loadBalancer": {
                    "servers": [
                        {
                            "url": "http:\/\/whoami\/"
                        },
                        {
                            "url": "http:\/\/whoami\/2"
                        }
                    ]
                }
            }
        }
    }
}
* Closing connection 0

system · December 3, 2022, 6:21am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Middleware from file provider does not seem to apply Traefik v2 docker , middleware	7	4344	November 14, 2021
Can someone give me a sanity check why my dynamic config does not load? Traefik v2 file	2	1056	July 13, 2022
Traefik reads but doesn't load dynamic configuration Traefik v2 file	3	424	December 13, 2023
Can't make Traefik work and lost in troubleshooting progress Traefik v2 docker	3	584	March 27, 2023
File provider error Traefik v2 file	3	2812	June 13, 2023

Traefik is not applying config from http provider

Related topics