StatsD Metrics setup with Treafik v2.2

Traefik Traefik v2
#1

I'm not able to receive metrics from Traefik. I'm currently trying to setup the statsd service but I keep receiving the following error and no metrics are available in the host:

time="2020-05-17T20:17:32Z" level=info msg="[err write udp 127.0.0.1:60514->127.0.0.1:8125: write: connection refused]" metricsProviderName=statsd
time="2020-05-17T20:17:32Z" level=info msg="[during WriteTo err write udp 127.0.0.1:60514->127.0.0.1:8125: write: connection refused]" metricsProviderName=statsd

My current configuration file is the following on the metrics section:

[metrics]
  [metrics.statsd]
    address = "127.0.0.1:8125"
    addEntryPointsLabels = true
    addServicesLabels = true
    pushInterval = "10s"
    prefix = "traefik"

Traefik version: 2.2.1 built on 2020-04-29T18:02:09Z

Am i missing something on the metrics configuration? At least this is all required to enable the metrics as per the docs.

Thanks

#2

Are you running traefik standalone or as a container? And where is statsd running, standalone or a container also?

#3

Traefik is running as a container while Statsd is running as a service in the host machine. I’m actually running a collectd service on my sever that pushes my metrics to my metrics database. Collectd has a plugin that implementa the statsd protocol. The plugin information is here. The collectd/plugin part is working fine as I’m able to send metrics to my database with test bash commands like this: echo "foo:1|c" | nc -u -w0 127.0.0.1 8125.

#4

So when in the traefik container you reference 127.0.0.1 you are not referencing the docker host, your are referencing the traefik container itself. You need to use one of the addressable ips of the host.

Personally I would move collectd into a container as it make it easily addressable by host and other containers.

If you are running docker for mac/windows you can utilise the special hostname host.docker.internal to address the container host. I don't like using this, as it does not translate well into linux.

Create error page with the provide file
MERN stack - api call from React not redirecting to back end with Traefik + docker
#5

Yes, that makes sense. I thought Traefik would take that into account automatically since its main use case is running as a container. But I agree it would be easier to move collectd to a container as well. As I'm running it on a Linux machine, host.docker.internal is not an option for me.

Thanks for the help :slight_smile:

#6

So I have it working on my system now and as there are some important details, I'd like to share the main points of my configuration:

Traefik.toml

[metrics]
  [metrics.statsd]
    address = "172.18.0.1:8125"
    addEntryPointsLabels = true
    addServicesLabels = true
    prefix = "traefik"

Important remark here: the address needs to be the host ip, as this is running in Linux, host.docker.internal is not an option so I'm using the ip. Usually, the ip address for the host machine is 172.18.0.1 when you are on a bridge ( user-defined) network but it may be different on other machines. For more information, please check this answer on stackoverflow.

Firewall setup
If you are using a firewall, you need to allow the udp message flow from the Docker containers to the host, as I'm using iptables, I set the following rule: iptables -A INPUT -p udp -s 172.18.0.0/16 -d 172.18.0.1 --dport 8125 -j ACCEPT. This will allow the udp traffic on the port 8125 from any container on the docker network to the host.

PS: It would be easier to setup collectd on another container but that's not the best option on my case due to some other constraints, so I have the collectd service running on my host and Traefik on a container.