Can Traefik cause a downtime during dynamic configuration reload?

Background

We are using Traefik for around 2 weeks now after a switch from an Apache Proxy setup to docker swarm based setup. The project is rather large with around 400 domains including Letsencrypt certificates and 50 separated yaml dynamic configuration files containing 190 routes and 80 middlewares.

During the switch the configuration was modified quite often, because new Domains moved to the new infrastructure, fixing typo's in the config or just adding features like rate-limiter. There could be around 20 changes per day or more - mostly because of new Domains - thus creating new certificates.

The Happening

Most of the changes went smooth without much problems. But sometimes we had response codes like 502 Bad Gateway or an exotic response like 406 Not acceptable which I think came from the application because of missing Accept Header but not sure why. The responses lasted from a few seconds to 2 minutes and was suddenly fixed again without any further changes.

Questions

To rule out Traefik and learn somthing:

How does Traefik update it's configuration? Can there be a downtime during reconfiguring middlewares, routes and containers/services somehow? Or is this just not possible, because traefik loads the whole config, creates internal structures and then does an somewhat atomic switch of everything?

Many thanks for your time and help

Hello,

Traefik prepares the configuration before applying it, the switch of the routing (routers, services, middleware) is done in memory and it's instantaneous.

But before applying the configuration, Traefik needs to "prepare" it, so the changes are not instantaneous.

So there is no downtime during dynamic configuration reload.

2 Likes

hello,
i searched about this extensively. there is definitely downtime while updating the dynamic file.

how to test?
edit the file without changing anything, just modified date should change. (i use winscp and npp for this)
save the file while refreshing the page, you should see 404 for couple of seconds. after that page loads correctly.

how do i debug this issue?

If there are no changes, the configuration is not reloaded.

In your case, when you edit your file, your editor creates an empty file.

1 Like

thank you very much :man_facepalming::grin:

tried with nano on the server, no 404's. that was the problem.
as i understand it, if old file is deleted and new file is created every time i edit and save the file, traefik cleans the deleted file's config and when it discovers the new file it re-applies all the configs?(something to do with filesystem internal workings, inode change?) hence couple of seconds long downtime. i set providersthrottleduration to 10seconds but that won't help either.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.