Zookeeper polling/refresh interval

We are currently implementing a new set of traefik instances in our infrastructure. For us the best option would be to use zookeeper (kv) as we have that for our dynamic configs. What I have seen is the traefik instance (bare metal) doesn't pick up changes to data under the traefik root in zookeeper.

Is there a refresh or watch to detect the key changes? I don't see it in the code base in github but maybe I am wrong.

For example changing a weight of a service, doesn't not update in the dashboard or on the /api/http/services

Here is a little more information on what we see via the api /api/http/services

{"weighted":{"services":[{"name":"bapache01","weight":1},{"name":"bapache04","weight":1},{"name":"bapache05","weight":1}]}

The logs from traefik on startup show that we are using the zk provider just fine

time="2022-04-13T13:41:49-04:00" level=info msg="Traefik version 2.6.3 built on 2022-03-29T13:09:27Z"
time="2022-04-13T13:41:49-04:00" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
time="2022-04-13T13:41:49-04:00" level=info msg="Starting provider aggregator.ProviderAggregator"
time="2022-04-13T13:41:49-04:00" level=info msg="Starting provider *traefik.Provider"
time="2022-04-13T13:41:49-04:00" level=info msg="Starting provider *zk.Provider"

When changing the zookeeper key

[zk: bzk04:9540(CONNECTED) 2] get /traefik/http/services/b-docroot/weighted/services/bapache01/weight
0

There are no notifications in the zookeeper log and we see the weights haven't changed

{"weighted":{"services":[{"name":"bapache01","weight":1},{"name":"bapache04","weight":1},{"name":"bapache05","weight":1}]}

After a restart of the traefik service we now see the updated weight

{"weighted":{"services":[{"name":"bapache01","weight":0},{"name":"bapache04","weight":1},{"name":"bapache05","weight":1}]}

Again nothing in the logs concerning this.