How to use mirroring to implement HA

Traefik Traefik v3 (latest)
1

Hi,

I'm experimenting with using Traefik mirroring to send JSON logs to two separate Victoria Logs instances. This is to implement a form of high availability.

This works with the following configuration:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: vlog-mirror-ingress-route
  namespace: monitoring
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix(`/insert/loki/api/v1/push`)
    kind: Rule
    services:
    - name: vlog-mirror-traefik-service
      namespace: monitoring
      kind: TraefikService

apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: vlog-mirror-traefik-service
  namespace: monitoring

spec:
  mirroring:
    name: vlogs-sandbox-0      # svc1 receives 100% of the traffic
    namespace: monitoring
    port: 9428
    mirrors:
      - name: vlogs-sandbox-1  # svc2 receives a copy of 100% of this traffic
        namespace: monitoring
        port: 9428
        percent: 100

One issue though is that if vlogs-sandbox-0 fails, then I believe that requests will not be mirrored to vlogs-sandbox-1. Is there a way of getting mirroring to work so that at least one of the backends gets the requests?

Thanks

2

Do you want to send the logs to both services most of the time? Or rather only have a fallback? Did you check Traefik failover for services?

3

We want the logs sent to both services all the time. The Victoria Logs Pods run on different Nodes. If we lose one of those Nodes we still need Victoria Logs to be usable via the remaining Node.

Thanks for sharing the failover feature - I think it doesn't help us in this case though unfortunately.