Bypass Authentik Forward Auth for Local Addresses

I have a service that I'm using Authentik forward auth for, but it would be nice if the forward auth is bypassed on the local subnet. I've read some posts about people being able to do it with a regex but I'm not exactly sure how to implement it and in their scenarios it seems as though they are just using it to bypass basic auth. Below is my current config for the forward auth.

http:
  routers:
    notes:
      entryPoints:
        - "https"
        - "http"
      rule: "Host(`notes.DOMAINNAME.com`)"
      middlewares:
        - chain-pre-auth
      priority: 10
      tls:
        certResolver: dynu
        domains:
          - main: "DOMAINNAME.com"
            sans:
              - "*.DOMAINNAME.com"
      service: notes
    notes-router-auth:
      rule: "Host(`notes.DOMAINNAME.com`) && PathPrefix(`/outpost.goauthentik.io/`)"
      priority: 15
      service: authentik-router-auth

  services:
    notes:
      loadBalancer:
        servers:
          - url: "http://172.16.0.80:3000"
        passHostHeader: true
    authentik-router-auth:
      loadBalancer:
        servers:
          - url: "http://172.16.0.8:9000/outpost.goauthentik.io"
        passHostHeader: true
  
  middlewares:
    authentik-auth:
      forwardAuth:
        address: http://172.16.0.8:9000/outpost.goauthentik.io/auth/traefik
        trustForwardHeader: true
        authResponseHeaders:
          - X-authentik-username
          - X-authentik-groups
          - X-authentik-email
          - X-authentik-name
          - X-authentik-uid
          - X-authentik-jwt
          - X-authentik-meta-jwks
          - X-authentik-meta-outpost
          - X-authentik-meta-provider
          - X-authentik-meta-app
          - X-authentik-meta-version

I'm only using YAML config for this particular service.

I attempted to add another router with a lower priority and added the IPAllowList middleware thinking it would just hit that rule first then continue on, but alas it doesn't seem to work like firewall rules (go figure).

Any help would be appreciated as I'm really just getting started in Traefik.

You can create another router with rule Host() && ClientIP() without forward auth middleware, re-use the service.