Slow file download when in the same network

I'm running Traefik v2 on K8s and using it to expose applications to the internet. I am on the same network as the K8s setup (k8s in garage, pc in office on the same 192.168.1.0/24)

If I access https://test.domain.com/jsfile.js from outside the network like my phones internet connection then the file is displayed in full. If I access it on my office pc it takes whole minutes to download.

My tests with curl shows it downloads 24677 bytes and then stalls after about 50 seconds it picks up again and stalls. This keeps repeating until it finishes downloading the file, the size of each downloaded chunk appears to change as well.

From an external connection:

% curl https://ha.home.mikebell.io/frontend_latest/app.e45b0549.js > test.js
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  472k    0  472k    0     0   504k      0 --:--:-- --:--:-- --:--:--  503k

Apparently it took less than 10 seconds. From inside my network:

-> % curl https://ha.home.mikebell.io/frontend_latest/app.e45b0549.js > test.js
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  472k    0  472k    0     0    786      0 --:--:--  0:10:14 --:--:--  5899

A whole 10 minutes!

I know that this might not be a Traefik problem but I'd be happy for any help in debugging this issue since I'm not hugely well versed in networking.