Let's Encrypt not able to open acme.json

The K8s cluster is new and on Digital Ocean, created with terraform.

I have a helm chart that has a sub-chart to start traefik. This is the
standard helm chart (newest release) and I only changed the values file.
I am trying to use Let's Encrypt with the DNS resolver to GoDaddy.
I have the GoDaddy keys as a K8s secret.

The traefik pod did restart once.

pod/rhf-traefik-5d4558cf9b-vb2p7 1/1 Running 1 (45s ago) 53s

I get the error message 

time="2022-09-29T13:44:57Z" level=info msg="More details on: https://doc.traefik.io/traefik/contributing/data-collection/"
time="2022-09-29T13:44:57Z" level=error msg="The ACME resolver \"letsencrypt\" is skipped from the resolvers list because: unable to get ACME account: open acme.json: read-only file system"
time="2022-09-29T13:44:57Z" level=warning msg="Traefik Pilot is deprecated and will be removed soon. Please check our Blog for migration instructions later this year."

and when I go to the https site I get a self-signed cert with the ID of:

CN=TRAEFIK DEFAULT CERT

When I log into the pod I see that /data is there and I can write a file to it.

/ $ ls -l /data
total 16
-rw-------    1 65532    65532            0 Sep 29 13:44 acme.json
drwxrws---    2 root     65532        16384 Sep 29 13:44 lost+found
/ $ echo "Hi There" > /data/foobar
/ $ ls -l /data
total 20
-rw-------    1 65532    65532            0 Sep 29 13:44 acme.json
-rw-r--r--    1 65532    65532            9 Sep 29 13:54 foobar
drwxrws---    2 root     65532        16384 Sep 29 13:44 lost+found
/ $ 

I deleted the traefik pod and allowed it to re-start then logged into the new
pod and verified that the file I created in the last pod was still there.

➜  helm git:(digital-ocean) ✗ kubectl exec --stdin --tty rhf-traefik-XXXXXXXXXXXXXXXX -- /bin/sh
/ $ ls -l /data
total 20
-rw-rw----    1 65532    65532            0 Sep 29 13:44 acme.json
-rw-rw-r--    1 65532    65532            9 Sep 29 13:54 foobar
drwxrws---    2 root     65532        16384 Sep 29 13:44 lost+found
/ $ cat /data/foobar 
Hi There
/ $

I have run a helm template on the traefik chart and looked at the output and as far as
I can see everything looks good. I have verified that there is a pvc and a pv that are bound
I am not sure what I am doing wrong.

➜ dev-pwd-1 git:(digital-ocean) ✗