Traefik V2 not getting acme certificate

After following this https://docs.traefik.io/v2.0/user-guides/crd-acme/ tutorial I was able to access the Traefik dashboard on http://0.0.0.0:80 with port-forward

But when I check traefik pods for log found

time="2019-09-26T18:26:53Z" level=error msg="Unable to obtain ACME certificate for domains \"traefik.domain2.ca\": unable to generate a certificate for the domains [traefik.domain2.ca]: acme: Error -> One or more domains had a problem:\n[traefik.domain2.ca] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Connection refused, url: \n" rule="Host(`traefik.domain2.ca`) && PathPrefix(`/tls`)" routerName=kube-system-ingressroutetls-b40dfb7b33675822951c providerName=default.acme
time="2019-09-26T18:26:55Z" level=error msg="Unable to obtain ACME certificate for domains \"traefik.domain2.ca\": unable to generate a certificate for the domains [traefik.domain2.ca]: acme: Error -> One or more domains had a problem:\n[traefik.domain2.ca] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Connection refused, url: \n" providerName=default.acme routerName=kube-system-ingressroutetls-b40dfb7b33675822951c rule="Host(`traefik.domain2.ca`) && PathPrefix(`/tls`)"

But this curl http://traefik.domain.ca:80/notls works
this curl http://traefik.lightcloud.ca/tls doesn't work.

I have this config

      - name: traefik
        image: traefik:v2.0
        args:
          - --api.insecure
          - --accesslog
          - --entrypoints.web.Address=:80
          - --entrypoints.websecure.Address=:443
          - --providers.kubernetescrd
          - --certificatesresolvers.default.acme.tlschallenge
          - --certificatesresolvers.default.acme.email=first.last@domain2.com
          - --certificatesresolvers.default.acme.storage=acme.json
          # Please note that this is the staging Let's Encrypt server.
          # Once you get things working, you should remove that whole line altogether.
          - --certificatesresolvers.default.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory

Also tried to comment --certificatesresolvers.default.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory but same error.

Update :

I switched to DNS Challenge

  - name: traefik
    image: traefik:v2.0
    args:
      - --api.insecure
      - --accesslog
      - --entrypoints.web.Address=:80
      - --entrypoints.websecure.Address=:443
      - --providers.kubernetescrd
      - --certificatesresolvers.default.acme.email=first.last@domain2.com
      - --certificatesresolvers.default.acme.storage=acme.json
      - --certificatesresolvers.default.acme.dnschallenge=true
      - --certificatesresolvers.default.acme.dnschallenge.provider=route53
      - --certificatesresolvers.default.acme.dnsChallenge.delayBeforeCheck=0
    env:
      - name: AWS_REGION
        valueFrom:
          configMapKeyRef:
            name: aws-config
            key: aws_region
      - name: AWS_HOSTED_ZONE_ID
        valueFrom:
          configMapKeyRef:
            name: aws-config
            key: aws_hosted_zone_id
      - name: AWS_ACCESS_KEY_ID
        valueFrom:
          secretKeyRef:
            name: aws-secret
            key: access_key
      - name: AWS_SECRET_ACCESS_KEY
        valueFrom:
          secretKeyRef:
            name: aws-secret
            key: secret_key

everything worked.

Also how can I get wild card certificate, because above config trying to get certificate for traefik.domain.ca.

How can I specify the wildcard request (*.domain.ca)? and subdomain wildcard request (*.test.domain.ca)?

It looks like you are using tlsChallenge. As documentation says:

As described on the Let's Encrypt community forum, when using the TLS-ALPN-01 challenge, Traefik must be reachable by Let's Encrypt through port 443

It does not appear from your configuration, that your Traefik is reachable by Let's Encrypt through port 443. Can you check that please?

I switched to dnsChallenge and everything worked