Okay, digging around and reading the docs multiple times, I find out that it is not possible with Traefik to make a "http AND https" ingress.
To enable TLS, the annotation traefik.ingress.kubernetes.io/router.tls: "true" is necessary (there are other ways too).
But then, the http endpoint will not work.
The only possible way is using 2 routers or one router with a http-https-redirect middleware.
See these for more information: