Traefik v2 annotations

Traefik Traefik v2 (latest)
1

Hi Team,

We are migrating from v1.7 to v2.4.8. Are there any annotations that got deprecated in v2.4.8.

Like

traefik.frontend.priority: "1"

I read verified that after migration, all the ingress resources will still be working but are there any annotations that get getting deprecated in v2.4.8.

I didn't find any documentation for it. Could you please help.

Thanks

2

Hello @Manmohan,

Ingress annotation support was removed initially in Traefik v2.

Annotation support was re-added, but minimally, as the CRD is more extensible.

Assume that all annotations have been deprecated, as most have been removed or modified.

A quick peek at our migration notes (Traefik v1 to v2 - Traefik) links to our migration tool that can help migrate your old ingress objects to new CRD objects! Then you can see how the new CRD should look.

3

Hi @daniel.tomcej ,

Thanks for your response. Migration tool doesn't convert every annotation.
For example, I have

traefik.frontend.priority: "1"

So, migration-tool doesn't convert it for traefik v2.
Hence, my question is do I consider that all the annotations in v1.7 are deprecated after migrating from v2.4.8. I wanted to make sure no Ingress breaks after migrating to v2.4.8.

I have large group of kubernetes clusters so, I was looking if there is any document that states, the list of annotations that are deprecated/removed.

5

@daniel.tomcej Sorry, I just want some clarity in my understanding.

Will all the annotations mentioned in v1.7 work in v2?

v1.7: Kubernetes Ingress - Traefik | Site | v1.7
v2 : Kubernetes Ingress - Traefik

6

Hello @Manmohan.

None of the annotations in v1 will work in v2.

7

Hi @daniel.tomcej ,

Thanks for getting back to me. So, as none of the annotations in v1 will work in v2, what is the best way to migrate ingress resources, as we migrate, all the existing annotations will break and we might have to manually update all the ingress resources.

In order to avoid any kind of downtime, I am looking for an option in which, there is no downtime and everything continues to work after migration.

Currently, all the clusters are using Ingress and using IngressRoute is an option but I want all the ingress to work in traefik v2 as well. While using migration tool, it converts the annotations in the form of IngressRoutes. Hence, that might not be useful if we would like to continue with "Ingress" instead of "IngressRoute" in traefik v2.

I was thinking to update our Ingress resources with traefik v2 annotation even before migration so that once they migrate, they have all the new annotations already but would like to check if this is a viable option or do you have any suggestion.

Thanks,
Manmohan

8

Hello @Manmohan

Thanks for using Traefik!

If you have a choice, I would consider to migrate all existing Kubernetes ingress to IngressRoute CRD as it is provides more powerful and flexible of creating ingress resources.

I am keeping fingers crossed for the migration.

9

Hi @jakubhajek

Thanks for responding to my query. Yes, moving to "IngressRoute" will be the plan in future but considering, all our customers are using "Ingress" right now. I wanted to migrate to traefik v2 without downtime and then give them an option to move to "IngressRoute". I was thinking, all the "Ingress" resources will work in traefik v2 without any downtime while migrating traefik from v1 to v2 but it looks like if some customers are using "Ingress" with "annotations" then there might be a downtime.

Hence, "IngressRoute" is an option but then it involves downtime right? i.e. I have to first migrate traefik v2 and then ask customers to move to "IngressRoute", which involves downtime in production and we can't afford that. So, getting a equivalent annotation (Ingress) for traefik v2 will be beneficial.

I tried using migration tool but then it convert the "Ingress" resources to "IngressRoute", and it doesn't provide any equivalent annotation instead converts the annotations as a middleware or config itself.

Hence, I am struggling to identify the equivalent annotation that customers could use for "Ingress" resource in traefik v2.

Here are some of the annotations that we are using and I would like to find the equivalent of it in traefik v2. If I have equivalent annotations then may be I will ask customers to include those before I migrate to traefik v2. and if traefik v2 annotations doesn't work in traefik v1 then probably, I will have a list of annotation that they can replace once migration is done.

"ingress.kubernetes.io/allowed-hosts":
"ingress.kubernetes.io/app-root":
"ingress.kubernetes.io/auth-secret":
"ingress.kubernetes.io/auth-type":
"ingress.kubernetes.io/custom-request-headers":
"ingress.kubernetes.io/preserve-host":
"ingress.kubernetes.io/priority":
"ingress.kubernetes.io/protocol":
"ingress.kubernetes.io/proxy-body-size":
"ingress.kubernetes.io/proxy-headers":
"ingress.kubernetes.io/rewrite-target":
"ingress.kubernetes.io/ssl-redirect":
"ingress.kubernetes.io/use-port-in-redirects":
"kubernetes.io/ingress.class":
"traefik.backend.loadbalancer.sticky":
"traefik.frontend.priority":
"traefik.frontend.redirect.regex":
"traefik.frontend.redirect.replacement":
"traefik.frontend.rule.type":
"traefik.ingress.kubernetes.io/affinity":
"traefik.ingress.kubernetes.io/app-root":
"traefik.ingress.kubernetes.io/error-pages":
"traefik.ingress.kubernetes.io/frontend-entry-points":
"traefik.ingress.kubernetes.io/redirect-entry-point":
"traefik.ingress.kubernetes.io/redirect-permanent":
"traefik.ingress.kubernetes.io/redirect-regex":
"traefik.ingress.kubernetes.io/redirect-replacement":
"traefik.ingress.kubernetes.io/request-modifier":
"traefik.ingress.kubernetes.io/rewrite-target":
"traefik.ingress.kubernetes.io/rule-type":
"traefik.ingress.kubernetes.io/session-cookie-name":
"traefik.ingress.kubernetes.io/ssl-redirect":

In short, I want to avoid any downtime for the existing "Ingress" resources while I migrate traefik to v2. Once, everything is working in traefik v2 then we can try moving to IngressRoute.

I hope, I am able to explain my situation better. Please let me know if you have any suggestions?

10

Hello @Manmohan

Thank you for your reply and for explaining in detail the plan.

Just to clarify Kubernetes Ingress (the standard k8s implementation) is fully available in v2. However, as @daniel.tomcej mentioned none of the annotations that were supported in v1 won't work in v2.
I understand that there makes the migration more complex because you need to manually select the annotation supported by v2 with v1.

I would definitely consider migrating to CRD while moving from v1 to v2. It is not clear to me why you are suspecting to have downtime with CRD.

What would be the difference between migrating ingress v1 to ingress v2 comparing to ingress v1 to CRD? Please help me to understand your doubts.

Thank you,

11

Hi @jakubhajek

Thanks for responding back. The reason why I said downtime is because

  1. I was in impression that we can migrate the ingress controller from v1 to v2 and all the existing ingress resources will work well.
  2. Above statement is true when we don't have annotations in our "Ingress" resources. So for example, we migrate ingress controller from v1 to v2, all the Ingress resources having annotations will break unless we migrate them to "IngressRoute" along with Middleware and other CRD's.
  3. All our customers are managing their Ingress resources and so, I think now they have to migrate their Ingress Resources parallelly while we migrate ingress controller from v1 to v2.

Do I make sense? Please let me know if I need to explain it better.

Another question is, what if we have both i.e. Ingress and IngressRoute with same host (URL). Which one will take precedence over another in traefik v2. Is there a way to control that. In order to avoid any kind of downtime after migrating to Ingress Controller, I was thinking to create CRD's in traefik v1 environments itself (Only CRD's and IngressRoute resources) and the moment we migrate the ingress controller from v1 to v2, "IngressRoute" is taken into account and "Ingress" gets ignored. That way, we could avoid any kind of downtime. We have a big environment and it's important that we consider downtime.

Excuse me for asking lot of questions and thanks for your time in going through it.

Thanks,
Manmohan

12

Hello @Manmohan

To migrate from v1 to v2, wich solution did you choise to minimize downtime ?

Did you convert first all not compliant Ingress to IngressRoute+Middleware then switch to v2 ?

Which one will take precedence over another in traefik v2. Is there a way to control that.

To force IngressRoute to be prior we will use spec.routes.priority, did you used this attribute ?
@jakubhajek do you confirm the behavior of spec.routes.priority ?

13

Hello @Berber-31-33

The priority is being used to avoid path overlapping. By default, routes are sorted in descending order based on the length of the rule. The priority is equal to the length of the rule, which means if the rule is quite long it will get the higher priority.

By setting spec.routes.priority you can decide what is the rule priority and overwrite the calculated priority.

I hope that helps.

Thank you,

1 Like
14

Hi @Berber-31-33 ,

We almost completed traefik migration. So, here are the few things we did.

  1. Created a document, that defines the replacement for every annotation in IngressRoute. As some of the annotations do not have direct replacement and some needs work around.
  2. Requested customers to create IngressRoute using Traefik migration tool and assign a higher priority. As, @jakubhajek mentioned, priority is taken based on length of the rule. So, we selected a value of 555 for all the IngressRoutes. That's a random number we selected. This is before migration and once migration of traefik is done, it selects IngressRoutes as it has higher priority and that avoids downtime.
  3. Few things we realised after migration is that some of the settings in traefik v1 are set by default and in traefik v2, we had to enable it.
  4. Deleted the Ingress configuration as it is obsolete now.

P.s. Ingress with annotations, needs a conversion to IngressRoutes.

I hope, this is helpful. I will be happy to answer if you have more questions.

Thanks,
Manmohan Rawat

15

Thank you for your answer !
(We are in the same situation, migrating v1 to v2, with more than 2500 ingress to check)

And when digging further into not compliant annotations, we found v1 annotations on services (Kubernetes Ingress | Traefik | v1.7) seems to be not compliant with v2 (but nothing mentionned in migration guide Traefik v1 to v2 - Traefik)

And found v2 annotations for services are listed here: Kubernetes Ingress - Traefik

So @Manmohan did you check also traefik annotations on services to be sure to do not break when switching to traefik v2 ?

And @jakubhajek do you confirm v1 annotation on services are not compliant and should be transformed to v2 (listed here Kubernetes Ingress - Traefik) ?
And do you confirm traefik migration tool does not support annotations on services ?

Thanks in advance

16

Hello @Berber-31-33

You are working on an interesting and challenging project, 2500 ingress resources sound like a massive migration! We are keeping our fingers crossed for the successful migration!

Going back to your questions:

  • annotations on a service level seem to be specific on v2 only.
  • The other thing is that the migration tool converts Kubernetes ingresses to IngressRoutes CRDs, so no v2 Ingresses annotations are produced by this tool. If I understands what you are trying to achieve, migration from ingresses configured for v1 to ingresses configured for v2 is something that cannot be addressed by the migration tool.

Let me give you an example:

the v1 annotation ingress.kubernetes.io/protocol is corresponding to the v2 annotation traefik.ingress.kubernetes.io/service.serversscheme, but the migration tools make use of IngressRoute configuration, so it configures the IngressRoute service to have scheme: something property.

I hope that helps.

1 Like
17

Thank you for your instant answer ! i appreciate it !

(yes we use kubernetes massively)

I think i wasn't very clear, let me rephrase that:
for now we use traefik v1 and have also services with v1 annotations (Kubernetes Ingress | Traefik | v1.7), for example like this:

apiVersion: v1
kind: Service
metadata:
  annotations:  
    traefik.ingress.kubernetes.io/affinity: "true"
    traefik.ingress.kubernetes.io/session-cookie-name: so019-affinity-tic-a
  name: so019-peama-service
  namespace: so019-peama-tic-a  
spec:
...

so

  1. these annotations are not compliant with traefik v2, right ?

    and corresponding v2 annotations seems to be (regarding v2 doc: Kubernetes Ingress - Traefik):

  2. and this service manifest could not be converted by the migration tool, right ?

18

Hi @Manmohan

We have a similar use case. Would you be able to help me with equivalent CRDs for these annotations to be used with traefik v2.7:

ingress.kubernetes.io/protocol: h2c
kubernetes.io/ingress.class: traefik-grpc