Upgrade of hub-agent is not successful

Traefik Hub
1

Hello community,

I recently upgraded from traefik version 20.8.0 and hub-agent version 1.2.1. However, after the upgrade, the deployment of hub-agent-dev-portal pods kept restarting after version to version 1.4.0. Although Lens Dashboard indicated a successful upgrade, I proceeded with upgrading to version 1.5.7.

The 4 pods of hub-agent-dev-portal are restarting, I tried to delete them but the pods keep restarting.

Unfortunately, I could only upgrade traefik to version 21.0.0, and upon attempting to upgrade to version 21.1.0, I received an error message:

"Error: UPGRADE FAILED: cannot patch "traefik-dashboard" with kind IngressRoute: Internal error occurred: failed calling webhook "hub-agent.hub-agent.svc": failed to call webhook: Post "https://hub-agent-controller.hub-agent.svc:443/ingress?timeout=10s": EOF"

I am not sure how to resolve this issue. Should I delete the traefik, hub-agent and reinstall it? I have also attempted to rollback, but I received a "Bad Request" message.

I appreciate any guidance or suggestions you may have.

Thank you, Ranganath.

2

Hi @Exnadella !

Could you please:

  • Tell us the exact versions of Traefik Proxy and Traefik Hub and the Helm chart you're running right now?
  • Share your configuration?

Thank you!

3

Hi @svx,

Thank you for looking into the issue.

Over the weekend, I deleted the entire hub-agent and all my configurations from the traefik dashboard. I then reinstalled the latest versions of hub-agent (v1.4.2) and traefik proxy (traefik v2.9.10) using helm versions 1.5.7 and 22.1.0, respectively. I also tried deleting the validatingwebhookconfigurations & mutatingwebhookconfigurations as suggested in one of the blogs, but I couldn't find either of them.

However, even though traefik and hub-agent were successfully deployed, the problem remains the same. Since then I am facing an issue where a loadbalancer service that I published is not published, even though one of the service has been published.

Both the services were previously working and reachable before the upgrade.

4

Hi @Exnadella,

could you run both, Traefik and The Hub Agent in DEBUG mode and post the output?
Could you also share an example service configuration (without private settings)

Thanks!

5

Hi @svx,

How to run the Helm in debug mode? I can export multiple logs for tunnel, controller, dev-portal, auth-server and traefik will that be sufficient?

6

Hey @Exnadella,

For Traefik Hub:

If you use the default chart, the log level is already set to DEBUG, so just posting/exporting them will be enough.
If you changed the level, you have to set them the level to debug and re-deploy.

For Traefik:
You can adjust the level in the chart and re-deploy.

7

Hi @svx,

I have successfully managed to publish all my services. However, I faced a challenge when I tried to delete all the traefik-hub published services on the Hub-Dashboard. One service was stuck with the old deployment and it was causing an issue. To resolve the problem, I deleted and re-deployed the problematic deployment, and then republished it. Nonetheless, I am still facing an issue with the API-server not being visible on the Traefik-Hub Dashboard.