Creating or replacing an Middleware object in Kubernetes using the Kubernetes Java Client Library doesn't work (400/404)

Traefik Traefik v2 (latest)
,
1

Hi,
I already put this into an Issue on GitHub but it was closed. I assume that the decision was made that this isn't a bug.

I'm using a Java application (running inside a Kubernetes cluster) to create or update a Traefik Middleware config.
A sample getting started:

ApiClient apiClient = Config.defaultClient();
Configuration.setDefaultApiClient(apiClient);
CustomObjectsApi customObjectsApi = new CustomObjectsApi(apiClient);
String kubeConfGroup = "traefik.containo.us";
String kubeConfVersion = "v1alpha1";
String kubeMiddlewareName = "ip-whitelist";
String namespace = "ingress";
apiClient.setDebugging(true);

Check for an existing object:

// Note the "middlewares" here
String kubeConfPlural = "middlewares";
customObjectsApi.getNamespacedCustomObject(kubeConfGroup, kubeConfVersion, namespace, kubeConfPlural, kubeMiddlewareName);

This one works. If I do the same thing with the value I would expect (kubeConfPlural="Middleware") it doesn't work:

String kubeConfPlural = "Middleware"
customObjectsApi.getNamespacedCustomObject(kubeConfGroup, kubeConfVersion, namespace, kubeConfPlural, kubeMiddlewareName);

Gives me response:

--> GET https://kubernetes/apis/traefik.containo.us/v1alpha1/namespaces/ingress/middlewares/ip-whitelist
Accept: application/json
Content-Type: application/json
User-Agent: Kubernetes Java Client/16.0.0-SNAPSHOT
--> END GET
<-- 200 https://kubernetes/apis/traefik.containo.us/v1alpha1/namespaces/ingress/middlewares/ip-whitelist (149ms)
audit-id: f8961307-dc71-408f-904c-57da8f68b4ca
cache-control: no-cache, private
content-type: application/json
x-kubernetes-pf-flowschema-uid: c9112f42-1645-4e11-84b7-a1ac09633f5e
x-kubernetes-pf-prioritylevel-uid: 82e2fd63-52b0-4fb4-b9e0-4786d9a7f5ab
content-length: 831
date: Fri, 10 Feb 2023 10:11:28 GMT
 
 {"apiVersion":"traefik.containo.us/v1alpha1","kind":"Middleware","metadata":{"creationTimestamp":"2023-02-10T06:37:10Z","generation":3,"managedFields":[{"apiVersion":"traefik.containo.us/v1alpha1","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{".":{},"f:ipWhiteList":{}}},"manager":"kubectl-client-side-apply","operation":"Update","time":"2023-02-10T06:37:10Z"},{"apiVersion":"traefik.containo.us/v1alpha1","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:ipWhiteList":{"f:sourceRange":{}}}},"manager":"kubectl-replace","operation":"Update","time":"2023-02-10T09:37:37Z"}],"name":"ip-whitelist","namespace":"ingress","resourceVersion":"355735","uid":"deaeaece-fbb9-4d44-93a1-dee5c80f677b"},"spec":{"ipWhiteList":{"sourceRange":["127.0.0.1/32","127.1.0.4/32","127.0.0.2/32","127.0.0.5/32","10.244.0.0/24","10.244.1.1/24"]}}}

 <-- END HTTP (831-byte body)

So using the "middlewares" is my solution here. However, replacing an existing object or creating a object doesn't seem to work.

customObjectsApi.replaceNamespacedCustomObject(kubeConfGroup, kubeConfVersion, namespace, kubeConfPlural, kubeMiddlewareName, jsonTraefikMiddleware, null, null);

Updating an existing object doesn't work. I get the following response:

--> PUT https://kubernetes/apis/traefik.containo.us/v1alpha1/namespaces/ingress/middlewares/ip-whitelist
Content-Length: 531
Accept: application/json
Content-Type: application/json
User-Agent: Kubernetes Java Client/16.0.0-SNAPSHOT

"{\n    \"apiVersion\": \"traefik.containo.us/v1alpha1\",\n    \"kind\": \"Middleware\",\n    \"metadata\": {\n        \"name\": \"ip-whitelist\",\n        \"namespace\": \"ingress\"\n    },\n    \"spec\": {\n        \"ipWhiteList\": {\n            \"sourceRange\": [\n                \"127.0.0.1/32\",\n                \"127.0.0.211/32\",\n                \"127.0.0.121/32\",\n                \"127.0.0.112/32\",\n                \"10.244.0.0/24\",\n                \"10.244.1.0/24\"\n            ]\n        }\n    }\n}\n"
--> END PUT (531-byte body)
<-- 400 https://kubernetes/apis/traefik.containo.us/v1alpha1/namespaces/ingress/middlewares/ip-whitelist (263ms)
audit-id: b3055434-5238-486c-bc51-4d96ddd85408
cache-control: no-cache, private
content-type: application/json
x-kubernetes-pf-flowschema-uid: c9112f42-1645-4e11-84b7-a1ac09633f5e
x-kubernetes-pf-prioritylevel-uid: 82e2fd63-52b0-4fb4-b9e0-4786d9a7f5ab
content-length: 456
date: Fri, 10 Feb 2023 10:13:18 GMT

{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the object provided is unrecognized (must be of type Middleware): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" } (227b5c6e202020205c2261706956657273696f6e5c223a205c2274726165 ...)","reason":"BadRequest","code":400}

<-- END HTTP (456-byte body)

Doing the previous request again (but now with the "Middleware" value, gives me the following response back:

--> PUT https://kubernetes/apis/traefik.containo.us/v1alpha1/namespaces/ingress/Middleware/ip-whitelist?fieldManager=kubectl-replace
Content-Length: 531
Accept: application/json
Content-Type: application/json
User-Agent: Kubernetes Java Client/16.0.0-SNAPSHOT

"{\n    \"apiVersion\": \"traefik.containo.us/v1alpha1\",\n    \"kind\": \"Middleware\",\n    \"metadata\": {\n        \"name\": \"ip-whitelist\",\n        \"namespace\": \"ingress\"\n    },\n    \"spec\": {\n        \"ipWhiteList\": {\n            \"sourceRange\": [\n                \"127.0.0.1/32\",\n                \"127.0.0.211/32\",\n                \"127.0.0.121/32\",\n                \"127.0.0.112/32\",\n                \"10.244.0.0/24\",\n                \"10.244.1.0/24\"\n            ]\n        }\n    }\n}\n"
--> END PUT (531-byte body)
<-- 404 https://kubernetes/apis/traefik.containo.us/v1alpha1/namespacesingress/Middleware/ip-whitelist?fieldManager=kubectl-replace (26ms)
audit-id: 4ecdd462-2b5a-4750-8d11-d55b669a65b6
cache-control: no-cache, private
content-type: text/plain; charset=utf-8
x-content-type-options: nosniff
x-kubernetes-pf-flowschema-uid: c9112f42-1645-4e11-84b7-a1ac09633f5e
x-kubernetes-pf-prioritylevel-uid: 82e2fd63-52b0-4fb4-b9e0-4786d9a7f5ab
content-length: 19
date: Fri, 10 Feb 2023 10:28:30 GMT

404 page not found

<-- END HTTP (19-byte body)

The last one returns a 404 page not found (if I relate that back to the middlewares thing, I can of course explain this issue).

Below is an example json that I use. The exact same request from the kubectl command works, and I don't have any errors there when I put the logging on -v10.

{
    "apiVersion": "traefik.containo.us/v1alpha1",
    "kind": "Middleware",
    "metadata": {
        "name": "ip-whitelist",
        "namespace": "ingress"
    },
    "spec": {
        "ipWhiteList": {
            "sourceRange": [
                "127.0.0.1/32","127.1.0.4/32","127.0.0.2/32","127.0.0.5/32","10.244.0.0/24","10.244.1.1/24"
            ]
        }
    }
}

I get 404 page not found on Middleware and 400 on middlewares. Using the kubectl command line (and the before mentioned json) works without issues. I also tested by Java code with a different Custom module, and that one works without any issues. I suspect this is related to the CRD config within the Helm chart, but I could not find a cause of it when looking over the CRD yaml's.

I use the following Traefik version installed on a clean/new Azure Kubernetes Cluster:
traefik version
Version: 2.9.6
Codename: banon
Go version: go1.19.4
Built: 2022-12-07T14:17:58Z
OS/Arch: linux/amd64

2

With a lot of trial and error I finally found the cause of the issue. For the Traefik CRD the object that is being send to the Kubernetes API server needs to be a Gson object. A String based Json doesn't seem to be accepted. For those looking to do the same thing I did, here is a mockup solution:

//CreateMiddleware.java
Gson gson = new Gson();
JsonObject jsonObject = gson.fromJson(jsonTraefikMiddleware, JsonObject.class);
customObjectsApi.createNamespacedCustomObject(kubeConfGroup, kubeConfVersion, namespace, kubeConfPlural, jsonObject, null, null, null);

When updating and existing object, you will need to add the current resource version to the json. Below example:

//ExistingMiddleware.java
Object obj = customObjectsApi.getNamespacedCustomObjectWithHttpInfo(kubeConfGroup, kubeConfVersion, namespace, kubeConfPlural, kubeMiddlewareName).getData();
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
String json = ow.writeValueAsString(obj);
Gson gson = new Gson();
JsonObject jsonObject = gson.fromJson(json, JsonObject.class);
String resourceVersion = jsonObject.getAsJsonObject("metadata").get("resourceVersion").toString();
String jsonTraefikMiddleware = String.format(jsonTraefikMiddlewareStringMultiline, kubeMiddlewareName, namespace, jsonIpsStringMultiline, resourceVersion);
customObjectsApi.replaceNamespacedCustomObject(kubeConfGroup, kubeConfVersion, namespace, kubeConfPlural, kubeMiddlewareName, gson.fromJson(jsonTraefikMiddleware, obj.getClass()), null, null);

Don't forget that the examples are of "works on my machine" and "from the top of my head".

3

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.