Custom day 2 actions for Tanzu in Aria Automation
Implementing a Kubernetes REST API client for Tanzu clusters
Table of Contents
Introduction
In my previous post on Running Helm in vRO we discussed that Aria Automation provides a certificate based authentication to the deployed Tanzu clusters via API at /cmx/api/resources/k8s/clusers/{id}/kube-config
The same Kubeconfig can be downloaded on the GUI:
We will use the access provided by the certificates in this Kubeconfig to connect to the Kubernetes REST API of the cluster.
Authentication by client certificates within Orchestrator plugin
The HTTP REST plugin of vRO is capable of using client certificates as authentication method. However, this feature is not available for transient hosts:
- Transient hosts passed between workflow items as input/output might not work in all cases. Transient hosts rely on workflow cache, which doesn't work when asynchronous workflows are started, for example. Nested workflows might also fail.
- Only GET and HEAD requests get redirected automatically. URL redirection uses the default strategy.
- Host name verification is not supported.
- Client certificate authentication is not supported.
I do not want to create persistent hosts in the plugin inventory as there may be a large number of clusters and this would lead a long list of host in the inventory, plus we have to make sure these hosts (with the certificates) are created and removed within the cluster lifecycle.
Downloading the certificates with Python
So let's switch to Python and see what needs to be done. In order to use client certificate authentication, we need to save the certificates into files and pass them to ssl module context.
In Getting token within vRO polyglot code I described a method to acquire a token to use with Orchestrator/Automation. We can also gather Automation external URL from Orchestrator configuration so no need to provide it as an input. Please note this will not work in a multitenant environment where the actual Automation URL contains the tenant name. The Kubeconfig is in yaml format, but there is no yaml parsing module available by default in the Orchestrator implementation so we do a very basic string parsing to capture the certificates. We could import and place into a ZIP package the missing module but I wanted to keep the solution as simple as possible. Here is the code that prints the certificates within the Kubeconfig:
1import json, http, ssl
2from urllib.parse import urlparse
3from base64 import b64decode
4
5def handler(context, inputs):
6 jsonOut=json.dumps(inputs, separators=(',', ':'))
7 print("Inputs were {0}".format(jsonOut))
8
9 headers = {'Authorization': "Bearer " + context["getToken"]() }
10
11 conn = http.client.HTTPConnection("localhost", 8280)
12 conn.request("GET", "/vco/api/server-configuration/settings", '', headers)
13 res = conn.getresponse()
14 print("HTTP status: " + str(res.status))
15 data = res.read().decode("ascii")
16 print("server configuration settings: " + data)
17 settings = json.loads(data)
18 hostname = urlparse(settings["vco.csp.external.url"]).hostname
19
20 conn = http.client.HTTPSConnection(hostname, context=ssl.SSLContext())
21 conn.request("GET", "/cmx/api/resources/k8s/clusters/" + inputs["clusterId"] + "/kube-config", "", headers)
22 res = conn.getresponse()
23 print("HTTP status: " + str(res.status))
24 data = res.read().decode("ascii")
25 if res.status != 200:
26 raise Exception("Could not fetch Kubeconfig of cluster with id " + inputs["clusterId"] + ". " + data)
27 for line in data.splitlines():
28 key, sep, value = line.strip().partition(":")
29 if key == "certificate-authority-data":
30 print(b64decode(value.strip(' "')).decode("ascii"))
31 if key == "client-certificate-data":
32 print(b64decode(value.strip(' "')).decode("ascii"))
33 if key == "client-key-data":
34 print(b64decode(value.strip(' "')).decode("ascii"))
35 return "done"
This code needs the ID of cluster within Automation, which we can get by Automation API:
1$ curl -ksS -H "Authorization: Bearer $access_token" 'https://vra8.corp.local/cmx/api/resources/k8s/clusters' |
2> jq -r '.content[]|[.id,.name]|@tsv'
30f882cc4-8a84-4cf5-aa38-597e028d2a9d tst
42fda385d-8b80-40e4-b277-be747804ff1a t3
5f9f61b48-a11d-4eb8-ae34-cd895af5c765 r2
62f0faa48-cd89-4831-801f-b0e690c27465 t1
7f2dd9fe9-3f5e-4b45-a468-3f51cc2af14c t2
8adad53a3-c090-44f0-8a65-e6313466cbc2 api5
An alternative method is to peek on the GUI and get the ID from the URL:
Here is the output of the code, with the certificates printed:
kubeRestClient action
Let's make some helper functions for the final action. The headers parameter contains the Bearer token for authentication.
The followin function gets the URL of Automation (see the limitations remark of this method in the previous section, not to be used in a multitenant environment):
1def get_hostname(headers):
2 conn = http.client.HTTPConnection("localhost", 8280)
3 conn.request("GET", "/vco/api/server-configuration/settings", '', headers)
4 res = conn.getresponse()
5 data = res.read().decode("ascii")
6 settings = json.loads(data)
7 return urlparse(settings["vco.csp.external.url"]).hostname
This function gets the IP and port of Kubernetes control plane:
1def get_address(hostname, headers, clusterId):
2 conn = http.client.HTTPSConnection(hostname, context=ssl.SSLContext())
3 conn.request("GET", "/cmx/api/resources/k8s/clusters/" + clusterId, "", headers)
4 res = conn.getresponse()
5 data = res.read().decode("ascii")
6 if res.status != 200:
7 raise Exception("Could not find cluster with id " + clusterId + ". " + data)
8 return urlparse(json.loads(data)["address"])
The next function saves the cluster client authentication certificates.
1def save_certs(hostname, headers, clusterId):
2 conn = http.client.HTTPSConnection(hostname, context=ssl.SSLContext())
3 conn.request("GET", "/cmx/api/resources/k8s/clusters/" + clusterId + "/kube-config", "", headers)
4 res = conn.getresponse()
5 data = res.read().decode("ascii")
6 if res.status != 200:
7 raise Exception("Could not fetch Kubeconfig of cluster with id " + clusterId + ". " + data)
8 for line in data.splitlines():
9 key, sep, value = line.strip().partition(":")
10 if key == "certificate-authority-data":
11 with open("/root/ca.crt", "w") as f:
12 f.write(b64decode(value.strip(' "')).decode("ascii"))
13 if key == "client-certificate-data":
14 with open("/root/cert.crt", "w") as f:
15 f.write(b64decode(value.strip(' "')).decode("ascii"))
16 if key == "client-key-data":
17 with open("/root/cert.key", "w") as f:
18 f.write(b64decode(value.strip(' "')).decode("ascii"))
The main function gathers input variables, identify kubernetes API address and saves the certificates, then calls K8S API with the provided input.
1import json, http, ssl
2from urllib.parse import urlparse
3from base64 import b64decode
4
5
6def handler(context, inputs):
7 jsonOut=json.dumps(inputs, separators=(',', ':'))
8 print("Inputs were {0}".format(jsonOut))
9 clusterId = inputs["clusterId"]
10 method = inputs["method"] or "GET"
11 path = inputs["path"]
12 payload = inputs["payload"]
13
14 headers = {'Authorization': "Bearer " + context["getToken"]() }
15 hostname = get_hostname(headers)
16 address = get_address(hostname, headers, clusterId)
17 save_certs(hostname, headers, clusterId)
18 context = ssl.create_default_context(cafile="/root/ca.crt")
19 context.load_cert_chain(certfile="/root/cert.crt", keyfile="/root/cert.key")
20
21 conn = http.client.HTTPSConnection(address.hostname, address.port, context=context)
22 conn.request(method, path, payload)
23 res = conn.getresponse()
24 data = res.read().decode("utf-8")
25 print(data)
26
27 if res.status >= 400:
28 raise Exception("Error calling Kubernetes API of cluster with id " + clusterId + ": " + data)
29 else:
30 return data
Sample day 2 resource action: Create Namespace
Let's create a simple workflow with kubeRestClient action that creates a namespace via K8S API.
First, we need to extract the Automation ID of the cluster:
1var resourceProperties = System.getContext().getParameter("__metadata_resourceProperties");
2var id = resourceProperties.id;
3clusterId = id.substr(id.lastIndexOf("/") + 1);
Then call the REST client action:
1var payload = {
2 "kind": "Namespace",
3 "apiVersion": "v1",
4 "metadata": {
5 "name": namespace
6 }
7}
8
9System.getModule("com.test.tanzu")
10 .kubeRestClient(clusterId, "POST", "/api/v1/namespaces", JSON.stringify(payload));
Define a resource action with the workflow:
Testing the custom action
Let's try to create a new namespace:
1$ KUBECONFIG=~/Downloads/t1 kubectl get ns
2NAME STATUS AGE
3default Active 20d
4kube-node-lease Active 20d
5kube-public Active 20d
6kube-system Active 20d
7ns1 Active 6m38s
8vmware-system-auth Active 20d
9vmware-system-cloud-provider Active 20d
10vmware-system-csi Active 20d
Let's try to create the same namespace again:
This time it failed as the namespace already exists.
Summary
The presented action is capable of interacting with the managed Tanzu Kubernetes cluster via its REST API, allowing to build further automation workflows (day 2 actions, Event Broker Subscriptions, etc.) in an easy way.
Download the com.test.tanzu package containing the sample workflow and actions from GitHub: https://github.com/kuklis/vro8-packages