vRO8 Dump Workflow Logs

How to get and filter workflow logs from vRO8 via API

Introduction

vRealize Orchestrator version 8 logs include workflow item names a separate log entries. Let's compare the output of the same workflow with the legacy version 7 client:

While this extra information is useful when debugging to find the exact location of code running, it makes reading the log harder. You may or may not like the new interface.

vRO API

The vRealize Orchestrator API provides a REST API to interact with vRO. The Workflow Run Service allows us to get workflow logs.

We need to know the Id of the workflow (workflowId) and the executionId to get the logs. Below is a sample output of
curl -sSk -H 'Content-Type: application/json' -H "Authorization: Bearer $access_token" 'https://vra8.corp.local/vco/api/workflows/a64af743-a5c5-4420-96d3-5dbf83a27b71/executions/6333c049-9abc-4ea1-955b-8ac942a127f0/syslogs' | jq .

 1{
 2  "logs": [
 3    {
 4      "entry": {
 5        "origin": "system",
 6        "short-description": "__item_stack:/item1",
 7        "time-stamp": "2022-07-28T08:33:50.319+00:00",
 8        "time-stamp-val": 1658997230319,
 9        "severity": "info"
10      }
11    },
12    {
13      "entry": {
14        "origin": "system",
15        "short-description": "0",
16        "time-stamp": "2022-07-28T08:33:50.321+00:00",
17        "time-stamp-val": 1658997230321,
18        "severity": "info"
19      }
20    },
21    {
22      "entry": {
23        "origin": "system",
24        "short-description": "__item_stack:/item3",
25        "time-stamp": "2022-07-28T08:33:52.326+00:00",
26        "time-stamp-val": 1658997232326,
27        "severity": "info"
28      }
29    },
30    {
31      "entry": {
32        "origin": "system",
33        "short-description": "__item_stack:/item1",
34        "time-stamp": "2022-07-28T08:33:52.331+00:00",
35        "time-stamp-val": 1658997232331,
36        "severity": "info"
37      }
38    },
39    ...
40        {
41      "entry": {
42        "origin": "system",
43        "short-description": "Completed.",
44        "time-stamp": "2022-07-28T08:34:10.448+00:00",
45        "time-stamp-val": 1658997250448,
46        "severity": "warning"
47      }
48    },

There are options to fine-tune this query with contitions. We'll use two conditions, one will filter by severity the other by timestamp:
conditions=severity=info will get all logs above info level (info, warning, error)
conditions=timestamp>1658997232331 will get all logs after 2022-07-28T08:33:52.331+00:00

We'll also limit the number of logs fetched with a single request to avoid API overloading:
maxResult=100

The Code

I chose Python as I plan to run the same code from vRO and standalone. The inputs are vRO hostname, username, password, workflowname (full path), severity and executionId. Severity is info by default, and executionId is the last execution if omitted.

urllib is used as it is available in vRO Python by default, no additional package installation required.

First we search for the workflow by name, and check (later) the containing folder to make sure we query the correct one.

1wfLinks = doReq("GET", "/vco/api/catalog/System/Workflow?conditions=name=" + parse.quote(workflowname))["link"]

If no executionId provided, we search for the latest one:

1executionLinks = doReq("GET", "/vco/api/catalog/System/WorkflowExecution/?maxResult=1&sortOrders=-startDate&conditions=workflowId=" + workflow["id"])["link"]

The main loop will check if the workflow is still running (this will allow to follow the logs ) and if there are any logs to print left. It will poll the running workflow every second for new log entries. Not running (completed, cancelled) workflow logs are fetched and printed (by 100 entries).

1while state == "running" or timestamp != lasttimestamp:
2    timestamp = lasttimestamp
3    lasttimestamp = getLogs(workflow["id"], executionId, severity, timestamp)
4    state = doReq("GET", "/vco/api/workflows/" + workflow["id"] + "/executions/" + executionId)["state"]
5    if state == "running" and timestamp == lasttimestamp:
6        time.sleep(1)

You can download the com.vro.python package containing the workflow from GitHub: https://github.com/kuklis/vro8-packages

Make sure to adjust workflow variables to your environment (hostname, credentials).

See the output below. We prefix each log entry with the timestamp, severity and workflow item number to preserve useful information:

Running it standalone

The motivation to create a standalone Python program is to follow long-running workflow output live. The vRO workflow above does logging in a single Python scriptable task, and will provide logs once the task is finished, at once. We might break it into multiple steps to dump logs from time to time, but we'd introduce the default workflow item number logging problem again.

We keep the default handler() function but will feed it from command line arguments argv.

 1def main():
 2    try:
 3        inputs = {
 4        "hostname": sys.argv[1],
 5        "username": sys.argv[2],
 6        "password": sys.argv[3],
 7        "workflow": sys.argv[4],
 8        "color_warn": "\033[93m",
 9        "color_fail": "\033[91m",
10        "color_end": "\033[0m"
11        }

Running the code in console also allows us some coloring:

The code can be downloaded from https://github.com/kuklis/vro8-python/blob/main/followVROlogs.py
The com.vro.python package containing the workflow is available at: https://github.com/kuklis/vro8-packages