Complex nested parsing with Remap (waninfo)

[binarylogic]

From a user in our chat:

Given this event:

{
	"waninfo": "name=wan1,bytes=6042294/1139384031,packets=49527/1902253;name=wan2,bytes=6042294/1150716872,packets=49527/1909091;"
}

And this Remap script:

.message.waninfo = split(.message.waninfo, ";")

The following is produced:

{
	"waninfo": [
	      "name=wan1,bytes=6042294/1139384031,packets=49527/1902253",
	      "name=wan2,bytes=6042294/1150716872,packets=49527/1909091",
	      ""
	  ]
}

How would a user go about parsing each n items in the result? Or, better yet, what is the best UX we could provide to parse this?

[JeanMertz]

I suspect there is a high likelihood that people want to iterate over maps or arrays. I'm not talking about traversing nested maps/arrays, but simply iterating over a single-level map/array.

I'd personally expect something like this to work (written out over multiple lines for clarity):

waninfos = compact(split(.message.waninfo, ";"))
// ["...", "..."]

indexed_waninfos = enumerate(waninfos)
// {"0": "...", "1": "..." }

for index, info in indexed_waninfos {
	index = to_int(index) // 0, 1
	.message.waninfo[index] = parse_key_value(info)
}

Or:

for index, info in enumerate(compact(split(.message.waninfo, ";"))) {
	.message.waninfo[to_int(index)] = parse_key_value(info)
}

For this to work, there are three additions we need to make:

• add the enumerate function
• add for-loop statements (I have a draft for this locally)
• add dynamic path indexing

[JeanMertz]

After playing around with it locally, I think we should just "automatically" add an index if a second argument is provided when iterating an array:

array without index:

for value in ["foo", "bar"] {
	value // "foo", "bar"
}

array with index:

for index, value in ["foo", "bar"] {
	index // 0, 1
	value // "foo", "bar"
}

map:

for key, value in { "foo": "bar" } {
	key // "foo"
	value // "bar"
}

[JeanMertz]

Closing this, as we now support iterating arbitrary objects/arrays in VRL (starting with for_each, map_values and map_keys).