JSON Output is Formatted Incorrectly

[ajschmidt8]

Expected Behavior

results.json file should have valid JSON formatting like this:

[
  {"type":"Metric","data":{"name":"http_reqs","type":"counter","contains":"default","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_reqs"},
  {"type":"Point","data":{"time":"2018-08-07T23:14:42.290209028-04:00","value":1,"tags":{"group":"","method":"GET","name":"http://test.loadimpact.com","proto":"HTTP/1.1","status":"200","url":"http://test.loadimpact.com"}},"metric":"http_reqs"},
  {"type":"Metric","data":{"name":"http_req_duration","type":"trend","contains":"time","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_req_duration"},
  {"type":"Point","data":{"time":"2018-08-07T23:14:42.290209028-04:00","value":6.491032,"tags":{"group":"","method":"GET","name":"http://test.loadimpact.com","proto":"HTTP/1.1","status":"200","url":"http://test.loadimpact.com"}},"metric":"http_req_duration"},
  {"type":"Metric","data":{"name":"http_req_blocked","type":"trend","contains":"time","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_req_blocked"},
  {"type":"Point","data":{"time":"2018-08-07T23:14:42.290209028-04:00","value":4.977785,"tags":{"group":"","method":"GET","name":"http://test.loadimpact.com","proto":"HTTP/1.1","status":"200","url":"http://test.loadimpact.com"}},"metric":"http_req_blocked"},
  {"type":"Metric","data":{"name":"http_req_connecting","type":"trend","contains":"time","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_req_connecting"}
]

Actual Behavior

results.json is invalid as seen below. it is not an array of objects...just simply lines of objects

{"type":"Metric","data":{"name":"http_reqs","type":"counter","contains":"default","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_reqs"}
{"type":"Point","data":{"time":"2018-08-07T23:14:42.290209028-04:00","value":1,"tags":{"group":"","method":"GET","name":"http://test.loadimpact.com","proto":"HTTP/1.1","status":"200","url":"http://test.loadimpact.com"}},"metric":"http_reqs"}
{"type":"Metric","data":{"name":"http_req_duration","type":"trend","contains":"time","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_req_duration"}
{"type":"Point","data":{"time":"2018-08-07T23:14:42.290209028-04:00","value":6.491032,"tags":{"group":"","method":"GET","name":"http://test.loadimpact.com","proto":"HTTP/1.1","status":"200","url":"http://test.loadimpact.com"}},"metric":"http_req_duration"}
{"type":"Metric","data":{"name":"http_req_blocked","type":"trend","contains":"time","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_req_blocked"}
{"type":"Point","data":{"time":"2018-08-07T23:14:42.290209028-04:00","value":4.977785,"tags":{"group":"","method":"GET","name":"http://test.loadimpact.com","proto":"HTTP/1.1","status":"200","url":"http://test.loadimpact.com"}},"metric":"http_req_blocked"}
{"type":"Metric","data":{"name":"http_req_connecting","type":"trend","contains":"time","tainted":null,"thresholds":[],"submetrics":null,"sub":{"name":"","parent":"","suffix":"","tags":null}},"metric":"http_req_connecting"}

Steps to Reproduce the Problem

1. Run: k6 run -r json=results.json github.com/loadimpact/k6/samples/http_get.js
2. Examine results.json

Specifications

• Version: k6 v0.22.1
• Platform: Linux (Ubuntu 18.04)

[na--]

jq is so good at handling this type of output that I hadn't noticed until now 😅 Still, it's a bug and we'll have to do a somewhat breaking change to fix it, but (edit: see comments below) I think it might be worth it to also leave the current behavior. It would be hidden behind a flag of course, but it might be useful if people already depend on it and it would probably be easier for some streaming json parsers to deal with a bunch of object than with an enormous array.

[MichaelWeirWeather]

Having the output as separate chunks of JSON makes it a bit easier to process the file a line at a time. Sometimes the output files are quite large.

[alexgagnon]

I'd like to second this as an issue. Although @MichaelWeirWeather is correct if you are following output, it makes it difficult to parse it into JS out of the box. If you're dealing with the outputted file, you still need to load in the file and then navigate by line. For example, if you want to use the output in the browser either you do some post-processing on the created file, or load in the file using FileReader api and then use JSON.parse on each line, neither of which is ideal. This issue came up because I'm trying to use the output in a D3 chart.

[na--]

Re-reading this issue, I think this would be better implemented as an optional configuration in the current JSON metrics collector. That way we won't have to make a braking change and users who depend on the current output (which in some instances would be easier to parse than the full-array one) won't be affected. But if you want to output proper JSON arrays, you should just be able to set a flag and have the output that way.

Implementation wise, it would be best to do this after #587 is done, but we can also probably do it as a quick fix before that and have it accessible from an environment variable like K6_JSON_FORMAT=array k6 run --output json script.js

[benc-uk]

JSON array would be great, I can't process the result without pre-processing via jq
jq -s '.' ./result.json > result-array.json which is a huge pain

[enc]

Hi, since no one mentioned it. This is not a bug. The format is well know as newline delimited JSON. It is used in two cases. First you want to process that data before it is complete. Aka streaming data. Same use case as partitioning. All this is working pretty nice with command line tools. If we use this format.

It's kind of the same use case. But once the file is huge enough, like you are running a high performance test for 2 hours, it can become difficult to parse the whole file at once. The best way is to use one of the tools from above.

[na--]

@enc is right and this is not a bug, seems we have just forgotten to remove the label. Because of the reasons he mentioned and also because we don't want to make breaking changes, we are not going to change the current default behavior.

That said, I think having the option to generate well-formed JSON arrays may be useful to some people. 🤷‍♂️ It will be easy to implement and I imagine that some streaming JSON parsers might not support newline delimited JSON. The only reason we haven't implemented that option so far is that the JSON output currently doesn't have support for any options... 😅 That should be easy to add after the recent output refactoring (#1869), and we want to make other things configurable anyway:

k6/output/json/json.go

Lines 36 to 37 in a33b054

	// TODO: add option for emitting proper JSON files (https://github.com/k6io/k6/issues/737)
	const flushPeriod = 200 * time.Millisecond // TODO: make this configurable

[na--]

Hey, let's keep this open - as I mentioned, while it's not a bug, it's probably still a minor improvement if we add an option to output well-formed JSON arrays 🙂

[oleiade]

The maintainers' team has discussed this and agreed to close it in favor of grafana/k6-docs#1191.

The JSON output behaves as intended by using the JSON Lines format, but it is not well enough documented, and we want to address that instead 🙇🏻