Nested fields are not JSON

[jamtur01]

I'm using Logstash to parse out the sidekiq logs entries and everything is great until I get to the nested fields, args for example. This is sent as a hash not as JSON. So args is:

{"job_class"=>"ActionMailer::DeliveryJob", "job_id"=>"b6a0d138-1da9-4706-8b6e-06b42e81e75f", "provider_job_id"=>nil, "queue_name"=>"mailers", "priority"=>nil, "arguments"=>["AccountMailer", "password_reset", "deliver_now", {"_aj_globalid"=>"gid://backend/User/18"}, "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiJdLCJleHAiOjE1MDk1NDgzODEsImlhdCI6MTUwOTU0NjU4MSwiaXNzIjoiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiIsImxvY2siOjE1MDYwNDg4MTYsInNjb3BlIjoicmVzZXQiLCJzdWIiOiIxOCJ9.BdDtbfE7CZWkW0o-nzcWVK_PzGU9Lk-bSb8wUVYuUYs"], "executions"=>0, "locale"=>"en"}

Rather than:

{\"job_class\":\"ActionMailer::DeliveryJob\",\"job_id\":\"b6a0d138-1da9-4706-8b6e-06b42e81e75f\",\"provider_job_id\":null,\"queue_name\":\"mailers\",\"priority\":null,\"arguments\":[\"AccountMailer\",\"password_reset\",\"deliver_now\",{\"_aj_globalid\":\"gid://backend/User/18\"},\"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiJdLCJleHAiOjE1MDk1NDgzODEsImlhdCI6MTUwOTU0NjU4MSwiaXNzIjoiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiIsImxvY2siOjE1MDYwNDg4MTYsInNjb3BlIjoicmVzZXQiLCJzdWIiOiIxOCJ9.BdDtbfE7CZWkW0o-nzcWVK_PzGU9Lk-bSb8wUVYuUYs\"],\"executions\":0,\"locale\":\"en\"}

So in irb for example:

irb(main):013:0> args = {"job_class"=>"ActionMailer::DeliveryJob", "job_id"=>"b6a0d138-1da9-4706-8b6e-06b42e81e75f", "provider_job_id"=>nil, "queue_name"=>"mailers", "priority"=>nil, "arguments"=>["AccountMailer", "password_reset", "deliver_now", {"_aj_globalid"=>"gid://backend/User/18"}, "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiJdLCJleHAiOjE1MDk1NDgzODEsImlhdCI6MTUwOTU0NjU4MSwiaXNzIjoiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiIsImxvY2siOjE1MDYwNDg4MTYsInNjb3BlIjoicmVzZXQiLCJzdWIiOiIxOCJ9.BdDtbfE7CZWkW0o-nzcWVK_PzGU9Lk-bSb8wUVYuUYs"], "executions"=>0, "locale"=>"en"}
irb(main):014:0> args.class
=> Hash
irb(main):015:0> argsj = JSON.parse(args)
TypeError: no implicit conversion of Hash into String
	from /Users/james/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/json-2.0.2/lib/json/common.rb:156:in `initialize'
	from /Users/james/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/json-2.0.2/lib/json/common.rb:156:in `new'
	from /Users/james/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/json-2.0.2/lib/json/common.rb:156:in `parse'
	from (irb):15
	from /Users/james/.rbenv/versions/2.2.2/bin/irb:11:in `<main>'
irb(main):016:0> argjs = JSON.parse(args.to_json)
=> {"job_class"=>"ActionMailer::DeliveryJob", "job_id"=>"b6a0d138-1da9-4706-8b6e-06b42e81e75f", "provider_job_id"=>nil, "queue_name"=>"mailers", "priority"=>nil, "arguments"=>["AccountMailer", "password_reset", "deliver_now", {"_aj_globalid"=>"gid://backend/User/18"}, "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiJdLCJleHAiOjE1MDk1NDgzODEsImlhdCI6MTUwOTU0NjU4MSwiaXNzIjoiaHR0cHM6Ly9iYWNrZW5kLmVtcGF0aWNvLm9yZy9hdXRobiIsImxvY2siOjE1MDYwNDg4MTYsInNjb3BlIjoicmVzZXQiLCJzdWIiOiIxOCJ9.BdDtbfE7CZWkW0o-nzcWVK_PzGU9Lk-bSb8wUVYuUYs"], "executions"=>0, "locale"=>"en"}

[iMacTia]

Hey @jamtur01,
thank you for reporting this issue and apologies for the delay.
I've just had a quick look at the code and I found that this line might cause the issue: https://github.com/iMacTia/sidekiq-logstash/blob/master/lib/sidekiq/logging/shared.rb#L38

Unfortunately I'm not using this gem on any project at the moment, so I'm not able to test further or provide a fix at the moment.

I'd suggest you to try monkey-patching the gem removing that line and checking if it changes the behaviour or not. We might eventually make it optional upon configuration 👍