Stream IOs

[Blacksmoke16]

Description

Resolves #14
Resolves #15
Resolves #17
Resolves #19

• Refactors much of oq to more easily support additional formats.
  • .deserialize for Xml converter will follow later.
• Follows up on Refactors to optimize IO #6 to extend the streaming of data to YAML and XML in addition to JSON.
  • This makes oq much much more memory efficient as nothing is loaded into memory.

Big shoutout to @asterite for the gist that made this PR possible.

TODO

• Support #text keys again
• Add specs for each format
• Make sure current jq features still work

[Blacksmoke16]

@asterite would you mind giving this a review if you get a minute?

[asterite]

@Blacksmoke16 Sure! I'll do it later today.

[asterite]

Looks good! Just added one suggestion.

Eventually if I have time I can try to see if I can write code to handle YAML aliases and anchors. Sounds like something fun and challenging that I would enjoy, but it all depends on whether I'll have time to do it.

[asterite]

Oh, wow, I didn't remember you could access a named tuple with a string, I though only symbol was valid.

[asterite]

This is a bit of a smell, it means you can't call methods of this module concurrently. Is there a way to pass this at_root value to all methods that need it so it's not shared at all?

[Blacksmoke16]

Yea I could probably just pass it between the methods versus making it a class var.

EDIT: Or maybe it would be worth making these structs and using an initializer. Which could also be useful for setting ivars for the arg values?

[asterite]

Oh, I edited my comment and some part was gone. I forgot to say that this is probably a minor issue because this is a one-shot app meaning that you won't be running this concurrently, but it's still a good design pattern to avoid sharing stuff.

I thought about making it a class/struct but you'd still have the same problem that if someone uses that instance concurrently then the @at_root instance var would cause conflicts. I think there's no other way around passing it between methods. And instead of passing a lot of args you can consider creating a class holding these shared things and passing that instead.

But again, this is probably not needed for this kind of app 😊