Preserving comments, etc. with round trip decode/encode.

[scott-fleischman]

A few times now I have wanted to preserve comments, whitespace and formatting when doing a round trip, something like: encode . transform . decode.

The use case is a tool that updates some information in configuration files. It is common that configuration files have comments and formatting which help the human readability of the document. It is disappointing to do an update which strips comments and changes formatting and whitespace.

Would you be willing to support such a feature in this language (preservation of "trivia" such as comments and whitespace)? If so, how would you recommend going about this? I would be willing to contribute if there is a reasonable chance of it getting merged in.

It looks like https://hackage.haskell.org/package/yaml-0.8.23.1/docs/Text-Libyaml.html provides a good starting point for this kind of addition.

Thanks for your consideration!

[snoyberg]

I'm open to it, but highly doubt it can be done without massive amounts of work. We're using a C library under the surface to parse the yaml itself, and IIRC it does not support comments at all.

[xtianjohns]

Hey @snoyberg, when you talk about massive amounts of work, are you talking about an implementation in this package that supports comment preservation without a corresponding feature set in libyaml? Or are you addressing the work that would be required in libyaml to support comments? Both?

There is yaml/libyaml#42 for addressing comment preservation in C - if a feature could be delivered, would you gauge the work required to deliver the additional comment feature set in this package as "massive"?

Frankly, I'm new to Haskell but stumbled upon this from commercialhaskell/stack#1933 (which if closed would provide for a nifty feature when installing new dependencies in Stack). While this lack of a capability doesn't seem to be a major problem for consumers of this module, it does seem like implementing it would be useful!

I'd love to help! If we can close out yaml/libyaml#42, would you be open to PRs and perhaps some guidance on the corresponding updates here?

[snoyberg]

It's mostly work in libyaml. If the upstream library gets support for comments, it's significantly simpler to do that here. However, the current primary API (based on the aeson API) provides no method for dealing with comments, so we'd have to provide an alternative API. The underlying conduit-based API in Text.Libyaml should be much more easily extended.

CCing @NorfairKing, who may be mentoring a GSoC project related to this.

[NorfairKing]

@xtianjohns and @scott-fleischman There is a GSoC project about this: https://summer.haskell.org/ideas.html#format-preserving-yaml

If you can wait a summer, I'm quite certain this thing will exist by the end of the year.

[snoyberg]

Closing this as currently out-of-scope for the yaml package, and deferring to the GSoC project instead.

[scott-fleischman]

Was anything from GSoC merged into this project relating to this issue?

[snoyberg]

The plan was always to put any GSoC code into its own project. That project was creating a new parser, separate from the C parser used by this library.

[NorfairKing]

The GSoC project failed, by the way.