Improve CSV parsing to allow quoted and escaped fields #734
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The RadioReference talkgroup CSV output looks like the following:
However, when trunk-recorder parses the CSV, it does not strip out the quotes, which leads to some malformed entry errors, as well as the call JSON to have syntax errors like the following:
This PR fixes that error by switching to the Boost
escaped_list_separator
class, which is able to handle text being quoted and escaped. The previous classchar_separator
did not handle this.Both comma-separated values and tab-separated values are still supported by this parser. I've done some testing to make sure everything still works.
Besides this, I made a tiny change to the Dockerfile to make compiling trunk-recorder use all available CPU cores for faster compilation.