Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update JSON validation test to handle UTF-8 Byte-Order Mark (BOM) error #89

Closed
andrewki-msft opened this issue Dec 13, 2018 · 2 comments

Comments

@andrewki-msft
Copy link
Contributor

andrewki-msft commented Dec 13, 2018

The JSON validation test fails if Byte-Order Mark (BOM) is present in the file and throws an ambiguous error. The validation test should be updated to warn the user to take corrective action to remove BOM. It is recommended that BOM should not be used in UTF-8.

https://tools.ietf.org/html/rfc3629#page-6

Steps to reproduce:

  1. Use notepad++ or some text editor that prepends a UTF-8 BOM
  2. Save a new workbook as UTF-8
  3. Execute jsonValidationTest.js
  4. Test will fail, throwing an error saying unexpected "blank" token was found
SyntaxError: Unexpected token  in JSON at position 0
      at TryParseJson (test/jsonValidationTest.js:149:16)
      at validateJsonStringAndGetObject (test/jsonValidationTest.js:115:15)
      at results.filter.forEach.file (test/jsonValidationTest.js:55:21)
      at Array.forEach (<anonymous>)
      at browseDirectory (test/jsonValidationTest.js:54:18)
      at next (test/jsonValidationTest.js:96:31)
      at /home/travis/build/Microsoft/Application-Insights-Workbooks/test/jsonValidationTest.js:102:25
      at next (test/jsonValidationTest.js:96:31)
      at /home/travis/build/Microsoft/Application-Insights-Workbooks/test/jsonValidationTest.js:106:21
      at FSReqCallback.oncomplete (fs.js:161:5)

Fix diff for the workbook to remove the BOM:

@@ -1,4 +1,4 @@                                                                                                                                                       
-<EF><BB><BF>{                                                                                                                                                        
+{                                                                                                                                                                    
   "version": "Notebook/1.0",
@andrewki-msft andrewki-msft changed the title Update JSON validation test to handle UTF8 BOM error Update JSON validation test to handle UTF-8 Byte-Order Mark (BOM) error Dec 13, 2018
@gardnerjr
Copy link
Collaborator

I'm tryng to add @ericc1103 to who it is assigned to and somehow it autocompleted to 04diiguyi? wth github?

@gardnerjr gardnerjr assigned ericc1103 and unassigned 04diiguyi Dec 13, 2018
@gardnerjr
Copy link
Collaborator

gardnerjr commented Jan 16, 2019

from an ancient archive of nodejs 0.x:

nodejs/node-v0.x-archive#1918

it looks like the fix is to just strip it out after reading from the file:
body = body.replace(/^\uFEFF/, '');

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants