Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open link-verifier target files with encoding="utf8", errors='ignore' options #29

Merged

Conversation

paulbartell
Copy link
Contributor

  • Open link-verifier target files with encoding="utf8", errors='ignore' options
  • Print each file path that is processed to stdout for debugging purposes.

@paulbartell paulbartell requested a review from aggarw13 June 2, 2021 18:19
… options

Print each file name that is processed to stdout.
@paulbartell paulbartell force-pushed the paulbartell/fix-encoding-link-verifier branch from c8087ea to 75419e6 Compare June 2, 2021 18:19
@paulbartell
Copy link
Contributor Author

@aggarw13 here's a link to the file which triggered this issue: https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/ac2c383bc14b2577101cad238c8779f4d9c14d6c/portable/GCC/ARM7_AT91SAM7S/lib_AT91SAM7X256.h#L1469

Specifically, the "µ" character in "µsecond".

@paulbartell paulbartell requested a review from aggarw13 June 2, 2021 19:18
@paulbartell paulbartell merged commit 53ff74c into FreeRTOS:main Jun 2, 2021
@paulbartell paulbartell deleted the paulbartell/fix-encoding-link-verifier branch June 2, 2021 20:24
@paulbartell
Copy link
Contributor Author

To add some additional background:
Bytes from 0x80 to 0xBF are invalid starting bytes in UTF-8, however they are valid in other character sets. Some vendor code uses other fixed-width 8-bit character sets with valid characters in the range of 0x80 to 0xBF. This change drops those characters silently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants