Make sure that all file loading happens via SourceMap #63525
Conversation
rust-highfive
added
the
S-waiting-on-review
| cx.source_map().new_source_file(file.into(), src); | ||
|
|
||
| base::MacEager::expr(cx.expr_str(sp, interned_src)) | ||
| match cx.source_map().load_binary_file(&file) { |
There was a problem hiding this comment.
I am not sure whether include_str should normalize file contents... This approach does not do normalization, just like the previous version.
There was a problem hiding this comment.
I'm pretty sure it should.
The content included by include_str is still text / source code, if it's a long multi-line string encoded on Windows and stored in a file with UTF-8 BOM, then all those \r\ns and BOM shouldn't get into the string for the same reason why any other (non-include_bytes) inclusion method filters them away from string literals.
Am I right that #62948 applied the normalization to include_str as well and crater found no regression?
This should give as the right to make the change (in this PR or in #62948, not much difference).
|
r=me if you don't plan to address the |
petrochenkov
added
S-waiting-on-author
I think that's not the case
I've added commit that adds normalization to This is technically a breaking change, so we should probably FCP it? |
Actually, even this is not true: |
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
matklad
force-pushed
the
centraliza-file-loading
branch
2 times, most recently
from
a5d9445
to
9f09e91
Compare
That example kinda makes me rethink the prohibition of bare Actually, we may want to prohibit only |
It looks like Git treats files with lone CR as binary and doesn't convert them: I also begin to feel that forbidding lone CR causes more problems that it solves. Sole |
rustbot
added
S-waiting-on-review
|
Let's just keep the status quo behavior of I wanted to write out some possible use cases for various flavor of normalization, but I just need to go and sleep now. TLDR is that BOM+EOL normalization is the common case requirement for text ( |
matklad
force-pushed
the
centraliza-file-loading
branch
from
9f09e91
to
0885a8d
Compare
That way, callers don't need to repeat "let's add this to sm manually for tracking dependencies" trick. It should make it easier to switch to using `FileLoader` for binary files in the future as well
matklad
force-pushed
the
centraliza-file-loading
branch
from
0885a8d
to
14bc998
Compare
|
based on #63525 (comment) @bors r=petrochenkov I've removed the normalization commit, but I've added the test that normalization doesn't happen. I think the way forward here is to complete #62948 first, and then have a separate discussion/crater runs about
I feel this can be build on top of fn my_text_file_with_crlf() -> &'static str {
unsafe {
std::str::from_utf8_unchecked(&include_bytes!("my_text_file_with_crlf.txt")[..])
}
}
#[test]
fn my_text_file_is_acutualy_a_text_file() {
std::str::from_utf8(&include_bytes!("my_text_file_with_crlf.txt")[..])
.unwrap()
} |
|
📌 Commit 14bc998 has been approved by |
bors
added
S-waiting-on-bors
|
💡 This pull request was already approved, no need to approve it again.
|
|
📌 Commit 14bc998 has been approved by |
…petrochenkov Make sure that all file loading happens via SourceMap That way, callers don't need to repeat "let's add this to sm manually for tracking dependencies" trick. It should make it easier to switch to using `FileLoader` for binary files in the future as well cc rust-lang#62948 r? @petrochenkov
…petrochenkov Make sure that all file loading happens via SourceMap That way, callers don't need to repeat "let's add this to sm manually for tracking dependencies" trick. It should make it easier to switch to using `FileLoader` for binary files in the future as well cc rust-lang#62948 r? @petrochenkov
Rollup of 10 pull requests Successful merges: - #60492 (Add custom nth_back for Chain) - #61780 (Finalize the error type for `try_reserve`) - #63495 ( Remove redundant `ty` fields from `mir::Constant` and `hair::pattern::PatternRange`.) - #63525 (Make sure that all file loading happens via SourceMap) - #63595 (add sparc64-unknown-openbsd target) - #63604 (Some update for vxWorks) - #63613 (Hygienize use of built-in macros in the standard library) - #63632 (A couple of comment fixes.) - #63634 (ci: properly set the job name in CPU stats) - #63636 (ci: move linkcheck from mingw-2 to mingw-1) Failed merges: r? @ghost
|
☔ The latest upstream changes (presumably #63640) made this pull request unmergeable. Please resolve the merge conflicts. |
bors
added
S-waiting-on-author
That way, callers don't need to repeat "let's add this to sm manually
for tracking dependencies" trick.
It should make it easier to switch to using
FileLoaderfor binaryfiles in the future as well
cc #62948
r? @petrochenkov