Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow arbitrary path lengths in compiler #26357

Merged
merged 12 commits into from
Jan 7, 2025
Merged

Allow arbitrary path lengths in compiler #26357

merged 12 commits into from
Jan 7, 2025

Conversation

riftEmber
Copy link
Member

@riftEmber riftEmber commented Dec 4, 2024 •
edited
Loading

Removes usages of FILENAME_MAX and other hardcoded buffers for path strings, enabling the use of arbitrarily long file paths.

This PR has the same purpose as #17517, but isn't based off of it as the prior PR was too far behind main.

Resolves #26261 and contributes to #8757.

[reviewer info placeholder]

Testing:

  • no error for >4096 character dependency paths on my Mac
  • paratest
  • gasnet paratest
  • C backend paratest
  • GPU tests
  • --no-compiler-driver paratest

@riftEmber riftEmber mentioned this pull request Dec 4, 2024
Closed
3 tasks
@riftEmber riftEmber marked this pull request as draft December 4, 2024 21:04
@riftEmber riftEmber force-pushed the compiler-no-filename-max branch 9 times, most recently from fd59f03 to d6ce41b Compare December 6, 2024 17:20
This was referenced Dec 9, 2024
Merged
Closed
Closed
@riftEmber riftEmber marked this pull request as ready for review December 9, 2024 21:17
@riftEmber riftEmber force-pushed the compiler-no-filename-max branch from d6ce41b to 47b5b9e Compare December 9, 2024 21:17
jabraham17
jabraham17 approved these changes Dec 9, 2024
View reviewed changes
compiler/codegen/library.cpp Outdated Show resolved Hide resolved
compiler/codegen/library.cpp Outdated Show resolved Hide resolved
compiler/include/library.h Outdated Show resolved Hide resolved
compiler/main/arg.cpp
else
printf("''");
break;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definitely outside the scope of this PR, but it would be nice to change 'S' to use std::string too. Maybe open an issue for refactoring the arg parsing code to use more modern C++?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there something you would want to see change about how it functions, besides the internals? I feel "if it ain't broke" about refactoring it, personally.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing specific in terms of the API, I mostly am just tired of dealing with very old C-style C++ code, which is true for much of the compiler

I feel "if it ain't broke" about refactoring it, personally.

I kinda tend to agree, but I think there is an aspect of "leave things better than you found it" too. Definitely not advocating you do anything in this regard for this PR.

compiler/main/arg.cpp Show resolved Hide resolved
compiler/main/driver.cpp Outdated Show resolved Hide resolved
compiler/util/misc.cpp Outdated Show resolved Hide resolved
@riftEmber riftEmber requested a review from jabraham17 December 10, 2024 21:46
jabraham17
jabraham17 approved these changes Dec 10, 2024
View reviewed changes
compiler/util/stringutil.cpp Outdated Show resolved Hide resolved
riftEmber added 10 commits January 6, 2025 16:51
@riftEmber
Use std::getline to restore unlimited length driver strings
1d35813 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Avoid recomputing tmp path
cc3cfae 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
First pass at removing compiler uses of FILENAME_MAX
b01c06d 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Fix processing std::string path args
03cb434 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Use C++ string manipulation for stripLibPrefix
b2f4ecf 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Fix generateFileName
51d4108 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Fix removing .o extension for incremental
991e0d8 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Apply most review feedback
ac5b865 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Use string_view in cleanFilename
357509c 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber
Use string_view for driver save/restore
f8685a6 
Signed-off-by: Anna Rift <[email protected]>
@riftEmber riftEmber force-pushed the compiler-no-filename-max branch from 8e815ff to 99e8e4a Compare January 7, 2025 00:51
@riftEmber riftEmber merged commit af560a8 into chapel-lang:main Jan 7, 2025
8 checks passed
@riftEmber riftEmber deleted the compiler-no-filename-max branch January 7, 2025 16:54
@bradcray
Copy link
Member

bradcray commented Jan 7, 2025

Thanks for picking this up, Anna!

riftEmber added a commit that referenced this pull request Jan 7, 2025
@riftEmber
Remove uses of FILENAME_MAX in the runtime (#26381)
0d8be34 
Replace uses of `FILENAME_MAX` for buffer sizes in the runtime,
replacing them with heap-allocated strings of the proper size that are
reallocated as needed.

Merges in and builds on the work by @rchinmay in
#17059.

Completes the second part of
#8757 (with
#26357).

[reviewed by @jabraham17 , thanks!]

Testing:
- [x] paratest
- [x] gasnet paratest
- [x] C backend paratest
- [x] GPU tests
- [x] manual run of a nightly config that covers (some) affected code
(`test-hpe-cray-ex-ofi.bash`)
@riftEmber riftEmber mentioned this pull request Jan 8, 2025
Merged
5 tasks
riftEmber added a commit that referenced this pull request Jan 8, 2025
@riftEmber
Fix mistakes from removing FILENAME_MAX (#26491)
5e5a0e9 
Fix a handful of errors made in removing `FILENAME_MAX` from the
compiler and runtime.

Follow up to #26357 and
#26381.

[trivial fixes, not reviewed]

Testing:
- [x] paratest
- [x] C paratest
- [x] gasnet paratest
- [x] GPU tests
- [x] manual run of an affected test config for each failure mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jabraham17 jabraham17 jabraham17 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: long dependency paths can break compiler driver
3 participants
@riftEmber @bradcray @jabraham17