Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: retrieve .text coordinates for large pages #33116

Closed

Conversation

gabrielschulhof
Copy link
Contributor

After locating the in-memory base address of the Node.js executable
using dl_iterate_phdr(3) we open the on-disk executable file and
extract the address and size of the .text section. We then add the
in-memory base address to the offset to locate the .text section in
memory.

This approach removes the need for a symbol to tell us where the
.text section is located, and, unlike the previous approach,
guarantees that we will always have the entire .text section
available for re-mapping.

Signed-off-by: @gabrielschulhof

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

After locating the in-memory base address of the Node.js executable
using `dl_iterate_phdr(3)` we open the on-disk executable file and
extract the address and size of the `.text` section. We then add the
in-memory base address to the offset to locate the `.text` section in
memory.

This approach removes the need for a symbol to tell us where the
`.text` section is located, and, unlike the previous approach,
guarantees that we will always have the entire `.text` section
available for re-mapping.

Signed-off-by: Gabriel Schulhof <[email protected]>
@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. labels Apr 28, 2020
@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@bnoordhuis bnoordhuis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a good idea for two reasons:

  1. Won't work when /proc isn't mounted.

  2. Does a lot of extra file I/O at startup.

The second one in particular is a deal breaker.

class_name(const class_name&) = delete; \
class_name(class_name&&) = delete; \
void operator= (const class_name&) = delete; \
void operator= (const class_name&&) = delete
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to have a DISALLOW_COPY_AND_ASSIGN macro for this but we moved to just writing them out by hand, see #26634. Can you do that here too?

struct stat file_info;
if (stat(fname, &file_info) == -1) goto fail;

fd_ = open(fname, O_RDONLY);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open(2) is interruptible by a signal. You should retry on EINTR:

Suggested change
fd_ = open(fname, O_RDONLY);
do
fd_ = open(fname, O_RDONLY);
while (fd == -1 && errno == EINTR);

As well, stat + open is race-y. Not that it really matters here but you still should do open + fstat. :-)

FORCE_INLINE ~MappedFilePointer() {
if (fd_ == -1) return;
if (close(fd_) == 0) return;
PrintSystemError(errno);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you if (errno == EINTR || errno == EINPROGRESS) return;?

close(2) is interruptible but you shouldn't retry - it's going to get closed anyway - and printing an error isn't useful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this.

for (uint32_t idx = 0; idx < ehdr->e_shnum; idx++) {
const ElfW(Shdr)* sh = &shdrs[idx];
const char* section_name = static_cast<const char*>(&snames[sh->sh_name]);
if (std::string(section_name) == ".text") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (std::string(section_name) == ".text") {
if (0 == strcmp(section_name, ".text")) {

It's kind of wasteful to create a heap-allocated throwaway string. It's probably going to be stored inline for short strings but you get my point.

}
ElfW(Shdr) text_section;
#if defined(__linux__)
dl_params.exename = "/proc/self/exe";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes /proc is mounted but that need not be the case.

@gabrielschulhof
Copy link
Contributor Author

@bnoordhuis I checked the startup time wrt. master and it does not seem to be different at all with this change. This feature used to rely on reading /proc/self/maps instead of dl_iterate_phdr(). There was file I/O and reliance on /proc in that. IMO removing the complicated build process and the groping in the dark of the segment with heuristics like

if (dl_params->reference_sym >= start &&
dl_params->reference_sym <= end) {
dl_params->start = start;
dl_params->end = end;
return 1;
}

and

if (lpstub_start > dl_params.start && lpstub_start <= dl_params.end) {
Debug("Hugepages info: Trimming end for lpstub: %p\n",
reinterpret_cast<void*>(lpstub_start));
dl_params.end = lpstub_start;
}

may be worth the renewed reliance on /proc.

@gabrielschulhof
Copy link
Contributor Author

@bnoordhuis I also added a Debug() in MappedFilePointer::Reset in case it fails because of a lack of /proc.

@bnoordhuis
Copy link
Member

I checked the startup time wrt. master and it does not seem to be different at all with this change.

The kernel reads the ELF header on startup so it's often in the page cache, but when it's not and you're running node off your Raspberry Pi's SD card, you're going to notice.

This feature used to rely on reading /proc/self/maps instead of dl_iterate_phdr(). There was file I/O and reliance on /proc in that.

procfs is a virtual fs though, there's no actual disk I/O taking place.

@gabrielschulhof
Copy link
Contributor Author

@bnoordhuis fair enough.

@gabrielschulhof gabrielschulhof deleted the large-pages-read-elf branch January 28, 2021 00:14
@gabrielschulhof gabrielschulhof restored the large-pages-read-elf branch January 28, 2021 05:40
@gabrielschulhof gabrielschulhof deleted the large-pages-read-elf branch February 3, 2021 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants