[Overview] Dependency Management RFC

[sophiewigmore]

Summary

Readable

Overview RFC of the three-phased approach to changing our dependency management system.

• Related to Buildpack Dependency Management Improvements paketo-community/explorations#8

Use Cases

Checklist

• I have viewed, signed, and submitted the Contributor License Agreement.
• I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
• I have added an integration test, if necessary.
• I have reviewed the styleguide for guidance on my code quality.
• I'm happy with the commit history on this PR (I have rebased/squashed as needed).

[ekcasey]

Is this really a compilation problem or is it a metadata problem?

For example our compilation of node runs fine on jammy apparently

docker run -it -v /tmp/paketo-node:/node ubuntu:jammy bash
root@eaa064b73682:/# cd node
root@eaa064b73682:/node# tar xzf node_v14.19.2_linux_x64_bionic_3f263e3e.tgz
root@eaa064b73682:/node# bin/node --help
Usage: node [options] [ script.js ] [arguments]
       node inspect [options] [ script.js | host:port ] [arguments]

And indeed dynamically links against the same (presumably stable) set of glilbc APIs as the upstream distribution

cnb@0e9092f382aa:/workspace$ ldd /layers/paketo-buildpacks_node-engine/node/bin/node
	linux-vdso.so.1 (0x00007ffd61cd1000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4729087000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4728cfe000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4728960000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4728748000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4728529000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4728138000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f472dc00000)

In cases where the results of compilation differ between stacks (e.g. nodejs for alpine or nginx probably) we will still need multiple version per stack even if we fetch binaries from from upstream.

Not suggesting that we shouldn't move away from recompiling, but that in some cases we could add support for more stacks with our current set of deps.

[sophiewigmore]

Yea you're right. It won't be uniform across the board, some dependencies will just work as they are now, but it'll be up to maintainers to figure that out per dependency/stack offering and make the choice that works for that dependency

[ekcasey]

Can we approve one of these phases without the others? Seems like we can't approve the overview without approving the phases at least. In which case, I am not sure what gain from splitting the RFC up. I could definitely see us tracking implementation in three phases but maybe it would be better to have the entire plan (including the three phases) in a single document so we can approve it all in one go and reference the details in one place.

[sophiewigmore]

The phases are pretty dependent on each other, but I think there are some details that are independent. We could agree on Phase 1 (buildpack-related changes) and get that approved, and then go back to the Phase 2 RFC and hash out the details afterwards. The separation of RFCs is tied to the phased implementation, but I do think it makes the content more digestible by separating concerns. It's a lot of content to put into one RFC in my opinion.

[sophiewigmore]

It makes it easier for me as the RFC author to have a sort of separation of concerns, but there is some overlap that may make it confusing for others. My intention with the phases was to make it clearer!

[fg-j]

Meeting today to discuss.
Details: https://paketobuildpacks.slack.com/archives/C011S6EL49L/p1657818502769609

[joshuatcasey]

2022-07-26 Working Group notes:

@ryanmoran encourages merging, we have clarity around the broad strokes in these dependency RFCs and we can go ahead and merge, recommends not waiting for the stack removal from https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md. We should be able to migrate from the current dependency metadata to the "new" metadata easily enough.