Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Support 'rename' in configure_file, build_target and custom_target #13960

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

keith-packard
Copy link

@keith-packard keith-packard commented Nov 26, 2024

This allows built results from any build target, custom target or configure file to be renamed during installation, just as with install_data.

This feature is motivated by the need to install multiple files from a single source directory into different install directories but with the same basename. In my case, I build dozens of 'libc.a' files which need to be installed into the toolchain's multilib heirarchy correctly.

This replaces #11954 and I can use this instead of #4037.

@keith-packard keith-packard force-pushed the target-rename branch 5 times, most recently from a165c27 to 6ff146a Compare November 26, 2024 03:39
docs/yaml/functions/_build_target_base.yaml Outdated Show resolved Hide resolved
mesonbuild/backend/backends.py Outdated Show resolved Hide resolved
mesonbuild/build.py Outdated Show resolved Hide resolved
@@ -3482,6 +3488,11 @@ def build_target(self, node: mparser.BaseNode, args: T.Tuple[str, SourcesVarargs
target = targetclass(name, self.subdir, self.subproject, for_machine, srcs, struct, objs,
self.environment, self.compilers[for_machine], kwargs)

if len(kwargs['rename']) not in {0, 1, len(target.outputs)}:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this still have the soname issue that was mentioned in earlier attempts?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably. I don't need that case to work for my projects, and we have had zero traction in getting build subdirectory support merged in meson, so I'm trying a simpler patch to see if I can at least get my use case working without thousands (literally) of warnings. To fix the soname issue, it might be easier to fix up the existing -soname code paths to permit a custom name for that too?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that my alternative plan, #14002, does not suffer from this problem.

@keith-packard keith-packard force-pushed the target-rename branch 2 times, most recently from b525553 to 1b8f0e5 Compare December 6, 2024 18:28
@keith-packard keith-packard force-pushed the target-rename branch 9 times, most recently from f51c06d to fc55a87 Compare December 7, 2024 17:16
@keith-packard
Copy link
Author

In getting the tests working, I discovered that shared libraries can generate additional files, like the Windows import library, which aren't listed in the outputs array. As you can see from this test.json file, those get installed using the original names, without any rename support.

I could:

  1. Ignore this
  2. Have the rename list include entries for these additional files
  3. Make the rename value affect only the directory and basename, then generate the actual names using the directory, prefix, basename and suffix much like the current code does for the target name to filename mapping.

Other than this issue, I think this MR is ready to go; it's passing all of the tests (save for a couple which seem to just have CI issues?).

@keith-packard
Copy link
Author

I missed a case that I need -- configure_file. I've added another commit which fixes that.

@keith-packard keith-packard changed the title build: Support 'rename' in build_target and custom_target build: Support 'rename' in configure_file, build_target and custom_target Dec 7, 2024
@dcbaker
Copy link
Member

dcbaker commented Dec 9, 2024

I believe that the additional windows files the .lib and .pdb must have the same name as the .dll to be automatically loaded, I think that Windows .exe files can also generate a .pdb, and those need to match the name of the .exe.

@keith-packard
Copy link
Author

keith-packard commented Dec 9, 2024

I believe that the additional windows files the .lib and .pdb must have the same name as the .dll to be automatically loaded, I think that Windows .exe files can also generate a .pdb, and those need to match the name of the .exe.

If so, then just taking the rename value and editing that to the correct form might be a reasonable first approximation. Would it be OK if the first version of this feature simply disallowed use of rename when generating windows shared libraries?

@dcbaker
Copy link
Member

dcbaker commented Dec 9, 2024

Hmmm... IIUC, then we would just need to install the .lib and .pdb with the same name but .pdb, though maybe we have someone with more Windows expertise?

skipping Windows might be a possibility if getting it working is too hard, but I'd like to at least try to get it working.

It would also need to exclude Vala because there are implicit outputs from that too.

@keith-packard
Copy link
Author

Hmmm... IIUC, then we would just need to install the .lib and .pdb with the same name but .pdb, though maybe we have someone with more Windows expertise?

"it's complicated". Cygwin uses a 'cyg' prefix rather than 'lib', while still using '.dll.a' for various reasons.

skipping Windows might be a possibility if getting it working is too hard, but I'd like to at least try to get it working.

Instead of attempting to automate this renaming, I've just added two new keyword args, 'import_rename' and 'debug_rename' that allow the user to explicitly specify the install names for those two files. This ensures that the
user has complete control over all of the installed filenames.

It would also need to exclude Vala because there are implicit outputs from that too.

Vala uses the existing 'rename' mechanism as it extends the 'output' array with the new files. That means the user will
already have to make the 'rename' array contain entries for the additional files.

A couple of options I considered:

  • Implicit names for the import library and debuginfo files based upon the provided rename value. This would have been a bit tricky to make sure the 'correct' part of the rename value was used.
  • Switching the rename values to just the basename and applying the name_prefix and name_suffix values to the main install file as well as applying the same editing rules to construct suitable import library and debuginfo file names. This would have made these new rename uses different from existing rename uses in the system, and that seemed like a user trap.
  • Adding these new rename values to the existing rename list. That would have placed an implicit ordering on the list, which contrasts with mesons use of keyword arguments when order is not relevant. New keyword arguments make it easy to tell what the script is going to generate without having to know the order of the rename list.

@keith-packard keith-packard force-pushed the target-rename branch 2 times, most recently from 68b3693 to b152a17 Compare December 9, 2024 22:18
@bonzini
Copy link
Collaborator

bonzini commented Jan 30, 2025

Can you rebase, hoping that will improve CI results a bit?

@keith-packard
Copy link
Author

I've rebased both this and #14002. As I mentioned over there, I greatly prefer #14002 to this kludge, but this would at least let me get my multi-target clang builds working.

This allows built results to be renamed during installation, just as
with install_data.

This feature is motivated by the need to install multiple files from a
single source directory into different install directories but with
the same basename. In my case, I build dozens of 'libc.a' files which
need to be installed into the toolchain's multilib heirarchy
correctly.

The rename parameter must either be empty, indicating that renaming is
disabled, or have length equal to the number of outputs
generated. Using an empty list to disable renaming matches the
behavior of install_data.

Signed-off-by: Keith Packard <[email protected]>
This allows generated configuration files to be renamed during
installation, just as with install_data.

The rename parameter must either be an empty list, indicating that
renaming is disabled, or a single string.

Signed-off-by: Keith Packard <[email protected]>
Executables, shared libraries and shared modules may all install two
additional files, an import library and a debuginfo file. Provide new
keyword arguments, 'import_rename', and 'debug_rename' so that these
can be installed with names matching the output filename when
using the 'rename' keyword argument.

Signed-off-by: Keith Packard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants