Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tell cudf_kafka to use header-only fmt #12796

Merged
merged 1 commit into from
Feb 16, 2023

Conversation

vyasr
Copy link
Contributor

@vyasr vyasr commented Feb 16, 2023

Description

The changes to spdlog/fmt packaging in rmm caused an undefined symbol issue in cudf_kafka. To unblock CI in #12783, I simply added fmt to the list of required libraries for the Python package (see caf7adf and the explanation). The underlying issue turns out to be that by default usage of fmt results in the dependent assuming that the headers are compiled in not header-only mode. When using CMake to manage the build, fmt relies on use of the appropriate target fmt::fmt-header-only to configure the build such that anything using fmt knows to use it in header-only mode, which sets the FMT_HEADER_ONLY preprocessor macro under the hood. Since the Python cudf_kafka package is not built using CMake, however, this information was not propagated to its build from its dependencies (libcudf/libcudf_kafka). As a result, it was compiled expecting an external definition of some fmt symbols rather than inlining them. This PR removes the undesirable library dependency on fmt introduced in #12783 in favor of properly telling fmt to expect inlined symbols.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@vyasr vyasr added bug Something isn't working 3 - Ready for Review Ready for review by team non-breaking Non-breaking change labels Feb 16, 2023
@vyasr vyasr requested a review from a team as a code owner February 16, 2023 20:03
@vyasr vyasr self-assigned this Feb 16, 2023
@github-actions github-actions bot added the Python Affects Python cuDF API. label Feb 16, 2023
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the explanation in the PR description! This change looks good, and the rationale makes total sense to me. I am not aware of any other non-CMake consumers of libcudf besides cudf_kafka.

Should we document somehow that libcudf requires this to be defined for builds not consuming libcudf via CMake?

@vyasr
Copy link
Contributor Author

vyasr commented Feb 16, 2023

Thank you for the explanation in the PR description! This change looks good, and the rationale makes total sense to me. I am not aware of any other non-CMake consumers of libcudf besides cudf_kafka.

Should we document somehow that libcudf requires this to be defined for builds not consuming libcudf via CMake?

This isn't even libcudf, it's coming from librmm. I had a chat with Rob about this and we have open questions about whether it should be possible to mix header-only fmt with library fmt. He's run into related issues in other places recently as well, so figuring out the best approach will be a follow-up task.

@vyasr
Copy link
Contributor Author

vyasr commented Feb 16, 2023

/merge

@rapids-bot rapids-bot bot merged commit cf82fb8 into rapidsai:branch-23.04 Feb 16, 2023
@vyasr vyasr deleted the fix/kafka_fmt_header branch February 16, 2023 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team bug Something isn't working non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants