[FEA] cudf_kafka: add unit tests

[jameslamb]

Is your feature request related to a problem? Please describe.

As far as I can tell (please correct me if I'm wrong):

• cudf_kafka wheels are not currently tested in this project's CI.
• the only testing of cudf_kafka conda packages is an import cudf_kafka run during the conda build:

cudf/conda/recipes/cudf_kafka/meta.yaml

Lines 76 to 80 in 1710e11

	test:
	requires:
	- cuda-version ={{ cuda_version }}
	imports:
	- cudf_kafka

Describe the solution you'd like

This project should test cudf_kafka functionality in its CI.

At a minimum, a unit test that just runs import cudf_kafka and checks something trivial about it would provide some coverage of concerns like "is the package syntactically-valid Python for each version of Python it claims to support" and might catch issues with linking to libcudf.

A simple tests like the ones proposed in #15245 would be sufficient to at least establish the pattern for where to add more tests.

This could look like:

• add a python/cudf_kafka/tests directory
• add a test there, maybe like the version tests from Use rapids-build-backend. #15245

import cudf_kafka

def test_version_constants_are_populated():
    # __git_commit__ will only be non-empty in a built distribution
    assert isinstance(cudf.__git_commit__, str)

    # __version__ should always be non-empty
    assert isinstance(cudf.__version__, str)
    assert len(cudf.__version__) > 0

• ensure that test is run in CI, on wheels and conda packages
  • conda tests probably go here: https://github.com/rapidsai/cudf/blob/1710e11c3ae9dd072305ca49e12e10d0f2e3aec0/ci/test_python_other.sh

Describe alternatives you've considered

I don't think this needs to go as far as #5571 to be valuable.

Another alternative is to just add more testing to the cudf_kafka directly in the conda build recipe... but I think having tests on the wheels is valuable, as it'll catch different things.

Additional context

Created this based on this conversation: #15245 (comment)

[bdice]

@jameslamb Last time this topic came up we concluded not to produce cudf_kafka wheels. There are very few consumers of this library, and at that point, the build system would have required us to build and ship all of libcudf as part of both the cudf-cu(11|12) and cudf_kafka-cu(11|12) wheel if we chose to produce one. Perhaps the new C++ wheels for libcudf would solve part of this problem (building and shipping less in the Python wheels) but we'd probably need libcudf_kafka wheels in addition to libcudf wheels. This would be pretty complicated for very little gain, given the low usage of this library.

I am inclined to add unit tests that are as minimal as possible for conda packages and not ship cudf_kafka wheels at all. That would split the scope of this issue: add unit tests 👍 , add wheels 👎.

References:

• Refactor cudf_kafka to use skbuild #14292 (comment)
• Refactor cudf_kafka to use skbuild #14292 (comment)
• Refactor cudf_kafka to use skbuild #14292 (comment)

[jameslamb]

Ah!!!

@bdice I didn't realize that we weren't shipping cudf_kafka wheels. I definitely did not mean for this issue to propose doing that! My fault for assuming and not checking more closely 🙈

I'll modify the title and description to reflect that this is only about adding tests for the conda package.

as minimal as possible

100%, my main motivation for this was wanting a test to confirm that cudf_kafka.__version__ is populated and available. I'm not proposing going any further than that.

[vyasr]

Agree with what Bradley said: yes to tests, no to wheels.