make conda recipe data-loading stricter

[jameslamb]

Contributes to rapidsai/build-planning#72

Proposes using [] subsetting instead of .get() in templating statements in the conda recipe that read data out of pyproject.toml. That'll ensure that we get a big loud build error if changes to pyproject.toml remove some sections that the conda recipe expects to exist.

Notes for Reviewers

How I tested this

Rendered the recipe.

git fetch upstream --tags

RAPIDS_DATE_STRING="2408" \
RAPIDS_PACKAGE_VERSION="24.8.0" \
conda render \
  -c conda-forge \
  -c rapidsai-nightly \
  conda/recipes/dask-cuda

It looks correct to me (click for details)

--------------
Hash contents:
--------------
{}
----------
meta.yaml:
----------
package:
  name: dask-cuda
  version: 24.8.0
source:
  path: /Users/jlamb/repos/dask-cuda
build:
  entry_points:
    - dask-cuda-worker = dask_cuda.cli:worker
    - dask-cuda-config = dask_cuda.cli:config
  number: '10'
  script:
    - /Users/jlamb/miniforge3/conda-bld/dask-cuda_1718216576022/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/python
      -m pip install . -vv
  string: py310_2408_g3a04719_10
requirements:
  host:
    - bzip2 1.0.8 h93a5062_5
    - ca-certificates 2024.6.2 hf0a4a13_0
    - libffi 3.4.2 h3422bc3_5
    - libzlib 1.3.1 hfb2fe0b_1
    - ncurses 6.5 hb89a1cb_0
    - python_abi 3.10 4_cp310
    - tzdata 2024a h0c530f3_0
    - xz 5.2.6 h57fd34a_0
    - yaml 0.2.5 h3422bc3_2
    - libsqlite 3.46.0 hfb93653_0
    - openssl 3.3.1 hfb2fe0b_0
    - readline 8.2 h92ec313_1
    - tk 8.6.13 h5083fa2_1
    - python 3.10.14 h2469fbe_0_cpython
    - attrs 23.2.0 pyh71513ae_0
    - packaging 24.1 pyhd8ed1ab_0
    - pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1
    - pyyaml 6.0.1 py310h2aa6e3c_1
    - rpds-py 0.18.1 py310h947b723_0
    - setuptools 70.0.0 pyhd8ed1ab_0
    - tomlkit 0.12.5 pyha770c72_0
    - wheel 0.43.0 pyhd8ed1ab_1
    - zipp 3.19.2 pyhd8ed1ab_0
    - importlib_resources 6.4.0 pyhd8ed1ab_0
    - pip 24.0 pyhd8ed1ab_0
    - referencing 0.35.1 pyhd8ed1ab_0
    - jsonschema-specifications 2023.12.1 pyhd8ed1ab_0
    - jsonschema 4.22.0 pyhd8ed1ab_0
    - rapids-dependency-file-generator 1.13.11 py_0
    - rapids-build-backend 0.3.1 py_0
  run:
    - pynvml>=11.0.0,<11.5
    - numpy>=1.23,<2.0a0
    - python_abi 3.10.* *_cp310
    - click >=8.1
    - rapids-dask-dependency==24.8.*,>=0.0.0a0
    - numba>=0.57
    - python >=3.10,<3.11.0a0
    - pandas>=1.3
    - zict>=2.0.0
test:
  commands:
    - dask cuda --help
    - dask-cuda-worker --help
    - dask cuda worker --help
    - dask-cuda-config --help
    - dask cuda config --help
  imports:
    - dask_cuda
about:
  dev_url: https://github.com/rapidsai/dask-cuda
  doc_url: https://docs.rapids.ai/api/dask-cuda/stable/
  home: https://github.com/rapidsai/dask-cuda
  license: Apache 2.0
  license_file:
    - ../../../LICENSE
  summary: Utilities for Dask and CUDA interactions
extra:
  copy_test_source_files: true
  final: true

[jameslamb]

Moving this back to draft, there is a failure I need to investigate. Sorry for the noise.

[jameslamb]

I know this one looks a little weird. I can explain.

Switching from this

{% for e in data.get("project", {}).get("scripts", {}).items() %}

to this

{% for e in data["project"]["scripts"].items() %}

results in an error like this:

  File "/__w/dask-cuda/dask-cuda/conda/recipes/dask-cuda/meta.yaml", line 24, in top-level template code
    {% for e in data["project"]["scripts"].items() %}
TypeError: 'NoneType' object is not callable

(build link)

It seems that calling dictionary methods like .items() or .iteritems() isn't supported... maybe conda-build overrides .get() or __getattr__ or getattr(), or maybe that's something Jinja2 is doing.

BUT... this pattern where you for-iterate over keys, and then use those to subset the list, totally works 😁

And it does come with a bit more safety. I tried removing the [project.scripts] table in pyproject.toml completely. On branch-24.08, that results in the recipe silently building "successfully" without those entrypoints defined.

On this branch, it results in an informative error.

Error: Failed to render jinja template in /Users/jlamb/repos/dask-cuda/conda/recipes/dask-cuda/meta.yaml:
'dict object' has no attribute 'scripts'

See "How I tested this" in the PR description for details on how I tested this.

[bdice]

"I can explain, I promise!" is always a good thing to write on a diff. 😉 I think this is totally fine.

[jameslamb]

haha yeah, thanks 😂

[bdice]

"I can explain, I promise!" is always a good thing to write on a diff. 😉 I think this is totally fine.

[jameslamb]

/merge