Update PyTorch to >=2.4.0 to get fix for CUDA array interface bug, and drop CUDA 11 PyTorch tests.

[bdice]

Description

This PR updates our PyTorch lower bound to 2.4.0 to get the bugfix from pytorch/pytorch#121458.

Also, this PR drops CUDA 11 tests because conda-forge no longer produces CUDA 11 builds of PyTorch. This was causing a failure on Hopper GPUs because the last available CUDA 11 builds from conda-forge do not include sm90 support.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[bdice]

/ok to test

[bdice]

The failing Hopper tests looked like this:

=========================== short test summary info ============================
FAILED tests/test_cuda_array_interface.py::test_cuda_array_interface_pytorch - torch.cuda.DeferredCudaCallError: CUDA call failed lazily at initialization with error: 
NVIDIA H100 PCIe with CUDA capability sm_90 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_35 sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_89 compute_89.
If you want to use the NVIDIA H100 PCIe GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

This occurs because pytorch 2.5.1 cuda118_py310h920319e_303 from conda-forge lacks sm90 support.

[bdice]

/merge