Use conda compilers

[jjacobelli]

This PR enables the usage of conda compilers to build conda packages. This is required to use mambabuild

[jakirkham]

Thanks Jordan! 😄

Generally this looks great. Had a few suggested simplifications below

[jjacobelli]

@jakirkham PR updated

[jakirkham]

Thanks Jordan! 🙏 LGTM 😄

[vyasr]

It looks like the most recent release of fsspec broke a few of our tests while our CI was offline. The issue is fsspec/filesystem_spec#961, which changes the behavior of fsspec.core.get_fs_token_path to correctly handle all globs. Previously, that function only handled globs containing an asterisk *, but that PR fixed the function to also handle ? and []. The problem is that we have tests of cudf.read_json with strings containing [] (because they contain lists), and when those globs read this line in the code they correspond to no preexisting files and so instead of getting a path to a nonexistent file like we would have before, we instead get an empty list of paths. That in turn leads to us hitting this error case.

Specifically, here is the difference:

# The new behavior
>>> fsspec.get_fs_token_paths(['[true]'], mode='rb')
(<fsspec.implementations.local.LocalFileSystem object at 0x7f93c7d58040>, '00e91f08872e08eaa06e0006dfc00ab6', [])

# The old behavior. Note that the third element of the tuple was not an empty string, but instead a path.
>>> fsspec.get_fs_token_paths(['[]'], mode='rb')
(<fsspec.implementations.local.LocalFileSystem object at 0x7fa94db5feb0>, '839c5f114235189ec7ed1e0a405abf70', ['/home/vyasr/[]'])

@rjzamora how do you recommend that we fix this? The change to fsspec seems correct, so it's not a bug for us to report upstream but rather something that we need to fix in our usage. I think we probably want to change this error check because using the fact that no paths were returned as an error condition presupposes that the input was a a valid path when it could be an arbitrary JSON string, but I don't have all the context here. Perhaps there is a better solution that addresses problems that I'm not seeing.

[galipremsagar]

It looks like the most recent release of fsspec broke a few of our tests while our CI was offline. The issue is fsspec/filesystem_spec#961, which changes the behavior of fsspec.core.get_fs_token_path to correctly handle all globs. Previously, that function only handled globs containing an asterisk *, but that PR fixed the function to also handle ? and []. The problem is that we have tests of cudf.read_json with strings containing [] (because they contain lists), and when those globs read this line in the code they correspond to no preexisting files and so instead of getting a path to a nonexistent file like we would have before, we instead get an empty list of paths. That in turn leads to us hitting this error case.

Specifically, here is the difference:

# The new behavior

>>> fsspec.get_fs_token_paths(['[true]'], mode='rb')

(<fsspec.implementations.local.LocalFileSystem object at 0x7f93c7d58040>, '00e91f08872e08eaa06e0006dfc00ab6', [])



# The old behavior. Note that the third element of the tuple was not an empty string, but instead a path.

>>> fsspec.get_fs_token_paths(['[]'], mode='rb')

(<fsspec.implementations.local.LocalFileSystem object at 0x7fa94db5feb0>, '839c5f114235189ec7ed1e0a405abf70', ['/home/vyasr/[]'])

@rjzamora how do you recommend that we fix this? The change to fsspec seems correct, so it's not a bug for us to report upstream but rather something that we need to fix in our usage. I think we probably want to change this error check because using the fact that no paths were returned as an error condition presupposes that the input was a a valid path when it could be an arbitrary JSON string, but I don't have all the context here. Perhaps there is a better solution that addresses problems that I'm not seeing.

Yea, I ran into this yesterday. I've worked on a fix. Will open the PR over the weekend.

[vyasr]

Thanks @galipremsagar! No rush, I doubt this will seriously block anyone before Monday. I just wanted to diagnose this ASAP.

[galipremsagar]

Opened a PR to address the json pytest failures here: #10924

@Ethyling could you admin merge this(10275) PR since everyone has approved it?