Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix func count for dtype O with numpy and numba #138

Merged
merged 2 commits into from
Aug 16, 2022

Conversation

aulemahal
Copy link
Contributor

This fixes #137.

Changes

  • Generalized the datetime reduce test so it tests all engines.
  • When func='count' and engine != 'flox', requires_numeric becomes True, which avoid type errors in numba and numpy.

Performance

The "flox" engine is not included because I saw a loss of performance in my (small) tests.

import xarray as xr
from flox.xarray import xarray_reduce

t = xr.DataArray(xr.cftime_range('1900-01-01', periods=100000, freq='H'), dims=('time',))
xarray_reduce(t, t.dt.month, func='count', engine=ENGINE)

With engine 'flox', this code was around 10% slower if I added the numeric conversion.

@dcherian
Copy link
Collaborator

Yeah I can see it being slower; flox runs isnull to convert to bool directly; the rest will convert to int and then bool.

This looks great, Thanks for catching it so early!

@dcherian dcherian merged commit 39870ee into xarray-contrib:main Aug 16, 2022
@aulemahal aulemahal deleted the fix-137 branch August 16, 2022 20:24
dcherian added a commit that referenced this pull request Oct 9, 2022
* main:
  Update ci-additional.yaml (#167)
  Refactor before redoing cohorts (#164)
  Fix mypy errors in core.py (#150)
  Add link to numpy_groupies (#160)
  Bump codecov/codecov-action from 3.1.0 to 3.1.1 (#159)
  Use math.prod instead of np.prod (#157)
  Remove None output from _get_expected_groups (#152)
  Fix mypy errors in xarray.py, xrutils.py, cache.py (#144)
  Raise error if multiple by's are used with Ellipsis (#149)
  pre-commit autoupdate (#148)
  Add mypy ignores (#146)
  Get pre commit bot to update (#145)
  Remove duplicate examples headers (#147)
  Add ci additional (#143)
  Bump mamba-org/provision-with-micromamba from 12 to 13 (#141)
  Add ASV benchmark CI workflow (#139)
  Fix func count for dtype O with numpy and numba (#138)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"count" reduction of cftime objects broken by new default
2 participants