Add mode #453
Add mode #453
Conversation
There was a problem hiding this comment.
This gets particularly hairy in a groupby scenario where each group may have different lengths
I think this is fine, because I think the computation is going to occur inside the group's context. Maybe adding a test - actually change an existing one with another column - to the summarise or mutate can clarify a little bit. WDYT?
Yes, between groups, but not within groups. So take the example where you summarise down to mean and mode. If mode is length 0 or >1, then you have a problem where within groups the aggregations are different lengths. Adding a test or two now to clarify as suggested 👍. |
|
Well it clarified in a different direction: summarise w/ mode returns a single value per group -- a list! We'll have to pick up #401 anyway. |
Change map helper functions' arguments
| @doc """ | ||
| Gets the most common value of the series. | ||
|
|
||
| ## Supported dtypes |
There was a problem hiding this comment.
Let’s just say all except lists? Otherwise it is easy for this to go out of date!
| non_list_dtypes = [ | ||
| non_list_dtypes = non_list_types() | ||
| list_dtypes = for dtype <- non_list_dtypes, do: {:list, dtype} | ||
| non_list_dtypes ++ list_dtypes |
There was a problem hiding this comment.
List types are recursive so this is theoretically incomplete. Instead of doing this list, what if we just append {:list, :any} instead?
There was a problem hiding this comment.
So it looks like this is used for tests? I'm going to leave it how it was (I just wrapped the non-list ones into a separate function) and then I think we should revisit it in a future PR.
Co-authored-by: Billy Lanchantin <[email protected]>
See: #452. This is a WIP because I'm not sure how to handle the fact that you can have no mode or multiple. This gets particularly hairy in a groupby scenario where each group may have different lengths (and thus would cause problems with other summary statistics like mean, median where you know the length will be 1).Pandas:
https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.mode.html
https://pandas.pydata.org/docs/reference/api/pandas.Series.mode.html#pandas.Series.mode
https://stackoverflow.com/a/54304691
R:
https://stackoverflow.com/questions/66972590/how-to-find-mean-median-mode-based-on-distinctive-groups-in-r
https://cran.r-project.org/web/packages/modeest/modeest.pdf
Okay! Thanks to #725 this is good to go. Closes #452.