-
Notifications
You must be signed in to change notification settings - Fork 369
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve performance of by() using NamedTuples
Remove GroupApplied and deprecate combine in favor of map(f, ::GroupedDataFrame). This avoids storing a copy of the per-group data returned by the user-provided function. Take advantage of this by allowing that function to return a NamedTuple. Introduce two completely different code paths depending on whether the first returned object is DataFrame or a NamedTuple, as the latter allows for more efficient operation by assuming that it represents a single row. Use the same progressive eltype widening approach as Base.map so that we fill column vectors whose types are known inside the kernel functions. This does not eliminate the type unstability due to the fact that the user-provided function takes a DataFrame, but ensuring type stability for half of the operations still improves performance significantly. Also parameterize GroupedDataFrame on the type of data frame it wraps, and make its column index have a concrete type. Deprecate an old map method for SubDataFrame. Fix a type unstability in hcat!.
- Loading branch information
Showing
8 changed files
with
369 additions
and
181 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -28,7 +28,7 @@ meltdf | |
```@docs | ||
allowmissing! | ||
categorical! | ||
combine | ||
map | ||
completecases | ||
deleterows! | ||
describe | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.