-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support efficient last N rows for stream table #2121
Comments
There is a hacky workaround if you have a stream table with an increasing sequence number (like a kafka table): last_n = kafka_stream.updateView("X=KafkaOffset % N").lastBy("X") or last_n = kafka_stream.updateView("X=KafkaOffset % N").lastBy("X").sort("KafkaOffset") |
Assuming a single partition - otherwise, may also want a |
mofojed
pushed a commit
that referenced
this issue
Jul 17, 2024
## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.86.0 ### Features * Add option to disable WebGL rendering ([#2134](deephaven/web-client-ui#2134)) ([011eb33](deephaven/web-client-ui@011eb33)) * Core plugins refactor, XComponent framework ([#2150](deephaven/web-client-ui#2150)) ([2571fad](deephaven/web-client-ui@2571fad)) * IrisGridTheme iconSize ([#2123](deephaven/web-client-ui#2123)) ([58ee88d](deephaven/web-client-ui@58ee88d)), closes [#885](deephaven/web-client-ui#885) * Partitioned Table UI Enhancements ([#2110](deephaven/web-client-ui#2110)) ([de5ce40](deephaven/web-client-ui@de5ce40)), closes [#2079](deephaven/web-client-ui#2079) [#2066](deephaven/web-client-ui#2066) [#2103](deephaven/web-client-ui#2103) [#2104](deephaven/web-client-ui#2104) [#2105](deephaven/web-client-ui#2105) [#2106](deephaven/web-client-ui#2106) [#2107](deephaven/web-client-ui#2107) [#2108](deephaven/web-client-ui#2108) [#2109](deephaven/web-client-ui#2109) [#2049](deephaven/web-client-ui#2049) [#2120](deephaven/web-client-ui#2120) [#1904](deephaven/web-client-ui#1904) ### Bug Fixes * error when edited cell is out of grid viewport ([#2148](deephaven/web-client-ui#2148)) ([3fccd43](deephaven/web-client-ui@3fccd43)), closes [#2087](deephaven/web-client-ui#2087) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.85.2 ### Bug Fixes * Fix missing scrim background on LoadingOverlay ([#2098](deephaven/web-client-ui#2098)) ([c9ed895](deephaven/web-client-ui@c9ed895)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.85.1 ## [0.85.1](deephaven/web-client-ui@v0.85.0...v0.85.1) (2024-07-08) ### Bug Fixes * re-export remaining types needed by dh ui from @react-types/shared ([#2132](deephaven/web-client-ui#2132)) ([2119a61](deephaven/web-client-ui@2119a61)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.85.0 ### Features * ComboBox - @deephaven/jsapi-components ([#2077](deephaven/web-client-ui#2077)) ([115e057](deephaven/web-client-ui@115e057)), closes [#2074](deephaven/web-client-ui#2074) ### Bug Fixes * Allow ComboBox to accept the FocusableRef for ref ([#2121](deephaven/web-client-ui#2121)) ([8fe9bad](deephaven/web-client-ui@8fe9bad)) * Ref was not being passed through for Picker ([#2122](deephaven/web-client-ui#2122)) ([a11e2ce](deephaven/web-client-ui@a11e2ce)) Co-authored-by: deephaven-internal <[email protected]>
mofojed
added a commit
to mofojed/deephaven-core
that referenced
this issue
Jul 18, 2024
Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.86.1 * add back panel prop to IrisGrid Plugin ([deephaven#2155](deephaven/web-client-ui#2155)) ([6362eb7](deephaven/web-client-ui@6362eb7)) * Remove the session wrapper from the embedded widget app ([deephaven#2158](deephaven/web-client-ui#2158)) ([b76c1d7](deephaven/web-client-ui@b76c1d7)) * Add option to disable WebGL rendering ([deephaven#2134](deephaven/web-client-ui#2134)) ([011eb33](deephaven/web-client-ui@011eb33)) * Core plugins refactor, XComponent framework ([deephaven#2150](deephaven/web-client-ui#2150)) ([2571fad](deephaven/web-client-ui@2571fad)) * IrisGridTheme iconSize ([deephaven#2123](deephaven/web-client-ui#2123)) ([58ee88d](deephaven/web-client-ui@58ee88d)) * Partitioned Table UI Enhancements ([deephaven#2110](deephaven/web-client-ui#2110)) ([de5ce40](deephaven/web-client-ui@de5ce40)) * error when edited cell is out of grid viewport ([deephaven#2148](deephaven/web-client-ui#2148)) ([3fccd43](deephaven/web-client-ui@3fccd43)) * Fix missing scrim background on LoadingOverlay ([deephaven#2098](deephaven/web-client-ui#2098)) ([c9ed895](deephaven/web-client-ui@c9ed895)) * re-export remaining types needed by dh ui from @react-types/shared ([deephaven#2132](deephaven/web-client-ui#2132)) ([2119a61](deephaven/web-client-ui@2119a61)) * ComboBox - @deephaven/jsapi-components ([deephaven#2077](deephaven/web-client-ui#2077)) ([115e057](deephaven/web-client-ui@115e057)) * Allow ComboBox to accept the FocusableRef for ref ([deephaven#2121](deephaven/web-client-ui#2121)) ([8fe9bad](deephaven/web-client-ui@8fe9bad)) * Ref was not being passed through for Picker ([deephaven#2122](deephaven/web-client-ui#2122)) ([a11e2ce](deephaven/web-client-ui@a11e2ce)) Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.84.0 * - @deephaven/jsapi-components - The contract of `useSearchableViewportData` to be more consistent with `useViewportData`. `usePickerWithSelectedValues` now requires `timeZone`. - @deephaven/jsapi-utils - `createSearchTextFilter` requires `timeZone` * useSearchableViewportData - support non-text filters ([deephaven#2092](deephaven/web-client-ui#2092)) ([7009e21](deephaven/web-client-ui@7009e21)) * `isElementOfType` Improved type inference ([deephaven#2099](deephaven/web-client-ui#2099)) ([e13c9d7](deephaven/web-client-ui@e13c9d7)) * Console does not scroll to bottom when code run from notebook ([deephaven#2114](deephaven/web-client-ui#2114)) ([e75e716](deephaven/web-client-ui@e75e716)) * make textValue default to key for Normalized Item ([deephaven#2113](deephaven/web-client-ui#2113)) ([bd3e944](deephaven/web-client-ui@bd3e944)) * Update IrisGridContextMenuHandler getHeaderActions return type to be more permissive ([deephaven#2117](deephaven/web-client-ui#2117)) ([4e08b79](deephaven/web-client-ui@4e08b79))
devinrsmith
pushed a commit
that referenced
this issue
Jul 18, 2024
## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.86.1 ### Bug Fixes * add back panel prop to IrisGrid Plugin ([#2155](deephaven/web-client-ui#2155)) ([6362eb7](deephaven/web-client-ui@6362eb7)) * Remove the session wrapper from the embedded widget app ([#2158](deephaven/web-client-ui#2158)) ([b76c1d7](deephaven/web-client-ui@b76c1d7)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.86.0 ### Features * Add option to disable WebGL rendering ([#2134](deephaven/web-client-ui#2134)) ([011eb33](deephaven/web-client-ui@011eb33)) * Core plugins refactor, XComponent framework ([#2150](deephaven/web-client-ui#2150)) ([2571fad](deephaven/web-client-ui@2571fad)) * IrisGridTheme iconSize ([#2123](deephaven/web-client-ui#2123)) ([58ee88d](deephaven/web-client-ui@58ee88d)) * Partitioned Table UI Enhancements ([#2110](deephaven/web-client-ui#2110)) ([de5ce40](deephaven/web-client-ui@de5ce40)) ### Bug Fixes * error when edited cell is out of grid viewport ([#2148](deephaven/web-client-ui#2148)) ([3fccd43](deephaven/web-client-ui@3fccd43)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.85.2 ### Bug Fixes * Fix missing scrim background on LoadingOverlay ([#2098](deephaven/web-client-ui#2098)) ([c9ed895](deephaven/web-client-ui@c9ed895)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.85.1 ## [0.85.1](deephaven/web-client-ui@v0.85.0...v0.85.1) (2024-07-08) ### Bug Fixes * re-export remaining types needed by dh ui from @react-types/shared ([#2132](deephaven/web-client-ui#2132)) ([2119a61](deephaven/web-client-ui@2119a61)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.85.0 ### Features * ComboBox - @deephaven/jsapi-components ([#2077](deephaven/web-client-ui#2077)) ([115e057](deephaven/web-client-ui@115e057)) ### Bug Fixes * Allow ComboBox to accept the FocusableRef for ref ([#2121](deephaven/web-client-ui#2121)) ([8fe9bad](deephaven/web-client-ui@8fe9bad)) * Ref was not being passed through for Picker ([#2122](deephaven/web-client-ui#2122)) ([a11e2ce](deephaven/web-client-ui@a11e2ce)) ## Release notes https://github.com/deephaven/web-client-ui/releases/tag/v0.84.0 ### ⚠ BREAKING CHANGES * - @deephaven/jsapi-components - The contract of `useSearchableViewportData` to be more consistent with `useViewportData`. `usePickerWithSelectedValues` now requires `timeZone`. - @deephaven/jsapi-utils - `createSearchTextFilter` requires `timeZone` ### Features * useSearchableViewportData - support non-text filters ([#2092](deephaven/web-client-ui#2092)) ([7009e21](deephaven/web-client-ui@7009e21)) ### Bug Fixes * `isElementOfType` Improved type inference ([#2099](deephaven/web-client-ui#2099)) ([e13c9d7](deephaven/web-client-ui@e13c9d7)) * Console does not scroll to bottom when code run from notebook ([#2114](deephaven/web-client-ui#2114)) ([e75e716](deephaven/web-client-ui@e75e716)) * make textValue default to key for Normalized Item ([#2113](deephaven/web-client-ui#2113)) ([bd3e944](deephaven/web-client-ui@bd3e944)) * Update IrisGridContextMenuHandler getHeaderActions return type to be more permissive ([#2117](deephaven/web-client-ui#2117)) ([4e08b79](deephaven/web-client-ui@4e08b79))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We don't currently have efficient support for retaining the last N rows of a stream table.
stream.tail(N)
itself is a streaming table (ie, it doesn't retain)stream.tailBy(N)
might have the semantics we are looking for, but the current implementation doesn't work for streaming tables (implemented usinggroupBy
which doesn't work on streaming tables)..tailBy(N)
may be more powerful than we need though, since it also supports grouping columns. (Follow-up question if.tailBy(N, ...)
should semantically ungroup automatically or not.) That said, retaining the last N rows of a stream table per group may be desired functionality as well.It should be relatively straightforward in either case; but the case without the grouping columns will probably be easier.
The text was updated successfully, but these errors were encountered: