refactor: Changes the DatabaseSelector and TableSelector to use the new Select component #16483
Conversation
| type DatabaseValue = { | ||
| label: string; | ||
| value: number; | ||
| id: number; | ||
| database_name: string; | ||
| backend: string; | ||
| }; |
There was a problem hiding this comment.
Changed the database type to include id, database_name, and backend.
| schema, | ||
| const [currentDb, setCurrentDb] = useState<DatabaseValue | undefined>( | ||
| db | ||
| ? { label: `${db.backend}: ${db.database_name}`, value: db.id, ...db } |
There was a problem hiding this comment.
Loaded db props in the initial value.
| } | ||
| if (onDbChange) { | ||
| onDbChange(db); | ||
| onDbChange(actualDb); |
There was a problem hiding this comment.
Changed to use actualDb instead of db
| value: { label: string; value: number }, | ||
| option: DatabaseValue, | ||
| ) { | ||
| const actualDb = option || db; |
There was a problem hiding this comment.
Changed the actualDB to use the option instead of value
| id: row.id, | ||
| database_name: row.database_name, | ||
| backend: row.backend, |
There was a problem hiding this comment.
Loaded the additional db information.
Codecov Report
@@ Coverage Diff @@
## master #16483 +/- ##
==========================================
- Coverage 76.95% 76.92% -0.03%
==========================================
Files 1007 1007
Lines 54149 54156 +7
Branches 7369 7369
==========================================
- Hits 41669 41661 -8
- Misses 12240 12255 +15
Partials 240 240
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
|
Here's an example of switching databases and querying different tables. Screen.Recording.2021-08-27.at.10.26.31.AM.mov |
|
/testenv up |
|
@michael-s-molina Ephemeral environment spinning up at http://54.212.66.149:8080. Credentials are |
|
This PR is also relevant and helps with the undefined engine bug: |
Thanks! Added a reference in the description. |
| expect(select).toBeInTheDocument(); | ||
| userEvent.click(select); | ||
| expect( | ||
| await screen.findByRole('option', { name: 'postgresql: test' }), |
There was a problem hiding this comment.
should we also add expect(props.onDbChange).toBeCalledTimes(1); here to catch the previous regression?
There was a problem hiding this comment.
Great suggestion. I created two new tests to catch the previous regression:
- Sends the correct db when changing the database
- Sends the correct schema when changing the schema
| @@ -227,10 +227,7 @@ class DatasourceControl extends React.PureComponent { | |||
| </Tooltip> | |||
| )} | |||
| {extra?.warning_markdown && ( | |||
| <WarningIconWithTooltip | |||
| warningMarkdown={extra.warning_markdown} | |||
| size={30} | |||
There was a problem hiding this comment.
can you attach a screenshot of this change? i seem to recall a reason for the different size
There was a problem hiding this comment.
Of course. I'm attaching a screenshot with both healthCheckMessage and warning_markdown enabled for comparison.
|
Some UX issues found during local testing:
Will take a look at the code once these issues are fixed. |
michael-s-molina
force-pushed
the
refactor-database-selector-selects
branch
2 times, most recently
from
14aa926 to
273231d
Compare
Done
Done in #16531
The idea here is to avoid unnecessary queries to the server. This component is used in Explore, SQL Editor, and the datasets list. Many times the user does not modify any of the selects but they are rendered. Some examples:
I do understand your concern though. Here we have a limitation because the tables endpoint does not currently support pagination. In talks with @villebro to add pagination support we ended up with the following:
So one of the reasons for the slow behavior is the lack of pagination support. For cases where the queries are heavy even with pagination, we created a prop called
This was a result of the lack of pagination support. The component thinks that we have more results available given the page size and total results. To overcome this, while we don't add pagination support to this endpoint, I added the following: This is just disabling the pagination in practice. We don't want to provide a prop to disable the pagination because we want to make sure all endpoints support this feature. This is especially important if we think about devices with slow connections that are using Superset endpoints.
I tested here with 50k values and the Select was able to render smoothly. Virtualization is native in Ant Design Select. In their page, we have an example with 100k items. So I don't know if it froze because of query performance or another reason. Maybe we can test again with
Fixed. @ktmud @etr2460 @villebro Thank you so much for the tests, reviews and help improving this feature. |
|
Table names and schema names are cached, so I don't think unnecessary db metadata queries are a huge concern. Re: pagination, shouldn't the Select itself has a built-in "async but no pagination" mode? Why is the pageSize hack necessary? The freeze might be related to the refetch. It indeed worked well with the initial rendering. |
|
@ktmud Let me provide you a little more context to help with the decision. When we started the select migration project, @geido compiled a great spreadsheet with all the different types of selects that we had and all the props that were available for each select type. We had a lot of props, more than 50! What we noticed was that this lead to multiple behaviors throughout the application and increased the complexity of components because of the different combinations. Then we paired with @villebro to design the component API and decided that we would try to diminish the number of props by establishing default behaviors and making the user experience more similar. Another objective was to decrease the complexity of the component. This context is important because we can resolve these two questions by adding new props to the component like
This can be resolved with the inclusion of the
The hack here is needed because the default behavior assumes that we should always try to fetch data using pagination. Adding a As these are subjective decisions, I still can add these props if you think they pay off. I just wanted to add the whole context to help with the decision. |
This is about improving the availability of UI elements for most cases. A lot of users land on SQL Lab page without immediately going to change the selected table, but when they do, it's better to have the list preloaded already. TBH I don't think pagination and async loading itself should be part of the default The Async behavior should either be an HOC or another hook. For example: const { options, onSearch } = useAsyncSelectOptions({
fetchOptions: ...,
preload: true
});
<Select options={options} onSearch={onSearch} .../>With the complexity abstracted away from the Select component itself, adding more options to the fetch behavior should not be that big of a concern. |
We consider this approach when designing the component but we chose to make it an integral part of the Select because we have UI behaviors that are more easily implemented as a native feature than as a HOC. It's not only about how to retrieve the data but the interaction with the component also changes.
We just replaced all the selects of the application and the majority of them require async loading. In many cases where the previous select received an options array, it was the parent component that was doing the fetch and managing the pagination. In fact, absorbing this behavior and avoiding this duplicated logic was one of the reasons to create the component in the first place. I think that in the interest of moving this PR forward, can we settle in creating the |
|
Most Select controls on the Explore page do not need async. Not sure if this would help, but recently I've created yet another Select component in another project using the amazing Downshift.js library and chakra-ui. It does not handle pagination yet, but the "letting hooks take over event handlers" approach that Downshift.js has taken really opened my eyes. Thought that would be relevant to the discussion here as well. This Select component also uses a custom render hook which has made it easy to break down a complex component into overridable smaller subcomponents. |
michael-s-molina
force-pushed
the
refactor-database-selector-selects
branch
3 times, most recently
from
2bb58c3 to
aa2ea31
Compare
Thanks for sharing this @ktmud. I'll definitely take a look for future improvements to the component. In the meantime, I removed the pagination and lazy loading, reverting the behavior to what it was before. We can continue with the review process. |
michael-s-molina
force-pushed
the
refactor-database-selector-selects
branch
2 times, most recently
from
bc70a61 to
f787ac8
Compare
|
/testenv up |
|
@michael-s-molina Ephemeral environment spinning up at http://54.185.15.154:8080. Credentials are |
|
/testenv up |
|
@geido Ephemeral environment spinning up at http://34.216.189.221:8080. Credentials are |
…ew Select component
michael-s-molina
force-pushed
the
refactor-database-selector-selects
branch
from
f787ac8 to
03a2788
Compare
|
/testenv up |
|
@michael-s-molina Ephemeral environment spinning up at http://54.189.216.32:8080. Credentials are |
|
Ephemeral environment shutdown and build artifacts deleted. |
…ew Select component (apache#16483)
…ew Select component (apache#16483)
mistercrunch
added
🏷️ bot
SUMMARY
Follow-up of #16334 which introduced #16475 and was reverted in #16478.
You can find the descriptions and videos of the changes in the original PR #16334.
Fixes #16475
Thanks, @etr2460 for providing the revert PR.
@ktmud @etr2460 @villebro
I'll add comments indicating the relevant changes from the original PR.
Is also worth mentioning this fix by @AAfghahi #16472
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Check the original PR.
TESTING INSTRUCTIONS
Check the original PR.
ADDITIONAL INFORMATION