Improve the SQL Lab to Explore flow UX

[benceorlai]

Motivation / Problem to solve / Target Users

A SQL Lab user can take the results of a query and open it Explore for visualization by clicking on the 'Explore' button in SQL Lab. This flow can be used by data analysts or data scientists to quickly explore visually the results of a SQL query. When this transition happens, Superset creates a new virtual dataset in the background, which becomes the datasource for the new visualization in Explore. There are a number of UX/UI issues with this flow:

• The user is not made aware that a new virtual dataset is being created in the background
• The user has no control over what the new virtual dataset is named, and a new name is generated using a random string suffix
• The user is always required to create a new dataset, even when updating an existing dataset is desired, which leads to the creation of unnecessary new datasets

Once the user lands in Explore, the UX/UI to change or edit the datasource has several issues:

• In the Change datasource flow, only physical datasets can be selected, the user can not select any virtual datasets
• In the Edit Datasource flow
  • the editing experience is different from the experience available from the Datasets CRUD view (as of October 20, 2020 the two experiences are almost identical since PR-10731 has been merged, but bugs remain)
  • the 'Use Legacy Datasource editor' option on the points to a deprecated UI
  • SYNC COLUMNS FROM DATASOURCE does not work for virtual datasets
  • on the Source tab, if the user changes the Dataset name, the changes are not saved (bug - looks fixed as of 10/20)
  • on the Metric tab, editing the column name, and the column metadata are two disparate UIs
  • on the Calculated column tab, editing the column name, and the column metadata are two disparate UIs

Summary of proposed solution

The scope of this project is to implement the following changes

1. When the user navigates from SQL Lab to Explore
   1. Inform them that a new virtual dataset is being created, and let them cancel the process
   2. Offer two options: create a new dataset or update an existing one
   3. For the new dataset option allow the user to enter a name for the dataset (instead of the randomly generated string)
2. In the Explore screen
   1. Change the order of the options in the overflow menu: Edit Datasource, Change Datasource, Explore in SQL Lab
   2. Make the Edit Datasource experience completely identical with the one available from the Datasets list view screen (pencil icon)
   3. Add support for SYNC COLUMNS FROM DATASOURCE in virtual datasets (PR-10645)
   4. Fix known issues in the Edit Datasource experience
   5. In the Change Datasource modal allow the user to select virtual datasets, as well as physical datasets
   6. In the Change Datasource modal add a warning when the user is about to change the datasource