Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

notebook for transpilation added #1421

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

sriram251-code
Copy link
Contributor

@sriram251-code sriram251-code commented Jan 17, 2025

Added a new notebook which can be use to perform transpilation using Remorph.The notebook includes instructions and code for installing the Remorph library, setting up necessary configuration parameters, and executing the transpilation process.Updated the README.md with the above details.

@sriram251-code sriram251-code self-assigned this Jan 17, 2025
@sriram251-code sriram251-code added the enhancement New feature or request label Jan 17, 2025
@sriram251-code sriram251-code marked this pull request as ready for review January 17, 2025 02:38
@sriram251-code sriram251-code requested a review from a team as a code owner January 17, 2025 02:38
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to put this file under src/databricks/labs/remorph/resources/transpile and copy this to remorph install directory during remorph installation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This notebook should source notebook not stored as .dbc and should be installed inside installation directory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

README.md Outdated

## NOTEBOOK LINK(TRANSPILATION only)
![Remorph-Transpilation.dbc](docs/notebook/Remorph-GA-Release090.dbc)
We would like to inform you that we have introduced a notebook-based installation process. However, please note that this notebook is exclusively designed for the purpose of transpilation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the README, we can suggest the user to refer to the notebook example given in the resource dir (see comment below for .dbc file) and use it if they don't want to use the usual installation process but still want to use a notebook.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This notebook should source notebook not stored as .dbc and should be installed inside installation directory.

Copy link
Contributor

@ericvergnaud ericvergnaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This notebook doesn't seem to have been tested against the current code base.
It might be ok to distribute it manually until the next release, but not as part of the code base since it would completely break with the next release


# Initialize the WorkspaceClient with the specified product and version and TranspileConfig object with the provided configuration parameters.
wsclient = WorkspaceClient(product="remorph", product_version="0.9.0")
mrophconfig = MorphConfig(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo in mrophconfig should be morph_config

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

# Initialize the WorkspaceClient with the specified product and version and TranspileConfig object with the provided configuration parameters.
wsclient = WorkspaceClient(product="remorph", product_version="0.9.0")
mrophconfig = MorphConfig(
source=source_dialect,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be source_dialect

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

wsclient = WorkspaceClient(product="remorph", product_version="0.9.0")
mrophconfig = MorphConfig(
source=source_dialect,
input_sql=input_source,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be input_source

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@sriram251-code
Copy link
Contributor Author

This notebook doesn't seem to have been tested against the current code base. It might be ok to distribute it manually until the next release, but not as part of the code base since it would completely break with the next release

Yes @ericvergnaud i agree that it will break for next release. But some customers are asking for it for the currect release so making this version of notebook for current release.

@ericvergnaud
Copy link
Contributor

This notebook doesn't seem to have been tested against the current code base. It might be ok to distribute it manually until the next release, but not as part of the code base since it would completely break with the next release

Yes @ericvergnaud i agree that it will break for next release. But some customers are asking for it for the currect release so making this version of notebook for current release.

@sriram251-code Then shouldn't it be distributed manually to them, since this PR is not compatible with the current code ?
What does this PR achieve ?

Copy link
Contributor

@ericvergnaud ericvergnaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please run make fmt before submitting


# Import necessary classes from the databricks SDK and remorph library
from databricks.sdk import WorkspaceClient
from databricks.labs.remorph.config import MorphConfig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use TranspileConfig


# Initialize the WorkspaceClient with the specified product and version and TranspileConfig object with the provided configuration parameters.
wsclient = WorkspaceClient(product="remorph", product_version="0.9.0")
mrophconfig = MorphConfig(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be TranspileConfig

)


status = execute.morph(workspace_client=wsclient, config=mrophconfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be execute.transpile, and is missing a TranspileEngine argument

@sriram251-code
Copy link
Contributor Author

sriram251-code commented Jan 21, 2025

This notebook doesn't seem to have been tested against the current code base. It might be ok to distribute it manually until the next release, but not as part of the code base since it would completely break with the next release

Yes @ericvergnaud i agree that it will break for next release. But some customers are asking for it for the currect release so making this version of notebook for current release.

@sriram251-code Then shouldn't it be distributed manually to them, since this PR is not compatible with the current code ? What does this PR achieve ?

@ericvergnaud This code is very specific to 090 version. This notebook can be used for all the clients who want to use remorph now.We will be making the changes for the next released version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants