Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INFO level logging is missing on Databricks #3639

Closed
noklam opened this issue Feb 21, 2024 · 5 comments
Closed

INFO level logging is missing on Databricks #3639

noklam opened this issue Feb 21, 2024 · 5 comments

Comments

@noklam
Copy link
Contributor

noklam commented Feb 21, 2024

Description

On Databricks:
image

On IPython:
image

Context

I assume all INFO level logging are not visible, I had issues with Databricks so I cannot run the pipeline.

Steps to Reproduce

  1. Load up the %load_ext kedro.ipython

Expected Result

The info logging should be shown on Databricks.

Actual Result

The INFO loggging is missing

@noklam noklam added this to the Something about logging milestone Feb 26, 2024
@merelcht merelcht moved this to To Do in Kedro Framework Apr 2, 2024
@DimedS DimedS self-assigned this Apr 9, 2024
@DimedS DimedS moved this from To Do to In Progress in Kedro Framework Apr 12, 2024
@DimedS
Copy link
Contributor

DimedS commented Apr 23, 2024

Hi @noklam , looks like it works well on my side. Could you please share your setup? Mine is Azure Databricks on Platform McK.
With %load_ext:
Screenshot 2024-04-23 at 15 36 16

and when running a pipeline:
Screenshot 2024-04-23 at 15 36 56

@noklam
Copy link
Contributor Author

noklam commented Apr 23, 2024

[email protected]:noklam/databricks-iris-19.git

I have the old repostory here, I don't think I have much time to look at it this week. I tried the platform mck but I struggled with setting up a databricks instance. We can either connect next week or if you could quickly clone the repo and see if it works.

@DimedS
Copy link
Contributor

DimedS commented Apr 24, 2024

[email protected]:noklam/databricks-iris-19.git

I have the old repostory here, I don't think I have much time to look at it this week. I tried the platform mck but I struggled with setting up a databricks instance. We can either connect next week or if you could quickly clone the repo and see if it works.

I tried your project and it works well. I can see the INFO logs. Let's discuss it further next week.

@noklam
Copy link
Contributor Author

noklam commented May 2, 2024

I couldn't reproduce the issue. Tried with 0.19.2 (the version that I reported this) and the latest 0.19.5, we can close this if there is nothing more to do.

@DimedS
Copy link
Contributor

DimedS commented May 3, 2024

We have confirmed that INFO logging messages are visible and functioning properly on Databricks. The issue is no longer relevant, so we are closing the ticket.

@DimedS DimedS closed this as completed May 3, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Kedro Framework May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

2 participants