Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Kerberos Authentication in Async Hive Sensors #357

Merged
merged 12 commits into from
May 23, 2022
Merged

Enable Kerberos Authentication in Async Hive Sensors #357

merged 12 commits into from
May 23, 2022

Conversation

rajaths010494
Copy link
Contributor

@rajaths010494 rajaths010494 commented May 17, 2022 •
edited
Loading

Previously HivePartitionSensorAsync & NamedHivePartitionSensorAsync supported only authMechansim='PLAIN'.
This PR add support to authMechansim='GSSAPI'

  • add auth mechanism Kerberos support for hive

closes #275

@rajaths010494
Add kerberos support for connection to hive
53491c7 
- add auth mechanism Kerberos to hive  sensor/operator
@codecov
Copy link

codecov bot commented May 17, 2022 •
edited
Loading

Codecov Report

Merging #357 (77bd440) into main (bcc2b29) will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #357      +/-   ##
==========================================
+ Coverage   96.78%   96.82%   +0.04%     
==========================================
  Files          56       56              
  Lines        2925     2931       +6     
==========================================
+ Hits         2831     2838       +7     
+ Misses         94       93       -1
Impacted Files Coverage Δ
...viders/apache/hive/sensors/named_hive_partition.py 100.00% <ø> (ø)
astronomer/providers/apache/hive/hooks/hive.py 100.00% <100.00%> (+2.27%) ⬆️
...er/providers/apache/hive/sensors/hive_partition.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bcc2b29...77bd440. Read the comment docs.

@pankajastro pankajastro changed the title Enhance HivePartitionSensor & NamedHivePartitionSensor to support Kerberos authentication Enable Kerberos Authentication for Hive May 17, 2022
@pankajastro pankajastro changed the title Enable Kerberos Authentication for Hive Enable Kerberos Authentication for Hive May 17, 2022
@pankajastro pankajastro changed the title Enable Kerberos Authentication for Hive Enable Kerberos Authentication for Hive May 17, 2022
@pankajastro pankajastro changed the title Enable Kerberos Authentication for Hive Enable Kerberos Authentication for Hive May 17, 2022
@pankajastro pankajastro changed the title Enable Kerberos Authentication for Hive Enable Kerberos Authentication for Hive May 17, 2022
@rajaths010494 rajaths010494 marked this pull request as ready for review May 17, 2022 14:33
phanikumv
phanikumv reviewed May 17, 2022
View reviewed changes
Copy link
Collaborator

@phanikumv phanikumv left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The subject line seems misleading, you are enabling kerberos for hive or Hive sensors ? @rajaths010494

phanikumv
phanikumv requested changes May 17, 2022
View reviewed changes
Copy link
Collaborator

@phanikumv phanikumv left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need these async sensors to be stress tested with 1000 tasks in a DAG, can you try that please?

@rajaths010494
Copy link
Contributor Author

The subject line seems misleading, you are enabling kerberos for hive or Hive sensors ? @rajaths010494

It's for hive update the same.

@rajaths010494
Copy link
Contributor Author

need these async sensors to be stress tested with 1000 tasks in a DAG, can you try that please?

I will run those today.

bharanidharan14
bharanidharan14 reviewed May 18, 2022
View reviewed changes
Copy link
Contributor

@bharanidharan14 bharanidharan14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks fine for me, only thing is you give more on the description part

@phanikumv
Copy link
Collaborator

The subject line seems misleading, you are enabling kerberos for hive or Hive sensors ? @rajaths010494

It's for hive update the same.

No, you are enabling the Hive Sensors to work with Kerberos authentication.

phanikumv
phanikumv reviewed May 23, 2022
View reviewed changes
Copy link
Collaborator

@phanikumv phanikumv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pankajastro please fix these review comment

astronomer/providers/apache/hive/sensors/hive_partition.py Outdated Show resolved Hide resolved
astronomer/providers/apache/hive/sensors/named_hive_partition.py Outdated Show resolved Hide resolved
@phanikumv phanikumv changed the title Enable Kerberos Authentication for Hive Enable Kerberos Authentication for Async Hive Sensors May 23, 2022
@pankajastro pankajastro force-pushed the hive-kerberos-hook branch from ac9495b to 7531092 Compare May 23, 2022 15:45
@phanikumv phanikumv changed the title Enable Kerberos Authentication for Async Hive Sensors Enable Kerberos Authentication in Async Hive Sensors May 23, 2022
@phanikumv phanikumv merged commit dc0256c into main May 23, 2022
@phanikumv phanikumv deleted the hive-kerberos-hook branch May 23, 2022 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bharanidharan14 bharanidharan14 bharanidharan14 left review comments

@phanikumv phanikumv phanikumv approved these changes

@pankajastro pankajastro Awaiting requested review from pankajastro pankajastro is a code owner

@pankajkoti pankajkoti Awaiting requested review from pankajkoti pankajkoti is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Build Async HivePartitionSensor & NamedHivePartitionSensor with GSSAPI (Kerberos authentication)
4 participants
@rajaths010494 @phanikumv @bharanidharan14 @pankajastro