Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement]: aws_glue_crawler with delta_target does not contain Create Native Tables option #29336

Closed
GowthamiRamachandran opened this issue Feb 10, 2023 · 6 comments · Fixed by #29566
Labels
enhancement Requests to existing resources that expand the functionality or scope. service/glue Issues and PRs that pertain to the glue service.
Milestone

Comments

@GowthamiRamachandran
Copy link

GowthamiRamachandran commented Feb 10, 2023

Description

resource "aws_glue_crawler" "example" {
  database_name = aws_glue_catalog_database.example.name
  name          = "example"
  role          = aws_iam_role.example.arn

  delta_target {
    connection_name = ""
    delta_tables    = ["s3://path/"]
    write_manifest  = false
  }
}

For creating AWS Glue crawler with Delta Lake target we can create table for querying with Create Native tables Option in aws_glue_crawler, for delta_target the option is required.
Screenshot 2023-02-09 at 6 49 01 pm

Affected Resource(s) and/or Data Source(s)

Affected Resource: aws_glue_crawler
DataSource: delta_target

Potential Terraform Configuration

delta_target {
    connection_name = ""
    delta_tables    = ["s3://path/"]
    write_manifest  = false
    create_native_delta_table = true
  }

References

https://docs.aws.amazon.com/cli/latest/reference/glue/create-crawler.html
In AWS CLI for create-crawler "CreateNativeDeltaTable" : true option is available.

Would you like to implement a fix?

None

@GowthamiRamachandran GowthamiRamachandran added enhancement Requests to existing resources that expand the functionality or scope. needs-triage Waiting for first response or review from a maintainer. labels Feb 10, 2023
@github-actions
Copy link

Community Note

Voting for Prioritization

  • Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

  • If you are interested in working on this issue, please leave a comment.
  • If this would be your first contribution, please review the contribution guide.

@github-actions github-actions bot added the service/glue Issues and PRs that pertain to the glue service. label Feb 10, 2023
@ewbankkit ewbankkit removed the needs-triage Waiting for first response or review from a maintainer. label Feb 10, 2023
@tomwelsh11
Copy link
Contributor

I'm interested in working on this issue - will give it a crack

@tomwelsh11
Copy link
Contributor

I've opened a PR (2 weeks ago) that will close this issue and provide the required functionality - is there any indication on the time frame for this to be reviewed/merged? Not having this functionality is blocking a high value piece for us.

@mai-nguyen-coinhako
Copy link

Yes, I am looking forward to this updates too.

@github-actions github-actions bot added this to the v4.59.0 milestone Mar 10, 2023
@github-actions
Copy link

This functionality has been released in v4.59.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 16, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement Requests to existing resources that expand the functionality or scope. service/glue Issues and PRs that pertain to the glue service.
Projects
None yet
4 participants