Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Escape Hatch Raw Overrides - AWS Glue Table - Does not work #4188

Closed
dangarfield opened this issue Sep 21, 2019 · 2 comments · Fixed by #4197
Closed

Escape Hatch Raw Overrides - AWS Glue Table - Does not work #4188

dangarfield opened this issue Sep 21, 2019 · 2 comments · Fixed by #4197
Assignees
Labels
@aws-cdk/aws-glue Related to AWS Glue bug This issue is a bug. in-progress This issue is being actively worked on.

Comments

@dangarfield
Copy link

Attempting to add a property override through escape hatch to set the SerDeInfo parameters on a glue table through cdk. The table construct does not allow access to these parameters. I create the table through the standard glue.Table(...) construct and attempt to add property override through tableVar.node.defaultChild.addPropertyOverride(...) however, the defaultChild is undefined and throws an error. As such, I cannot use the override and instead have to create the whole table through CfnTable.

That being said, I apologise if I am just doing something incorrect. The exact scenario works 100% correctly for some other resources, eg s3.Bucket

Reproduction Steps

const exampleTable = new glue.Table(this, 'insights-table-photo-match-sales', {
      columns: [
        {name: 'id', type: glue.Schema.STRING},
        {name: 'another', type: glue.Schema.STRING},
        {name: 'count', type: glue.Schema.SMALL_INT},
        {name: 'name', type: glue.Schema.STRING},
        {name: 'date', type: glue.Schema.TIMESTAMP}
      ],
      dataFormat: glue.DataFormat.Json,
      database: insightsDB,
      tableName: 'photo-match-sales'
    })
photoMatchSalesTable.node.defaultChild.addPropertyOverride('TableInput.StorageDescriptor.SerdeInfo.parameters', {'path': 'id,ship,count,name,date'})

Error Log

   photoMatchSalesTable.node.defaultChild.addPropertyOverride('TableInput.StorageDescriptor.SerdeInfo.parameters', {'path': 'id,ship,count,name,date'})
                                           ^

TypeError: Cannot read property 'addPropertyOverride' of undefined
    at new DigitechInsightsStack (/Users/DanGarfield/code/digitech-insights/lib/digitech-insights-stack.js:47:44)
    at Object.<anonymous> (/Users/DanGarfield/code/digitech-insights/bin/digitech-insights.js:7:1)
    at Module._compile (internal/modules/cjs/loader.js:702:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
    at Module.load (internal/modules/cjs/loader.js:612:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
    at Function.Module._load (internal/modules/cjs/loader.js:543:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:744:10)
    at startup (internal/bootstrap/node.js:238:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:572:3)

Environment

  • CLI Version : 1.8.0
  • Framework Version: 1.8.0
  • OS : OSX
  • Language : JS & TS, both

Other


This is 🐛 Bug Report

@dangarfield dangarfield added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 21, 2019
@nmussy
Copy link
Contributor

nmussy commented Sep 23, 2019

I'm pretty sure this can be easily fixed by setting CfnTable as the default child of the Table construct:

const tableResource = new CfnTable(this, 'Table', {

I'll make a PR

@nmussy
Copy link
Contributor

nmussy commented Sep 23, 2019

@dangarfield Once this is merged, you should be able to do the following:

const cfnTable = table.node.defaultChild as glue.CfnTable;
const tableInput = cfnTable.tableInput as glue.CfnTable.TableInputProperty;
const storageDescriptor = tableInput.storageDescriptor as glue.CfnTable.StorageDescriptorProperty;
const serdeInfo = storageDescriptor.serdeInfo as glue.CfnTable.SerdeInfoProperty;

// @ts-ignore - readonly
serdeInfo.parameters = {
  path: 'id,ship,count,name,date'
};

@SomayaB SomayaB added the @aws-cdk/aws-glue Related to AWS Glue label Sep 23, 2019
@SomayaB SomayaB added in-progress This issue is being actively worked on. and removed needs-triage This issue or PR still needs to be triaged. labels Sep 23, 2019
@mergify mergify bot closed this as completed in #4197 Sep 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-glue Related to AWS Glue bug This issue is a bug. in-progress This issue is being actively worked on.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants