Support auxiliary files in terraform deploy spec #272

endorama · 2022-02-09T10:04:57Z

What does this PR do?

add support for Terraform dependency lock file
add support for Terraform template files
add support for JSON files

Why is it important?

Terraform suggest to add it's dependency lock file to VCS, as it helps with reproducible installations
Terraform support template files, with .tftpl suggested extension
JSON files may be used for ausiliary content required by Terraform code.

Further discussion available in #269.

Checklist

I have added test packages to test/packages that prove my change is effective.
I have added an entry in versions/N/changelog.yml.

Related issues

Closes [Change Proposal] Support Terraform related files in deploy/tf #269

endorama · 2022-02-09T10:09:25Z

@mtojek I have a question about the changelog. The file header says

[...]
## Newer entries go at the bottom.
[...]

but it seems the opposite is true? Am I expected to append to v1.4.1 as is still unreleased or create a new version?

elasticmachine · 2022-02-09T10:11:09Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-02-09T12:29:55.398+0000
Duration: 6 min 4 sec

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

jsoriano · 2022-02-09T11:23:45Z

but it seems the opposite is true? Am I expected to append to v1.4.1 as is still unreleased or create a new version?

Put it at the bottom of the 1.4.1 section, yes :)

jsoriano · 2022-02-09T11:27:39Z

versions/1/changelog.yml

@@ -10,6 +10,9 @@
  - description: Prepare for next version
    type: enhancement
    link: https://github.com/elastic/package-spec/pull/265
+  - description: Support ausiliary files in terraform deploy spec
+    type: enhancement
+    link: https://github.com/elastic/package-spec/pull/272


endorama · 2022-02-09T11:28:18Z

I added a deploy_terraform test package, but I'm not sure is being used 🤔 given that I also added the same files to package good, should I remove it?

jsoriano · 2022-02-09T11:31:05Z

I added a deploy_terraform test package, but I'm not sure is being used thinking given that I also added the same files to package good, should I remove it?

Test packages are used in code/go/pkg/validator/validator_test.go, you will need to add the package there.

jsoriano

Some nitpicking, but LGTM in general.

jsoriano · 2022-02-09T11:38:05Z

test/packages/deploy_terraform/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,3 @@
+version: '2.3'
+services:
+  hello_world:


Nit. Is this file needed here? I guess it doesn't do any harm though.

leftover from copying over the deploy_docker module, thank you for catching this.

jsoriano · 2022-02-09T11:40:57Z

test/packages/good/data_stream/foo/_dev/deploy/tf/env.yml

Unrelated changes?

No but given the file is not required I'll remove it

jsoriano · 2022-02-09T11:43:42Z

versions/1/changelog.yml

@@ -10,6 +10,9 @@
  - description: Prepare for next version
    type: enhancement
    link: https://github.com/elastic/package-spec/pull/265
+  - description: Support ausiliary files in terraform deploy spec


Nit. Should it be "auxiliary"?

Suggested change

- description: Support ausiliary files in terraform deploy spec

- description: Support auxiliary files in terraform deploy spec

jsoriano · 2022-02-09T11:46:44Z

versions/1/_dev/deploy/tf/spec.yml

+      pattern: '^*.json$'
+      type: file
+      contentMediaType: "application/json"
+      required: false


Perhaps we end-up needing additionalContents: false if we find scenarios that would need arbitrary files. But we can reduce restrictions in the future if we find these cases.

jsoriano · 2022-02-09T11:50:35Z

versions/1/_dev/deploy/tf/spec.yml

+    - description: Terraform dependency lock file
+      pattern: ".terraform.lock.hcl"
+      type: file
+      required: true


Btw, this may be breaking for current packages. At least we will need to add this file to the AWS package when updating the spec in the integrations repo.
https://github.com/elastic/integrations/tree/8e600daead4b89e97271e4976b984722c8db7c74/packages/aws/data_stream/ec2_metrics/_dev/deploy/tf

Nice find, actually I don't want to introduce a breaking change here. Do we have a process here? I will make this not required, but I think it make sense to have it everywhere. Is there a way to notify users of this change? For example outputting a message: "this field is going to default to true from version x.y.z"

Maybe we can add this as optional by now, and after some time make it required. At the moment I think that only the AWS package uses it, so not many users affected :)

Yeah, let's mark it optional for now.

I would have liked to set this file as required, but it would introduce a breaking change. So I added a note to update this requirement in the future.

mtojek

LGTM

endorama requested a review from a team as a code owner February 9, 2022 10:04

endorama self-assigned this Feb 9, 2022

endorama requested a review from mtojek February 9, 2022 10:04

endorama force-pushed the 269-terraform-improvements branch from 611ac47 to 0c75cd1 Compare February 9, 2022 10:05

endorama force-pushed the 269-terraform-improvements branch from dc3ecfd to 04f26f8 Compare February 9, 2022 11:19

jsoriano reviewed Feb 9, 2022

View reviewed changes

endorama force-pushed the 269-terraform-improvements branch from 4b38af4 to dc0c152 Compare February 9, 2022 11:36

jsoriano previously approved these changes Feb 9, 2022

View reviewed changes

jsoriano reviewed Feb 9, 2022

View reviewed changes

endorama changed the title ~~Support ausiliary files in terraform deploy spec~~ Support auxiliary files in terraform deploy spec Feb 9, 2022

endorama added 5 commits February 9, 2022 13:28

add support for Terraform dependency lock file

ceb40eb

I would have liked to set this file as required, but it would introduce a breaking change. So I added a note to update this requirement in the future.

add support for JSON files

65a7153

add deploy_terraform test package

65f3af5

add terraform deploy files to good test package

4ab80a2

add changelog entry

9652403

endorama dismissed jsoriano’s stale review via 9652403 February 9, 2022 12:29

endorama force-pushed the 269-terraform-improvements branch from dc0c152 to 9652403 Compare February 9, 2022 12:29

mtojek approved these changes Feb 9, 2022

View reviewed changes

jsoriano approved these changes Feb 9, 2022

View reviewed changes

jsoriano merged commit 5462c12 into elastic:main Feb 9, 2022

endorama deleted the 269-terraform-improvements branch February 9, 2022 15:52

endorama mentioned this pull request Mar 2, 2022

Migrate some Metricbeat modules to GCP package elastic/integrations#490

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support auxiliary files in terraform deploy spec #272

Support auxiliary files in terraform deploy spec #272

endorama commented Feb 9, 2022 •

edited

Loading

endorama commented Feb 9, 2022

elasticmachine commented Feb 9, 2022 •

edited

Loading

Build stats

jsoriano commented Feb 9, 2022

jsoriano Feb 9, 2022

endorama commented Feb 9, 2022

jsoriano commented Feb 9, 2022

jsoriano left a comment

jsoriano Feb 9, 2022

endorama Feb 9, 2022

jsoriano Feb 9, 2022

endorama Feb 9, 2022

jsoriano Feb 9, 2022

jsoriano Feb 9, 2022

jsoriano Feb 9, 2022 •

edited

Loading

endorama Feb 9, 2022

jsoriano Feb 9, 2022 •

edited

Loading

mtojek Feb 9, 2022

mtojek left a comment

	- description: Support ausiliary files in terraform deploy spec
	- description: Support auxiliary files in terraform deploy spec

Support auxiliary files in terraform deploy spec #272

Support auxiliary files in terraform deploy spec #272

Conversation

endorama commented Feb 9, 2022 • edited Loading

What does this PR do?

Why is it important?

Checklist

Related issues

endorama commented Feb 9, 2022

elasticmachine commented Feb 9, 2022 • edited Loading

💚 Build Succeeded

Build stats

🤖 GitHub comments

jsoriano commented Feb 9, 2022

Choose a reason for hiding this comment

endorama commented Feb 9, 2022

jsoriano commented Feb 9, 2022

jsoriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano Feb 9, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano Feb 9, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtojek left a comment

Choose a reason for hiding this comment

endorama commented Feb 9, 2022 •

edited

Loading

elasticmachine commented Feb 9, 2022 •

edited

Loading

jsoriano Feb 9, 2022 •

edited

Loading

jsoriano Feb 9, 2022 •

edited

Loading