Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs/nia: reorganize tf module details to top level nav #9830

Merged
merged 2 commits into from
Feb 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions website/content/docs/integrate/nia-integration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ Consul-Terraform-Sync compatible Terraform module development process is fairly

- Consul [documentation](/docs)
- Consul-Terraform-Sync [documentation](/docs/nia)
- Writing Consul-Terraform-Sync compatible Terraform modules [guide](/docs/nia/installation/requirements#how-to-create-a-compatible-terraform-module)
- Writing Consul-Terraform-Sync compatible Terraform modules [guide](/docs/nia/terraform-modules)
- Example [module](https://registry.terraform.io/modules/PaloAltoNetworks/dag-nia/panos/latest) for reference
- Publishing to the Terraform Registry [guidelines](https://www.terraform.io/docs/registry/modules/publish.html)

### 3. Develop & Test

Terraform modules are written in HashiCorp Configuration Language (HCL). Writing [Terraform modules](https://www.terraform.io/docs/modules/index.html) or a [tutorial to build a module](https://learn.hashicorp.com/tutorials/terraform/module-create) are good resources to begin writing a new module.
Consul-Terraform-Sync compatible modules follow the [standard module structure](https://www.terraform.io/docs/modules/index.html#standard-module-structure). Modules can use syntax supported by Terraform version 0.13, or higher. Consul-Terraform-Sync is designed to integrate with any module that satisfies these [specifications](/docs/nia/installation/requirements#how-to-create-a-compatible-terraform-module). The guide will give you an introduction of the code structure and the basics of authoring a plugin that Terraform can interact with.
Consul-Terraform-Sync compatible modules follow the [standard module structure](https://www.terraform.io/docs/modules/index.html#standard-module-structure). Modules can use syntax supported by Terraform version 0.13, or higher. Consul-Terraform-Sync is designed to integrate with any module that satisfies these [specifications](/docs/nia/terraform-modules#module-specifications). The guide will give you an introduction of the code structure and the basics of authoring a plugin that Terraform can interact with.

It is recommended that partners develop modules that cater to specific workflows on an individual platform in their product portfolio rather than having overarching modules that try to cover multiple workflows across different platforms. This is to keep the automation modular and adoptable by a broad set of users with varying network infrastructure topologies. Partners are encouraged to test the functionality of the modules against their supported platforms.

Expand Down Expand Up @@ -87,15 +87,15 @@ Once the module development has been completed another email should be sent to n

At this stage, it is expected that the module is fully developed, all tests and documentation are in place, and that HashiCorp has reviewed the module to be compatible with Consul-Terraform-Sync.

Once this is done, HashiCorp will get the new module listed as Consul-Terraform-Sync compatible on [consul.io](/docs/nia/installation/requirements#using-terraform-modules), and then the partner will be asked to publish the Terraform module to the [Terraform Registry](https://registry.terraform.io/browse/modules).
Once this is done, HashiCorp will get the new module listed as Consul-Terraform-Sync compatible on [consul.io](/docs/nia/installation/requirements#partner-terraform-modules), and then the partner will be asked to publish the Terraform module to the [Terraform Registry](https://registry.terraform.io/browse/modules).
eikenb marked this conversation as resolved.
Show resolved Hide resolved

### 6. Support

Many partners view the release step to be the end of the journey, while at HashiCorp we view it to be the start. Getting the Consul-Terraform-Sync compatible module built is just the first step in enabling users to use it against the infrastructure. Once this is done, on-going effort is required to maintain the module and address any issues in a timely manner.

The expectation is to resolve all critical issues within 48 hours and all other issues within 5 business days. HashiCorp Consul and Terraform have an extremely wide community of users and contributors and we encourage everyone to report issues however small, as well as help resolve them when possible.

Partners who choose to not follow the process of NIA Integration Program for their Consul-Terraform-Sync compatible Terraform modules will not have their modules listed on [consul.io](/docs/nia/installation/requirements#using-terraform-modules).
Partners who choose to not follow the process of NIA Integration Program for their Consul-Terraform-Sync compatible Terraform modules will not have their modules listed on [consul.io](/docs/nia/installation/requirements#partner-terraform-modules).

### Contact Us

Expand Down
2 changes: 1 addition & 1 deletion website/content/docs/nia/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ discovered IP addresses for a set of Consul services.

A driver encapsulates the resources required to communicate the updates to the
network infrastructure. Terraform is the initial supported driver. It comes
with a set of providers that [enables supporting](/docs/nia/installation/requirements#how-to-create-a-compatible-terraform-module) a wide variety of
with a set of providers that [enables supporting](/docs/nia/terraform-modules) a wide variety of
infrastructure applications.
6 changes: 3 additions & 3 deletions website/content/docs/nia/index.mdx
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
layout: docs
page_title: Network Infrastructure Automation
sidebar_title: Network Infrastructure Automation <sup>Tech Preview</sup>
sidebar_title: Network Infrastructure Automation <sup>Beta</sup>
description: >-
Network Infrastructure Automation (NIA) is the concept of dynamically updating infrastructure devices triggered by service changes. Consul-Terraform-Sync is a tool that performs NIA and utilizes Consul as a data source that contains networking information about services and monitors those services. Terraform is used as the underlying automation tool and leverages the Terraform provider ecosystem to drive relevant changes to the network infrastructure.
---

# Network Infrastructure Automation <sup>Tech Preview</sup>
# Network Infrastructure Automation <sup>Beta</sup>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch with these 'Tech Preview's!


Network Infrastructure Automation (NIA) enables dynamic updates to network infrastructure devices triggered by service changes. Consul-Terraform-Sync utilizes Consul as a data source that contains networking information about services and monitors those services. Terraform is used as the underlying automation tool and leverages the Terraform provider ecosystem to drive relevant changes to the network infrastructure.

Expand Down Expand Up @@ -37,4 +37,4 @@ Consul-Terraform-Sync executes one or more automation tasks with the most recent
- [Contribute](https://github.com/hashicorp/consul-terraform-sync) to the open source project
- [Report](https://github.com/hashicorp/consul-terraform-sync/issues) bugs or request enhancements
- [Discuss](https://discuss.hashicorp.com/tags/c/consul/29/consul-terraform-sync) with the community or ask questions
- [Build integrations](/docs/nia/installation/requirements#how-to-create-a-compatible-terraform-module) for Consul-Terraform-Sync
- [Build integrations](/docs/nia/terraform-modules) for Consul-Terraform-Sync
105 changes: 6 additions & 99 deletions website/content/docs/nia/installation/requirements.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ If you would like to run a Consul cluster rather than a single agent, refer to [

Consul-Terraform-Sync integrations for the Terraform driver utilizes Terraform providers as plugins to interface with specific network infrastructure platforms. The Terraform driver of Consul-Terraform-Sync inherits the expansive collection of Terraform providers to integrate with, and with release of [Terraform 0.13](https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13/), this extends to include providers written by the community too by using [provider source](https://www.hashicorp.com/blog/adding-provider-source-to-hashicorp-terraform/).

### Finding and Using Terraform Providers
### Finding Terraform Providers

To find providers for the infrastructure platforms you use, browse the providers section of the [Terraform Registry](https://registry.terraform.io/browse/providers).

Expand All @@ -69,11 +69,13 @@ If there is no existing Terraform provider, a new Terraform provider can be [cre

## Network Integration (using a Terraform Module)

Working with a Terraform provider, you can write an integration task for Consul-Terraform-Sync by creating a Terraform module that is compatible with the Terraform driver.
The Terraform module for a task in Consul-Terraform-Sync is the core component of the integration. It declares which resources and how your infrastructure is dynamically updated.

-> **Note:** [Release 0.1.0-techpreview2](https://github.com/hashicorp/consul-terraform-sync/releases/tag/v0.1.0-techpreview2) is compatible with Terraform modules with syntax supported by Terraform version [0.13](https://github.com/hashicorp/terraform/blob/v0.13/CHANGELOG.md) - [0.14](https://github.com/hashicorp/terraform/blob/v0.14/CHANGELOG.md).
Working with a Terraform provider, you can write an integration task for Consul-Terraform-Sync by [creating a Terraform module](/docs/nia/terraform-modules) that is compatible with the Terraform driver or use a module built by partners below.

### Using Terraform Modules
Continue to the next page to [get started with configuring Consul-Terraform-Sync and how to use Terraform providers and modules for tasks.](/docs/nia/installation/configure)

### Partner Terraform Modules

The modules listed below are availabe to use and are compatible with Consul-Terraform-Sync.

Expand All @@ -97,98 +99,3 @@ The modules listed below are availabe to use and are compatible with Consul-Terr

- Dynamic Address Group (DAG) Tags: [Terraform Registry](https://registry.terraform.io/modules/PaloAltoNetworks/dag-nia/panos/latest) / [GitHub](https://github.com/PaloAltoNetworks/terraform-panos-dag-nia)
- Address Group and Dynamic Address Group (DAG) Tags: [Terraform Registry](https://registry.terraform.io/modules/PaloAltoNetworks/ag-dag-nia/panos/latest) / [GitHub](https://github.com/PaloAltoNetworks/terraform-panos-ag-dag-nia)

### How to Create a Compatible Terraform Module

-> **Note:** Consul-Terraform-Sync is currently in Beta. Specifications in this section may change going into the General Availability release.

You can read more on how to [create a module](https://www.terraform.io/docs/modules/index.html) or work through a [tutorial to build a module](https://learn.hashicorp.com/tutorials/terraform/module-create). Consul-Terraform-Sync is designed to integrate with any module that satisfies the specifications in the following section.

The repository [hashicorp/consul-terraform-sync-template-module](https://github.com/hashicorp/consul-terraform-sync-template-module) can be cloned and used as a starting point for structuring a compatible Terraform module.

#### Module Specifications

Compatible modules for Consul-Terraform-Sync follow the [standard module](https://www.terraform.io/docs/modules/index.html#standard-module-structure) structure. Modules can use syntax supported by Terraform version 0.13 and newer. There are **2 required elements for compatibility:**

1. **Root module** - Terraform has one requirement for files in the root directory of the repository to function as the primary entrypoint for the module. It should encapsulate the core logic to be used by Consul-Terraform-Sync for task automation. `main.tf` is the recommended filename for the main file where resources are created.
2. **`services` input variable** - Consul-Terraform-Sync requires all modules to have the following input variable declared within the root module. The declaration of the `services` variable can be included at the top of the suggested `variables.tf` file where other input variables are commonly declared. This variable functions as the response object from the Consul catalog API and surfaces network information to be consumed by the module. It is structured as a map of objects.

```hcl
variable "services" {
description = "Consul services monitored by Consul-Terraform-Sync"
type = map(
object({
id = string
name = string
address = string
port = number
meta = map(string)
tags = list(string)
namespace = string
status = string

node = string
node_id = string
node_address = string
node_datacenter = string
node_tagged_addresses = map(string)
node_meta = map(string)

cts_user_defined_meta = map(string)
})
)
}
```

Keys of the `services` map are unique identifiers of the service across Consul agents and data centers. Keys follow the format `service-id.node.datacenter` (or `service-id.node.namespace.datacenter` for Consul Enterprise). A complete list of attributes available for the `services` variable is included in the [documentation for Consul-Terraform-Sync tasks](/docs/nia/tasks#task-execution).

Terraform variables when passed as module arguments can be [lossy for object types](https://www.terraform.io/docs/configuration/types.html#conversion-of-complex-types). This allows Consul-Terraform-Sync to declare the full variable with every object attribute in the generated root module, and pass the variable to a child module that contains a subset of these attributes for its variable declaration. Modules compatible with Consul-Terraform-Sync may simplify the services variable within the module by omitting unused attributes. For example, the following services variable has 4 attributes with the rest omitted.

```hcl
variable "services" {
description = "Consul services monitored by Consul-Terraform-Sync"
type = map(
object({
id = string
name = string
node_address = string
port = number
status = string
})
)
}
```

#### Module Input Variables

Network infrastructure differs vastly across teams and organizations, and the automation needs of practitioners are unique based on their existing setup. [Input variables](https://www.terraform.io/docs/configuration/variables.html) can be used to serve as customization parameters to the module for practitioners.

1. Identify areas in the module where practitioners could tailor the automation to fit their infrastructure.
2. Declare input variables and insert the use of variables throughout module resources to expose these options to practitioners.
3. Include descriptions to capture what the variables are and how they are used, and specify [custom validation rules for variables](https://www.terraform.io/docs/configuration/variables.html#custom-validation-rules) to provide context to users the expected format and conditions for the variables.
4. Set reasonable default values for variables that are optional, or omit default values for variables that are required module arguments.
5. Set the [sensitive argument](https://www.terraform.io/docs/language/values/variables.html#suppressing-values-in-cli-output) for variables that contain secret or sensitive values. When set, Terraform will redact the value from output when Terraform commands are run.

Terraform is an explicit configuration language and requires variables to be declared, typed, and passed explicitly through as module arguments. Consul-Terraform-Sync abstracts this by creating intermediate variables at the root level from values intended for the module. These values are configured by practitioners within the [`task` block](/docs/nia/configuration#variable_files). Value assignments are parsed to interpolate the corresponding variable declaration and are written to the appropriate Terraform files. A few assumptions are made for the intermediate variables: the variables users provide Consul-Terraform-Sync are declared and supported by the module, matching name and type.

#### Module Guidelines

This section covers guidelines for authoring compatible Consul-Terraform-Sync modules.

##### Scope

We recommend scoping the module to a few related resources for a provider. Small modules are easier and more flexible for end users to adopt for Consul-Terraform-Sync. It allows them to iteratively combine different modules and use them as building blocks to meet their unique network infrastructure needs.

##### Complexity

Consider authoring modules with low complexity to reduce the run time for Terraform execution. Complex modules that have a large number of dependencies may result in longer runs, which adds delay to the near real time network updates.

##### Providers

The Terraform module must declare which providers it requires within the [`terraform.required_providers` block](https://www.terraform.io/docs/language/providers/requirements.html#requiring-providers). We suggest to also include a version constraint for the provider to specify which versions the module is compatible with.

Aside from the `required_providers` block, provider configurations should not be included within the sharable module for network integrations. End users will configure the providers through Consul-Terraform-Sync, and Consul-Terraform-Sync will then translate provider configuration to the generated root module appropriately.

##### Documentation

Modules for Consul-Terraform-Sync are Terraform modules and can effectively run independently from the `consul-terraform-sync` daemon and Consul environment. They should be written and designed with Terraform best practices and should be clear to a Terraform user what the module does and how to use it. Module documentation should be named `README` or `README.md`. The description should capture what the module should be used for and the implications of running it in automation with Consul-Terraform-Sync.
2 changes: 1 addition & 1 deletion website/content/docs/nia/network-drivers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ The following files of the root module are generated for each task. An [example
- `terraform` block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here.
- `provider` blocks - The provider blocks generated in the root module resemble the `terraform_provider` blocks in the configuration. They have identical arguments present and are set from the intermediate variable created per provider.
- `module` block - The module block is where the task's module is called as a [child module](https://www.terraform.io/docs/configuration/modules.html#calling-a-child-module). The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
- `variables.tf` - This file contains 2 types of variable declarations. The required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/installation/requirements#how-to-create-a-compatible-terraform-module) for more details) and various intermediate variables used to dynamically configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
- `variables.tf` - This file contains 2 types of variable declarations. The required `services` input variable which determines module compatibility with Consul-Terraform Sync (read more on [compatible Terraform modules](/docs/nia/terraform-modules) for more details) and various intermediate variables used to dynamically configure providers. These intermediate provider variables are interpolated from the provider blocks and arguments configured in the Consul-Terraform-Sync configuration.
- `variables.module.tf` - This file is conditionally created if there are [variables configured for the task](/docs/nia/configuration#variable_files) and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
- `terraform.tfvars` - The variable definitions file is where the services input variable is assigned values from the Consul catalog. It is periodically updated to reflect the current state of the configured set of services for the task.
- `terraform.tfvars.tmpl` - The template file is used by Consul-Terraform-Sync to template service information from the Consul catalog by using the HashiCorp configuration and templating library ([hashicorp/hcat](https://github.com/hashicorp/hcat)).
Expand Down
Loading