Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adding Azure Log Analytics module #639

Merged
merged 34 commits into from
Feb 16, 2021
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c530565
loganalytics module
wmattlong Sep 15, 2020
c1e14f0
chg terraform rg prefix
wmattlong Sep 17, 2020
b2a2ec3
move unit test
wmattlong Sep 17, 2020
5acd83d
rm azure ref
wmattlong Sep 17, 2020
447c058
refactor test
wmattlong Sep 18, 2020
2907652
loganalytics module
wmattlong Sep 15, 2020
bea61cb
chg terraform rg prefix
wmattlong Sep 17, 2020
2198e94
move unit test
wmattlong Sep 17, 2020
308fccf
rm azure ref
wmattlong Sep 17, 2020
a68792e
refactor test
wmattlong Sep 18, 2020
693862c
Merge branch 'pull-request-632' of https://github.com/rguthriemsft/te…
wmattlong Sep 25, 2020
8c4c745
rebase
wmattlong Sep 25, 2020
2307bad
Merge branch 'master' into pull-request-632
wmattlong Sep 28, 2020
c56cfc0
rm spaces
wmattlong Sep 28, 2020
17a9bb3
add comment
wmattlong Sep 29, 2020
1cdf9a9
cleanup nits
wmattlong Sep 30, 2020
7e7e4a2
consistency
wmattlong Oct 2, 2020
89f207f
fix cmts + func
wmattlong Oct 2, 2020
7746ebb
rm space
wmattlong Oct 2, 2020
6d36c6e
unit test
wmattlong Oct 2, 2020
eb437e6
rm 0 lgth string for ws
wmattlong Oct 2, 2020
831c1d3
Merge branch 'master' into pull-request-632
wmattlong Oct 5, 2020
e4a3c0f
refactor exists
wmattlong Oct 5, 2020
883a488
ref unit test
wmattlong Oct 5, 2020
c7c3864
Merge branch 'master' into pull-request-632
Oct 10, 2020
fd4f6b1
update pin version comments.
Oct 10, 2020
709bb0f
Fix linting errors
Oct 10, 2020
c94b737
Merge branch 'master' into pull-request-632
Oct 15, 2020
576f475
fix spacing.
Oct 15, 2020
0c2f9d5
Merge remote-tracking branch 'upstream/master' into pull-request-632
HadwaAbdelhalem Feb 9, 2021
ce92b6b
use tolower gfor sku assert
HadwaAbdelhalem Feb 9, 2021
523537b
add // +build azure
HadwaAbdelhalem Feb 9, 2021
562402c
resolve PR comments
HadwaAbdelhalem Feb 11, 2021
39c9b4f
update loganalytics_test
HadwaAbdelhalem Feb 11, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions examples/azure/terraform-azure-loganalytics-example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Terraform Azure Log Analytics Example

This folder contains a Terraform module that deploys resources in [Azure](https://azure.microsoft.com/) to demonstrate
how you can use TerraTest to write automated tests for your Azure Terraform code. This module deploys a Log Analytics Workspace.

- A [Log Analytics Workspace](https://docs.microsoft.com/azure/azure-monitor/platform/log-analytics-agent) that gives the module the following:
- [Name](https://docs.microsoft.com/azure/azure-monitor/learn/quick-create-workspace#:~:text=%20Create%20a%20Log%20Analytics%20workspace%20in%20the,and%20region%20as%20in%20the%20deleted...%20More%20) with the value specified in the `loganalytics_workspace_name` output variable.
- [Sku](https://docs.microsoft.com/azure/azure-monitor/learn/quick-create-workspace#:~:text=%20Create%20a%20Log%20Analytics%20workspace%20in%20the,and%20region%20as%20in%20the%20deleted...%20More%20) with the value specified in the `loganalytics_workspace_sku` output variable.
- [RetentionPeriodInDays](https://docs.microsoft.com/azure/azure-monitor/learn/quick-create-workspace#:~:text=%20Create%20a%20Log%20Analytics%20workspace%20in%20the,and%20region%20as%20in%20the%20deleted...%20More%20) with the value specified in the `loganalytics_workspace_retention` output variable.

Check out [test/azure/terraform_azure_loganalytics_example_test.go](/test/azure/terraform_azure_loganalytics_example_test.go) to see how you can write
automated tests for this module.

Note that the Log Analytics Workspace in this module don't actually do anything; it just runs the resources for
skunklab marked this conversation as resolved.
Show resolved Hide resolved
demonstration purposes.

**WARNING**: This module and the automated tests for it deploy real resources into your Azure account which can cost you
money. The resources are all part of the [Azure Free Account](https://azure.microsoft.com/free/), so if you haven't used that up,
it should be free, but you are completely responsible for all Azure charges.

## Running this module manually

1. Sign up for [Azure](https://azure.microsoft.com/)
1. Configure your Azure credentials using one of the [supported methods for Azure CLI
tools](https://docs.microsoft.com/cli/azure/azure-cli-configuration?view=azure-cli-latest)
1. Install [Terraform](https://www.terraform.io/) and make sure it's on your `PATH`
1. Ensure [environment variables](../README.md#review-environment-variables) are available
1. Run `terraform init`
1. Run `terraform apply`
1. When you're done, run `terraform destroy`

## Running automated tests against this module

1. Sign up for [Azure](https://azure.microsoft.com/)
1. Configure your Azure credentials using one of the [supported methods for Azure CLI
tools](https://docs.microsoft.com/cli/azure/azure-cli-configuration?view=azure-cli-latest)
1. Install [Terraform](https://www.terraform.io/) and make sure it's on your `PATH`
1. Configure your TerraTest [Go test environment](../README.md)
1. `cd test/azure`
1. `go build terraform_azure_loganalytics_example_test.go`
1. `go test -v -run TestTerraformAzureLogAnalyticsExample`
37 changes: 37 additions & 0 deletions examples/azure/terraform-azure-loganalytics-example/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# ---------------------------------------------------------------------------------------------------------------------
# DEPLOY LOG ANALYTICS
# This is an example of how to deploy a Log Analytics workspace resource.
# ---------------------------------------------------------------------------------------------------------------------
# See test/azure/terraform_azure_loganalytics_example_test.go for how to write automated tests for this code.
# ---------------------------------------------------------------------------------------------------------------------

provider "azurerm" {
version = "~> 2.20"
skunklab marked this conversation as resolved.
Show resolved Hide resolved
features {}
}

# PIN TERRAFORM VERSION

terraform {
# This module is now only being tested with Terraform 0.13.x. However, to make upgrading easier, we are setting
# 0.12.26 as the minimum version, as that version added support for required_providers with source URLs, making it
# forwards compatible with 0.13.x code.
required_version = ">= 0.12.26"
}

# ---------------------------------------------------------------------------------------------------------------------
# DEPLOY A RESOURCE GROUP
# ---------------------------------------------------------------------------------------------------------------------

resource "azurerm_resource_group" "resource_group" {
name = "terratest-log-rg-${var.postfix}"
location = var.location
}

resource "azurerm_log_analytics_workspace" "log_analytics_workspace" {
name = "log-ws-${var.postfix}"
location = azurerm_resource_group.resource_group.location
resource_group_name = azurerm_resource_group.resource_group.name
sku = "PerGB2018"
retention_in_days = 30
}
16 changes: 16 additions & 0 deletions examples/azure/terraform-azure-loganalytics-example/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
output "resource_group_name" {
value = azurerm_resource_group.resource_group.name
}

output "loganalytics_workspace_name" {
value = azurerm_log_analytics_workspace.log_analytics_workspace.name
}

output "loganalytics_workspace_sku" {
value = azurerm_log_analytics_workspace.log_analytics_workspace.sku
}

output "loganalytics_workspace_retention" {
value = azurerm_log_analytics_workspace.log_analytics_workspace.retention_in_days
}

32 changes: 32 additions & 0 deletions examples/azure/terraform-azure-loganalytics-example/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# ---------------------------------------------------------------------------------------------------------------------
# ENVIRONMENT VARIABLES
# Define these secrets as environment variables
# ---------------------------------------------------------------------------------------------------------------------

# ARM_CLIENT_ID
# ARM_CLIENT_SECRET
# ARM_SUBSCRIPTION_ID
# ARM_TENANT_ID

# ---------------------------------------------------------------------------------------------------------------------
# REQUIRED PARAMETERS
# You must provide a value for each of these parameters.
# ---------------------------------------------------------------------------------------------------------------------

# ---------------------------------------------------------------------------------------------------------------------
# OPTIONAL PARAMETERS
# These parameters have reasonable defaults.
# ---------------------------------------------------------------------------------------------------------------------


variable "location" {
description = "The location to set for the storage account."
type = string
default = "East US"
}

variable "postfix" {
description = "A postfix string to centrally mitigate resource name collisions"
type = string
default = "resource"
}
96 changes: 96 additions & 0 deletions modules/azure/loganalytics.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
package azure

import (
"context"
"fmt"

"github.com/Azure/azure-sdk-for-go/services/preview/operationalinsights/mgmt/2020-03-01-preview/operationalinsights"
"github.com/gruntwork-io/terratest/modules/testing"
"github.com/stretchr/testify/require"
)

// LogAnalyticsWorkspaceExists indicates whether the operatonal insights workspaces exists.
// This function would fail the test if there is an error.
func LogAnalyticsWorkspaceExists(t testing.TestingT, workspaceName string, resourceGroupName string, subscriptionID string) bool {
exists, err := LogAnalyticsWorkspaceExistsE(workspaceName, resourceGroupName, subscriptionID)
require.NoError(t, err)
return exists
}

// GetLogAnalyticsWorkspaceSku return the log analytics workspace SKU as string as one of the following: Free, Standard, Premium, PerGB2018, CapacityReservation; otherwise empty string "".
// This function would fail the test if there is an error.
func GetLogAnalyticsWorkspaceSku(t testing.TestingT, workspaceName string, resourceGroupName string, subscriptionID string) string {
sku, err := GetLogAnalyticsWorkspaceSkuE(workspaceName, resourceGroupName, subscriptionID)
require.NoError(t, err)
return string(sku)
}

func GetLogAnalyticsWorkspaceSkuE(workspaceName string, resourceGroupName string, subscriptionID string) (string, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: this function needs docs comment.

ws, err := GetLogAnalyticsWorkspaceE(workspaceName, resourceGroupName, subscriptionID)
if err != nil {
return "", err
}
return string(ws.Sku.Name), nil
}

// GetLogAnalyticsWorkspaceRetentionPeriodDays returns the log analytics workspace retention period in days.
// This function would fail the test if there is an error.
func GetLogAnalyticsWorkspaceRetentionPeriodDays(t testing.TestingT, workspaceName string, resourceGroupName string, subscriptionID string) int32 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this function, we want to keep the helpers slim to those that really simplify retrieving data from azure. Keep the int test but modify to get off client object.

periodInDays, err := GetLogAnalyticsWorkspaceRetentionPeriodDaysE(workspaceName, resourceGroupName, subscriptionID)
require.NoError(t, err)
return periodInDays
}

func GetLogAnalyticsWorkspaceRetentionPeriodDaysE(workspaceName string, resourceGroupName string, subscriptionID string) (int32, error) {
ws, err := GetLogAnalyticsWorkspaceE(workspaceName, resourceGroupName, subscriptionID)
if err != nil {
return -1, err
}

return *ws.RetentionInDays, nil
}

// GetLogAnalyticsWorkspaceE gets an operational insights workspace if it exists in a subscription.
func GetLogAnalyticsWorkspaceE(workspaceName, resoureGroupName, subscriptionID string) (*operationalinsights.Workspace, error) {
client, err := GetLogAnalyticsWorkspacesClientE(subscriptionID)
skunklab marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return nil, err
}

ws, err := client.Get(context.Background(), resoureGroupName, workspaceName)
if err != nil {
return nil, err
}
return &ws, nil
}

// LogAnalyticsWorkspaceExistsE indicates whether the operatonal insights workspaces exists and may return an error.
func LogAnalyticsWorkspaceExistsE(workspaceName string, resourceGroupName string, subscriptionID string) (bool, error) {
_, err := GetLogAnalyticsWorkspaceE(workspaceName, resourceGroupName, subscriptionID)
if err != nil {
if ResourceNotFoundErrorExists(err) {
return false, nil
}
return false, err
}
return true, nil
}

// GetLogAnalyticsWorkspacesClientE return workspaces client; otherwise error.
func GetLogAnalyticsWorkspacesClientE(subscriptionID string) (*operationalinsights.WorkspacesClient, error) {
subscriptionID, err := getTargetAzureSubscription(subscriptionID)
if err != nil {
fmt.Println("Workspace client error getting subscription")
skunklab marked this conversation as resolved.
Show resolved Hide resolved
return nil, err
}

client := operationalinsights.NewWorkspacesClient(subscriptionID)
authorizer, err := NewAuthorizer()
if err != nil {
fmt.Println("authorizer error")
return nil, err
}

client.Authorizer = *authorizer
return &client, nil
}
33 changes: 33 additions & 0 deletions modules/azure/loganalytics_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package azure

import (
"testing"

"github.com/stretchr/testify/assert"
)

/*
The below tests are currently stubbed out, with the expectation that they will throw errors.
If/when methods to create and delete log analytics resources are added, these tests can be extended.
*/

func TestLogAnalyticsWorkspace(t *testing.T) {
t.Parallel()

_, err := LogAnalyticsWorkspaceExistsE("fake", "", "")
assert.Error(t, err, "Workspace")
}

func TestLogAnalyticsSku(t *testing.T) {
t.Parallel()

_, err := GetLogAnalyticsWorkspaceSkuE("fake", "", "")
assert.Error(t, err, "Sku")
}

func TestLogAnalyticsRetentionPeriodDays(t *testing.T) {
t.Parallel()

_, err := GetLogAnalyticsWorkspaceRetentionPeriodDaysE("fake", "", "")
assert.Error(t, err, "RetentionPeriod")
}
58 changes: 58 additions & 0 deletions test/azure/terraform_azure_loganalytics_example_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// +build azure

// NOTE: We use build tags to differentiate azure testing because we currently do not have azure access setup for
// CircleCI.

package test

import (
"strconv"
"strings"
"testing"

"github.com/gruntwork-io/terratest/modules/azure"
"github.com/gruntwork-io/terratest/modules/random"
"github.com/gruntwork-io/terratest/modules/terraform"
"github.com/stretchr/testify/assert"
)

func TestTerraformAzureLogAnalyticsExample(t *testing.T) {
t.Parallel()

// subscriptionID is overridden by the environment variable "ARM_SUBSCRIPTION_ID"
subscriptionID := ""
uniquePostfix := random.UniqueId()

// website::tag::1:: Configure Terraform setting up a path to Terraform code.
terraformOptions := &terraform.Options{
// The path to where our Terraform code is located
TerraformDir: "../../examples/azure/terraform-azure-loganalytics-example",
// Variables to pass to our Terraform code using -var options
Vars: map[string]interface{}{
"postfix": uniquePostfix,
},
}

// website::tag::4:: At the end of the test, run `terraform destroy` to clean up any resources that were created
defer terraform.Destroy(t, terraformOptions)

// website::tag::2:: Run `terraform init` and `terraform apply`. Fail the test if there are any errors.
terraform.InitAndApply(t, terraformOptions)

// website::tag::3:: Run `terraform output` to get the values of output variables
resourceGroupName := terraform.Output(t, terraformOptions, "resource_group_name")
workspaceName := terraform.Output(t, terraformOptions, "loganalytics_workspace_name")
sku := terraform.Output(t, terraformOptions, "loganalytics_workspace_sku")
retentionPeriodString := terraform.Output(t, terraformOptions, "loganalytics_workspace_retention")

// website::tag::4:: Verify the Log Analytics properties and ensure it matches the output.
workspaceExists := azure.LogAnalyticsWorkspaceExists(t, workspaceName, resourceGroupName, subscriptionID)
assert.True(t, workspaceExists, "log analytics workspace not found.")

actualSku := azure.GetLogAnalyticsWorkspaceSku(t, workspaceName, resourceGroupName, subscriptionID)
assert.Equal(t, sku, strings.ToLower(actualSku), "log analytics sku mismatch")

var actualRetentionPeriod = azure.GetLogAnalyticsWorkspaceRetentionPeriodDays(t, workspaceName, resourceGroupName, subscriptionID)
expectedPeriod, _ := strconv.ParseInt(retentionPeriodString, 10, 32)
assert.Equal(t, int32(expectedPeriod), actualRetentionPeriod, "log analytics retention period mismatch")
}