Impl use of content of the service account key file in the environmen…

…t variable `YC_SERVICE_ACCOUNT_KEY_FILE` (#82)

* Set up-to-date Ubuntu version

* Support SA Key file content JSON format in `YC_SERVICE_ACCOUNT_KEY_FILE`

Closes #39

* Update generated docs

---------

Co-authored-by: Wilken Rivera <[email protected]>

.web-docs/components/builder/yandex/README.md

@@ -105,7 +105,7 @@ can also be supplied to override the typical auto-generated key:
 
 - `source_image_family` (string) - The source image family to create the new image
   from. You can also specify source_image_id instead. Just one of a source_image_id or
-  source_image_family must be specified. Example: `ubuntu-1804-lts`.
+  source_image_family must be specified. Example: `ubuntu-2204-lts`.
 
 <!-- End of code generated from the comments of the SourceImageConfig struct in builder/yandex/common_config.go; -->
 
@@ -130,9 +130,9 @@ can also be supplied to override the typical auto-generated key:
 
 - `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.
 
-- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
-  is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
-  `YC_SERVICE_ACCOUNT_KEY_FILE`.
+- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
+  This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
+  You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).
 
 - `max_retries` (int) - The maximum number of times an API request is being executed.
 
@@ -181,6 +181,8 @@ can also be supplied to override the typical auto-generated key:
 
 - `instance_cores` (int) - The number of cores available to the instance.
 
+- `instance_core_fraction` (int) - The vCPU performance level (core fraction) of the instance
+
 - `instance_gpus` (int) - The number of GPU available to the instance.
 
 - `instance_mem_gb` (int) - The amount of memory available to the instance, specified in gigabytes.

.web-docs/components/post-processor/yandex-export/README.md

@@ -74,9 +74,9 @@ Also, you should configure [ssh communicator](/packer/docs/communicators/ssh). D
 
 - `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.
 
-- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
-  is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
-  `YC_SERVICE_ACCOUNT_KEY_FILE`.
+- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
+  This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
+  You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).
 
 - `max_retries` (int) - The maximum number of times an API request is being executed.
 
@@ -131,6 +131,8 @@ Also, you should configure [ssh communicator](/packer/docs/communicators/ssh). D
 
 - `instance_cores` (int) - The number of cores available to the instance.
 
+- `instance_core_fraction` (int) - The vCPU performance level (core fraction) of the instance
+
 - `instance_gpus` (int) - The number of GPU available to the instance.
 
 - `instance_mem_gb` (int) - The amount of memory available to the instance, specified in gigabytes.

.web-docs/components/post-processor/yandex-import/README.md

@@ -53,9 +53,9 @@ file.
 
 - `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.
 
-- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
-  is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
-  `YC_SERVICE_ACCOUNT_KEY_FILE`.
+- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
+  This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
+  You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).
 
 - `max_retries` (int) - The maximum number of times an API request is being executed.
 

GNUmakefile

@@ -17,7 +17,7 @@ dev: build
 test:
 	@go test -race -count $(COUNT) $(TEST) -timeout=3m
 
-install-packer-sdc: ## Install packer sofware development command
+install-packer-sdc: ## Install packer software development command
 	@go install github.com/hashicorp/packer-plugin-sdk/cmd/packer-sdc@${HASHICORP_PACKER_PLUGIN_SDK_VERSION}
 
 plugin-check: install-packer-sdc build

builder/yandex/access_config.go

@@ -6,6 +6,7 @@
 package yandex
 
 import (
+	"encoding/json"
 	"errors"
 	"fmt"
 	"os"
@@ -15,6 +16,14 @@ import (
 	"github.com/yandex-cloud/go-sdk/iamkey"
 )
 
+type keyType int
+
+const (
+	Undefined keyType = iota
+	File
+	Content
+)
+
 const (
 	defaultEndpoint   = "api.cloud.yandex.net:443"
 	defaultMaxRetries = 3
@@ -24,9 +33,9 @@ const (
 type AccessConfig struct {
 	// Non standard API endpoint. Default is `api.cloud.yandex.net:443`.
 	Endpoint string `mapstructure:"endpoint" required:"false"`
-	// Path to file with Service Account key in json format. This
-	// is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
-	// `YC_SERVICE_ACCOUNT_KEY_FILE`.
+	// Contains either a path to or the contents of the Service Account file in JSON format.
+	// This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
+	// You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).
 	ServiceAccountKeyFile string `mapstructure:"service_account_key_file" required:"false"`
 	// [OAuth token](https://cloud.yandex.com/docs/iam/concepts/authorization/oauth-token)
 	// or [IAM token](https://cloud.yandex.com/docs/iam/concepts/authorization/iam-token)
@@ -35,6 +44,8 @@ type AccessConfig struct {
 	Token string `mapstructure:"token" required:"true"`
 	// The maximum number of times an API request is being executed.
 	MaxRetries int `mapstructure:"max_retries"`
+
+	saKeyType keyType
 }
 
 func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
@@ -66,8 +77,19 @@ func (c *AccessConfig) Prepare(ctx *interpolate.Context) []error {
 	}
 
 	if c.ServiceAccountKeyFile != "" {
-		if _, err := iamkey.ReadFromJSONFile(c.ServiceAccountKeyFile); err != nil {
-			errs = append(errs, fmt.Errorf("fail to read service account key file: %s", err))
+		// if ServiceAccountKeyFile is file path
+		if _, err := os.Stat(c.ServiceAccountKeyFile); err == nil {
+			if _, err := iamkey.ReadFromJSONFile(c.ServiceAccountKeyFile); err != nil {
+				errs = append(errs, fmt.Errorf("fail to read service account key file: %s", err))
+			}
+			c.saKeyType = File
+		} else {
+			// else check for a valid json data value
+			var f map[string]interface{}
+			if err := json.Unmarshal([]byte(c.ServiceAccountKeyFile), &f); err != nil {
+				errs = append(errs, fmt.Errorf("JSON in %q are not valid: %s", c.ServiceAccountKeyFile, err))
+			}
+			c.saKeyType = Content
 		}
 	}
 

builder/yandex/access_config_test.go

@@ -0,0 +1,67 @@
+package yandex
+
+import (
+	"errors"
+	"os"
+	"testing"
+
+	"github.com/hashicorp/packer-plugin-sdk/template/interpolate"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestAccessConfig_Prepare(t *testing.T) {
+	bytes, err := os.ReadFile(TestServiceAccountKeyFile)
+	require.NoErrorf(t, err, "failed to read file %s", TestServiceAccountKeyFile)
+
+	var TestServiceAccountKeyFileContent = string(bytes)
+
+	type fields struct {
+		Endpoint              string
+		ServiceAccountKeyFile string
+		Token                 string
+		MaxRetries            int
+		saKeyType             keyType
+	}
+	tests := []struct {
+		name   string
+		fields fields
+		want   []error
+	}{
+		{
+			name: "sa_key_as_file", fields: fields{
+				Endpoint:              "",
+				ServiceAccountKeyFile: TestServiceAccountKeyFile,
+				Token:                 "",
+			},
+			want: nil,
+		},
+		{
+			name: "sa_key_as_json_content", fields: fields{
+				ServiceAccountKeyFile: TestServiceAccountKeyFileContent,
+				Token:                 "",
+			},
+			want: nil,
+		},
+		{
+			name: "both_identities", fields: fields{
+				ServiceAccountKeyFile: TestServiceAccountKeyFileContent,
+				Token:                 "t1.super-token",
+			},
+			want: []error{errors.New("one of token or service account key file must be specified, not both")},
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			ctx := &interpolate.Context{}
+			c := &AccessConfig{
+				Endpoint:              tt.fields.Endpoint,
+				ServiceAccountKeyFile: tt.fields.ServiceAccountKeyFile,
+				Token:                 tt.fields.Token,
+				MaxRetries:            tt.fields.MaxRetries,
+				saKeyType:             tt.fields.saKeyType,
+			}
+			assert.Equalf(t, tt.want, c.Prepare(ctx), "Prepare(%v)")
+		})
+	}
+}

builder/yandex/common_config.go

@@ -251,7 +251,7 @@ func (c *InstanceConfig) Prepare(errs *packersdk.MultiError) *packersdk.MultiErr
 type SourceImageConfig struct {
 	// The source image family to create the new image
 	// from. You can also specify source_image_id instead. Just one of a source_image_id or
-	// source_image_family must be specified. Example: `ubuntu-1804-lts`.
+	// source_image_family must be specified. Example: `ubuntu-2204-lts`.
 	SourceImageFamily string `mapstructure:"source_image_family" required:"true"`
 	// The ID of the folder containing the source image.
 	SourceImageFolderID string `mapstructure:"source_image_folder_id" required:"false"`

builder/yandex/config_test.go

@@ -4,7 +4,6 @@
 package yandex
 
 import (
-	"io/ioutil"
 	"os"
 	"strings"
 	"testing"
@@ -15,9 +14,14 @@ import (
 const TestServiceAccountKeyFile = "./testdata/fake-sa-key.json"
 
 func TestConfigPrepare(t *testing.T) {
-	tf, err := ioutil.TempFile("", "packer")
+	tf, err := os.CreateTemp("", "packer")
 	require.NoError(t, err, "create temporary file failed")
 
+	bytes, err := os.ReadFile(TestServiceAccountKeyFile)
+	require.NoErrorf(t, err, "failed to read file %s", TestServiceAccountKeyFile)
+
+	var TestServiceAccountKeyFileContent = string(bytes)
+
 	defer os.Remove(tf.Name())
 	tf.Close()
 
@@ -47,6 +51,11 @@ func TestConfigPrepare(t *testing.T) {
 			TestServiceAccountKeyFile,
 			false,
 		},
+		{
+			"service_account_key_file",
+			TestServiceAccountKeyFileContent,
+			false,
+		},
 
 		{
 			"folder_id",

builder/yandex/driver_yc.go

@@ -5,6 +5,7 @@ package yandex
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"log"
 	"strings"
@@ -60,8 +61,7 @@ func NewDriverYC(ui packersdk.Ui, ac *AccessConfig) (Driver, error) {
 			sdkConfig.Credentials = ycsdk.OAuthToken(ac.Token)
 		}
 	case ac.ServiceAccountKeyFile != "":
-		log.Printf("[INFO] Use Service Account key file %q for authentication", ac.ServiceAccountKeyFile)
-		key, err := iamkey.ReadFromJSONFile(ac.ServiceAccountKeyFile)
+		key, err := iamKeyFromSAKey(ac)
 		if err != nil {
 			return nil, err
 		}
@@ -106,6 +106,21 @@ func NewDriverYC(ui packersdk.Ui, ac *AccessConfig) (Driver, error) {
 
 }
 
+func iamKeyFromSAKey(ac *AccessConfig) (*iamkey.Key, error) {
+	switch ac.saKeyType {
+	case File:
+		log.Printf("[INFO] Use Service Account key (file %q) for authentication", ac.ServiceAccountKeyFile)
+		return iamkey.ReadFromJSONFile(ac.ServiceAccountKeyFile)
+	case Content:
+		log.Printf("[INFO] Use Service Account key (as raw JSON) for authentication")
+		return iamkey.ReadFromJSONBytes([]byte(ac.ServiceAccountKeyFile))
+	case Undefined:
+		return nil, errors.New("`undefined` SA key type - something goes wrong")
+	default:
+		return nil, errors.New("failed to determine SA key type; perhaps skipped the preparation stage")
+	}
+}
+
 func (d *driverYC) GetImage(imageID string) (*Image, error) {
 	image, err := d.sdk.Compute().Image().Get(context.Background(), &compute.GetImageRequest{
 		ImageId: imageID,

docs-partials/builder/yandex/AccessConfig-not-required.mdx

@@ -2,9 +2,9 @@
 
 - `endpoint` (string) - Non standard API endpoint. Default is `api.cloud.yandex.net:443`.
 
-- `service_account_key_file` (string) - Path to file with Service Account key in json format. This
-  is an alternative method to authenticate to Yandex.Cloud. Alternatively you may set environment variable
-  `YC_SERVICE_ACCOUNT_KEY_FILE`.
+- `service_account_key_file` (string) - Contains either a path to or the contents of the Service Account file in JSON format.
+  This can also be specified using environment variable `YC_SERVICE_ACCOUNT_KEY_FILE`.
+  You can read how to create service account key file [here](https://cloud.yandex.com/docs/iam/operations/iam-token/create-for-sa#keys-create).
 
 - `max_retries` (int) - The maximum number of times an API request is being executed.
 

docs-partials/builder/yandex/SourceImageConfig-required.mdx

@@ -2,6 +2,6 @@
 
 - `source_image_family` (string) - The source image family to create the new image
   from. You can also specify source_image_id instead. Just one of a source_image_id or
-  source_image_family must be specified. Example: `ubuntu-1804-lts`.
+  source_image_family must be specified. Example: `ubuntu-2204-lts`.
 
 <!-- End of code generated from the comments of the SourceImageConfig struct in builder/yandex/common_config.go; -->

website/content/docs/builders/yandex.mdx

@@ -52,7 +52,7 @@ To get more information about this kind of authentication check [documentaion](h
   "type": "yandex",
   "token": "YOUR OAUTH TOKEN",
   "folder_id": "YOUR FOLDER ID",
-  "source_image_family": "ubuntu-1804-lts",
+  "source_image_family": "ubuntu-2204-lts",
   "ssh_username": "ubuntu",
   "use_ipv4_nat": "true"
 }