Skip to content

Commit

Permalink
mantle: overhaul azure authentication
Browse files Browse the repository at this point in the history 
The github.com/Azure/go-autorest library is deprecated and will
stop receiving support at the end of March [1]. This commit starts
the migration towards github.com/Azure/azure-sdk-for-go/sdk/azidentity
for authentication.

Since the azidentity library can't use file based authentication I'm
also overhauling our current two file based authentication scheme today
(azureProfile.json and azureAuth.json) to use just one file (azureCreds.json).
I've also updated the docs accordingly.

[1] https://azure.microsoft.com/en-us/updates/support-for-azure-sdk-libraries-that-do-not-conform-to-our-current-azure-sdk-guidelines-will-be-retired-as-of-31-march-2023/
  • Loading branch information
@dustymabe
dustymabe committed Feb 21, 2023
1 parent f35e2c8 commit 50f6205
Show file tree
Hide file tree
Showing 9 changed files with 94 additions and 216 deletions.
54 changes: 32 additions & 22 deletions docs/mantle/credentials.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -84,41 +84,51 @@ sudo emerge --ask awscli

## azure

`azure` uses `~/.azure/azureProfile.json`. This can be created using the `az` [command](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli):
```
$ az login`
```
It also requires that the environment variable `AZURE_AUTH_LOCATION` points to a JSON file (this can also be set via the `--azure-auth` parameter). The JSON file will require a service provider active directory account to be created.
The [azure sdk for go](https://github.com/Azure/azure-sdk-for-go) does
[not support any file based authentication schemes](https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/azidentity#defaultazurecredential).
We'll use a JSON file generated by hand to pass authentication to our
mantle code that will then use it to authenticate with azure services.

First we must have a service principal and set of credentials to authenticate
with. This can be created via the Azure CLI by the `az ad sp create-for-rbac`
command. You must know your subscription ID in order to run this command. This
can usually be picked up from `~/.azure/azureProfile.json` if you are logged
in via the Azure CLI:

Service provider accounts can be created via the `az` command (the output will contain an `appId` field which is used as the `clientId` variable in the `AZURE_AUTH_LOCATION` JSON):
```
az ad sp create-for-rbac
subscription='aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee'
az ad sp create-for-rbac --name name --role Contributor --scopes "/subscriptions/${subscription}"
```

The client secret can be created inside of the Azure portal when looking at the service provider account under the `Azure Active Directory` service on the `App registrations` tab.
The output of this command is JSON formatted. Store the output of it in a file
called `azureCreds.json`:

You can find your subscriptionId & tenantId in the `~/.azure/azureProfile.json` via:
```
cat ~/.azure/azureProfile.json | jq '{subscriptionId: .subscriptions[].id, tenantId: .subscriptions[].tenantId}'
{
"appId": "11111111-2222-3333-4444-555555555555",
"displayName": "name",
"password": "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"tenant": "66666666-7777-8888-9999-111111111111"
}
```

The JSON file exported to the variable `AZURE_AUTH_LOCATION` should be generated by hand and have the following contents:
All we need now is to add the subscription ID information to the `azureCreds.json`
so that the final file looks like:

```
{
"clientId": "<service provider id>",
"clientSecret": "<service provider secret>",
"subscriptionId": "<subscription id>",
"tenantId": "<tenant id>",
"activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
"resourceManagerEndpointUrl": "https://management.azure.com/",
"activeDirectoryGraphResourceId": "https://graph.windows.net/",
"sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
"galleryEndpointUrl": "https://gallery.azure.com/",
"managementEndpointUrl": "https://management.core.windows.net/"
"appId": "11111111-2222-3333-4444-555555555555",
"displayName": "name",
"password": "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"tenant": "66666666-7777-8888-9999-111111111111",
"subscription: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
}
```

This file can be placed at `$HOME/.azure/azureCreds.json` or the path
can be placed in the `$AZURE_CREDENTIALS` environment variable or passed
via the `--azure-credentials` option on the command line.

## do

`do` uses `~/.config/digitalocean.json`. This can be configured manually:
Expand Down
140 changes: 28 additions & 112 deletions mantle/auth/azure.go
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// Copyright 2023 Red Hat
// Copyright 2016 CoreOS, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -16,139 +17,54 @@ package auth

import (
"encoding/json"
"fmt"
"io"
"os"
"os/user"
"path/filepath"

"golang.org/x/text/encoding/unicode"
"golang.org/x/text/transform"

"github.com/coreos/coreos-assembler/mantle/platform"
)

const (
AzureAuthPath = ".azure/credentials.json"
AzureProfilePath = ".azure/azureProfile.json"
AzureCredentialsPath = ".azure/azureCreds.json"
)

// A version of the Options struct from platform/api/azure that only
// contains the ASM values. Otherwise there's a cyclical depdendence
// because platform/api/azure has to import auth to have access to
// the ReadAzureProfile function.
type Options struct {
*platform.Options

SubscriptionName string
SubscriptionID string

// Azure Storage API endpoint suffix. If unset, the Azure SDK default will be used.
StorageEndpointSuffix string
}

type azureEnvironment struct {
Name string `json:"name"`
StorageEndpointSuffix string `json:"storageEndpointSuffix"`
type AzureCredentials struct {
ClientID string `json:"appId"`
ClientSecret string `json:"password"`
SubscriptionID string `json:"subscription"`
TenantID string `json:"tenant"`
}

type azureSubscription struct {
EnvironmentName string `json:"environmentName"`
ID string `json:"id"`
Name string `json:"name"`
}

// AzureProfile represents a parsed Azure Profile Configuration File.
type AzureProfile struct {
Environments []azureEnvironment `json:"environments"`
Subscriptions []azureSubscription `json:"subscriptions"`
}

// AsOptions converts all subscriptions into a slice of Options.
// If there is an environment with a name matching the subscription, that environment's storage endpoint will be copied to the options.
func (ap *AzureProfile) asOptions() []Options {
var o []Options

for _, sub := range ap.Subscriptions {
newo := Options{
SubscriptionName: sub.Name,
SubscriptionID: sub.ID,
}

// find the storage endpoint for the subscription
for _, e := range ap.Environments {
if e.Name == sub.EnvironmentName {
newo.StorageEndpointSuffix = e.StorageEndpointSuffix
break
}
}

o = append(o, newo)
}

return o
}

// SubscriptionOptions returns the name subscription in the Azure profile as a Options struct.
// If the subscription name is "", the first subscription is returned.
// If there are no subscriptions or the named subscription is not found, SubscriptionOptions returns nil.
func (ap *AzureProfile) SubscriptionOptions(name string) *Options {
opts := ap.asOptions()

if len(opts) == 0 {
return nil
}

if name == "" {
return &opts[0]
} else {
for _, o := range opts {
if o.SubscriptionName == name {
return &o
}
}
}

return nil
}

// ReadAzureProfile decodes an Azure Profile, as created by the Azure Cross-platform CLI.
// ReadAzureCredentials picks up the credentials as described in the docs.
//
// If path is empty, $HOME/.azure/azureProfile.json is read.
func ReadAzureProfile(path string) (*AzureProfile, error) {
// If path is empty, $AZURE_CREDENTIALS or $HOME/.azure/azureCreds.json is read.
func ReadAzureCredentials(path string) (AzureCredentials, error) {
var azCreds AzureCredentials
if path == "" {
user, err := user.Current()
if err != nil {
return nil, err
path = os.Getenv("AZURE_CREDENTIALS")
if path == "" {
user, err := user.Current()
if err != nil {
return azCreds, err
}
path = filepath.Join(user.HomeDir, AzureCredentialsPath)
}

path = filepath.Join(user.HomeDir, AzureProfilePath)
}

contents, err := decodeBOMFile(path)
f, err := os.Open(path)
if err != nil {
return nil, err
}

var ap AzureProfile
if err := json.Unmarshal(contents, &ap); err != nil {
return nil, err
return azCreds, err
}
defer f.Close()

if len(ap.Subscriptions) == 0 {
return nil, fmt.Errorf("Azure profile %q contains no subscriptions", path)
content, err := io.ReadAll(f)
if err != nil {
return azCreds, err
}

return &ap, nil
}

func decodeBOMFile(path string) ([]byte, error) {
f, err := os.Open(path)
err = json.Unmarshal(content, &azCreds)
if err != nil {
return nil, err
return azCreds, err
}
defer f.Close()
decoder := unicode.UTF8.NewDecoder()
reader := transform.NewReader(f, unicode.BOMOverride(decoder))
return io.ReadAll(reader)

return azCreds, nil
}
4 changes: 2 additions & 2 deletions mantle/cmd/kola/options.go
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// Copyright 2023 Red Hat
// Copyright 2015 CoreOS, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -89,8 +90,7 @@ func init() {
sv(&kola.AWSOptions.IAMInstanceProfile, "aws-iam-profile", "kola", "AWS IAM instance profile name")

// azure-specific options
sv(&kola.AzureOptions.AzureProfile, "azure-profile", "", "Azure profile (default \"~/"+auth.AzureProfilePath+"\")")
sv(&kola.AzureOptions.AzureAuthLocation, "azure-auth", "", "Azure auth location (default \"~/"+auth.AzureAuthPath+"\")")
sv(&kola.AzureOptions.AzureCredentials, "azure-credentials", "", "Azure credentials file location (default \"~/"+auth.AzureCredentialsPath+"\")")
sv(&kola.AzureOptions.DiskURI, "azure-disk-uri", "", "Azure disk uri (custom images)")
sv(&kola.AzureOptions.Publisher, "azure-publisher", "CoreOS", "Azure image publisher (default \"CoreOS\"")
sv(&kola.AzureOptions.Offer, "azure-offer", "CoreOS", "Azure image offer (default \"CoreOS\"")
Expand Down
17 changes: 6 additions & 11 deletions mantle/cmd/ore/azure/azure.go
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// Copyright 2023 Red Hat
// Copyright 2016 CoreOS, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -31,10 +32,8 @@ var (
Short: "azure image and vm utilities",
}

azureProfile string
azureAuth string
azureSubscription string
azureLocation string
azureCredentials string
azureLocation string

api *azure.API
)
Expand All @@ -43,20 +42,16 @@ func init() {
cli.WrapPreRun(Azure, preauth)

sv := Azure.PersistentFlags().StringVar
sv(&azureProfile, "azure-profile", "", "Azure Profile json file")
sv(&azureAuth, "azure-auth", "", "Azure auth location (default \"~/"+auth.AzureAuthPath+"\")")
sv(&azureSubscription, "azure-subscription", "", "Azure subscription name. If unset, the first is used.")
sv(&azureCredentials, "azure-credentials", "", "Azure credentials file location (default \"~/"+auth.AzureCredentialsPath+"\")")
sv(&azureLocation, "azure-location", "westus", "Azure location (default \"westus\")")
}

func preauth(cmd *cobra.Command, args []string) error {
plog.Printf("Creating Azure API...")

a, err := azure.New(&azure.Options{
AzureProfile: azureProfile,
AzureAuthLocation: azureAuth,
AzureSubscription: azureSubscription,
Location: azureLocation,
AzureCredentials: azureCredentials,
Location: azureLocation,
})
if err != nil {
plog.Fatalf("Failed to create Azure API: %v", err)
Expand Down
56 changes: 12 additions & 44 deletions mantle/platform/api/azure/api.go
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// Copyright 2023 Red Hat
// Copyright 2016 CoreOS, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
Expand Down Expand Up @@ -27,18 +28,19 @@ import (
"github.com/Azure/azure-sdk-for-go/arm/network"
"github.com/Azure/azure-sdk-for-go/arm/resources/resources"
armStorage "github.com/Azure/azure-sdk-for-go/arm/storage"
"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
"github.com/Azure/azure-sdk-for-go/storage"
"github.com/Azure/go-autorest/autorest/azure/auth"
"github.com/coreos/pkg/capnslog"

internalAuth "github.com/coreos/coreos-assembler/mantle/auth"
"github.com/coreos/coreos-assembler/mantle/auth"
)

var (
plog = capnslog.NewPackageLogger("github.com/coreos/coreos-assembler/mantle", "platform/api/azure")
)

type API struct {
azIdCred *azidentity.DefaultAzureCredential
rgClient resources.GroupsClient
imgClient compute.ImagesClient
compClient compute.VirtualMachinesClient
Expand All @@ -53,43 +55,15 @@ type API struct {
// New creates a new Azure client. If no publish settings file is provided or
// can't be parsed, an anonymous client is created.
func New(opts *Options) (*API, error) {
if opts.StorageEndpointSuffix == "" {
opts.StorageEndpointSuffix = storage.DefaultBaseURL
}

profiles, err := internalAuth.ReadAzureProfile(opts.AzureProfile)
azCreds, err := auth.ReadAzureCredentials(opts.AzureCredentials)
if err != nil {
return nil, fmt.Errorf("couldn't read Azure profile: %v", err)
return nil, fmt.Errorf("couldn't read Azure Credentials file: %v", err)
}

subOpts := profiles.SubscriptionOptions(opts.AzureSubscription)
if subOpts == nil {
return nil, fmt.Errorf("Azure subscription named %q doesn't exist in %q", opts.AzureSubscription, opts.AzureProfile)
}

if os.Getenv("AZURE_AUTH_LOCATION") == "" {
if opts.AzureAuthLocation == "" {
user, err := user.Current()
if err != nil {
return nil, err
}
opts.AzureAuthLocation = filepath.Join(user.HomeDir, internalAuth.AzureAuthPath)
}
// TODO: Move to Flight once built to allow proper unsetting
os.Setenv("AZURE_AUTH_LOCATION", opts.AzureAuthLocation)
}

if opts.SubscriptionID == "" {
opts.SubscriptionID = subOpts.SubscriptionID
}

if opts.SubscriptionName == "" {
opts.SubscriptionName = subOpts.SubscriptionName
}

if opts.StorageEndpointSuffix == "" {
opts.StorageEndpointSuffix = subOpts.StorageEndpointSuffix
}
opts.SubscriptionID = azCreds.SubscriptionID
os.Setenv("AZURE_CLIENT_ID", azCreds.ClientID)
os.Setenv("AZURE_TENANT_ID", azCreds.TenantID)
os.Setenv("AZURE_CLIENT_SECRET", azCreds.ClientSecret)

api := &API{
opts: opts,
Expand All @@ -103,14 +77,8 @@ func New(opts *Options) (*API, error) {
}

func (a *API) SetupClients() error {
auther, err := auth.GetClientSetup(resources.DefaultBaseURI)
if err != nil {
return err
}
a.rgClient = resources.NewGroupsClientWithBaseURI(auther.BaseURI, auther.SubscriptionID)
a.rgClient.Authorizer = auther

auther, err = auth.GetClientSetup(compute.DefaultBaseURI)
var err error
a.azIdCred, err = azidentity.NewDefaultAzureCredential(nil)
if err != nil {
return err
}
Expand Down
Loading

0 comments on commit 50f6205

Please sign in to comment.