Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move connectedk8s CLI to track2 python SDK models #3711

Merged
merged 100 commits into from
Aug 2, 2021
Merged
Show file tree
Hide file tree
Changes from 99 commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
68800b6
Added spn and resource group creation in connectedk8s create
Nov 12, 2019
5090698
Vanilla CLI Code for Connected Cluster
Nov 22, 2019
fda117d
saved location data CLI changes after 0.1.2
Mar 11, 2020
e781176
Added the 0.1.2 version CLI code in master
Mar 11, 2020
969a45f
Removed a print statement
Mar 11, 2020
d60bd23
Fixed container status check
Mar 12, 2020
45e2db7
Updated pod status check
Mar 13, 2020
678c2b7
Merge pull request #1 from akashkeshari/akash/agent_monitor_fix
akashkeshari Mar 16, 2020
85aa7d9
Added 2020 version models and operations
Mar 24, 2020
d345685
Updated with key pair generation
Mar 29, 2020
4e90290
review of public private key pair
Apr 1, 2020
836e6fc
updated models with latest changes in swagger
Apr 4, 2020
a1bea49
Update CLI to support SPN less onboarding
Apr 7, 2020
6522421
Removed location data changes
Apr 13, 2020
dec8fab
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
9bd5580
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
854b36f
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
576704d
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
b26a0eb
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
6d445c3
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
9df8269
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
031639b
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
47ca292
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
b237d9c
Update src/connectedk8s/azext_connectedk8s/custom.py
akashkeshari Apr 21, 2020
6249ea7
Updated error messages
Apr 21, 2020
5223949
resolved a conflict
Apr 21, 2020
c1ecbf8
updated wrt style checks
Apr 21, 2020
f43ebfb
Added a recorded test
Apr 21, 2020
d237667
Updated forked master with original master
Apr 21, 2020
32c5089
Updated with latest master changes
Apr 21, 2020
a97f36e
Updated imports
Apr 21, 2020
025522c
removed python 2.x from setup.py
Apr 21, 2020
1967b02
Updated format.py
Apr 21, 2020
df87f51
Updated readme file
Apr 22, 2020
b747e92
Updated test file
Apr 22, 2020
9d9c954
updated setup.py
Apr 22, 2020
2db8c30
updated knack version in setup.py
Apr 22, 2020
bd26f72
updated knack version to 0.7.0rc4 in setup.py
Apr 22, 2020
9dbfa0b
Updated test file to pass kubeconfig
Apr 23, 2020
c45a3a6
Updated test file to pass kubeconfig
Apr 23, 2020
f27610a
Added async support for delete
Apr 24, 2020
9852a51
Added helm charts to test data folder
Apr 27, 2020
ddad5b7
Updated helm chart pull method
Apr 27, 2020
6e6cd75
corrected import order
Apr 27, 2020
eb723fb
Ran connectedk8s test
Apr 27, 2020
a5407c1
Updated some print statements
Apr 27, 2020
3b8f930
ran test with helm upgrade --install
Apr 27, 2020
9f72a47
Updated print to warning
Apr 27, 2020
5515568
Updated error message
Apr 27, 2020
fac6740
Updated exceptions to be generic
Apr 28, 2020
e0f5022
Added aks creation in tests
Apr 29, 2020
c9d1071
updated warning statements
Apr 29, 2020
3b78ee8
updated test to create aks cluster with ssh keys
Apr 29, 2020
96a2cbe
added spn calls in recorded tests
Apr 29, 2020
2c7b8b3
Updated api-version in recorded tests
Apr 29, 2020
b9fdd1e
removed mc delete as CI test needs it to be persistant
Apr 29, 2020
cfb21ab
switched to livescenariotest
Apr 29, 2020
005e414
updated params.py
Apr 30, 2020
ba268a1
corrected databricks change
Apr 30, 2020
0ff1943
updated pod monitoring
Apr 30, 2020
d77e433
updated identation
Apr 30, 2020
44f4547
updated print statement to warnings
Apr 30, 2020
3fa9b10
added telemetry code for connect cli
May 6, 2020
5c56299
Added exception telemetry for delete operation
May 12, 2020
4ddea41
Updated fork master with latest master
May 12, 2020
47fc01b
Updated with latest master
May 12, 2020
9239f83
Updated setup.py
May 12, 2020
ccdbff0
Changed default kubernetes distro
May 12, 2020
e6333d7
Distro detect.
pasankav May 12, 2020
1ef6cdd
Updated history
May 13, 2020
c1612c2
removed arm resourceid from telemetry
May 13, 2020
3ed99da
Merge branch 'akash/connect_telemetry' into pavan/distro_detect
pasankav May 14, 2020
8b26810
Add kubernetes distro and identifying it.
pasankav May 14, 2020
fdacc2b
Remove additional line.
pasankav May 15, 2020
3860c00
correct indentation.
pasankav May 15, 2020
b9b4915
Merge branch 'master' into pavan/distro_detect
pasankav May 15, 2020
f8c4e04
correct indentation.
pasankav May 15, 2020
b752e92
correct version number.
pasankav May 15, 2020
9404057
Merge pull request #5 from akashkeshari/pavan/distro_detect
akashkeshari May 18, 2020
669071f
Merge pull request #4 from akashkeshari/akash/connect_telemetry
akashkeshari May 18, 2020
65e69b8
Merge pull request #2 from akashkeshari/location_data
akashkeshari May 18, 2020
13b8795
Merge remote-tracking branch 'upstream/master'
Aug 13, 2020
da4a59b
merged with latest master
Aug 13, 2020
fcbda96
resolved merge conflict with upstream
Sep 6, 2020
6636a4f
Merge remote-tracking branch 'upstream/master' into master
Oct 16, 2020
a1ba684
Merge remote-tracking branch 'upstream/master' into master
Oct 28, 2020
6ea4e8a
Merge remote-tracking branch 'upstream/master' into master
Dec 2, 2020
3dec24b
Merge remote-tracking branch 'upstream/master' into master
Dec 21, 2020
025e0be
Merge remote-tracking branch 'upstream/master' into master
Jan 22, 2021
4375ef9
Merge remote-tracking branch 'upstream/master' into master
Feb 8, 2021
53aeb0e
Merge remote-tracking branch 'upstream/master' into master
Feb 19, 2021
c6490ec
Merge remote-tracking branch 'upstream/master' into master
May 27, 2021
b7ce14c
Merge remote-tracking branch 'upstream/main' into master
Jul 27, 2021
e6d8a4f
updated to track2 sdk
Jul 28, 2021
0f35f97
fixed linter issues
Jul 28, 2021
4195205
Merge remote-tracking branch 'upstream/main' into master
Jul 29, 2021
3061648
Merge branch 'master' into akash/ms.k8s_track2
Jul 29, 2021
566b86f
updated with latest master
Jul 29, 2021
35b9d18
fixed default timeout
Jul 29, 2021
07cbf05
re ran live tests
Aug 1, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/connectedk8s/HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@
Release History
===============

1.1.6
++++++
* Moved to track2 SDK
* `az connectedk8s connect`: Added onboarding timeout parameter
akashkeshari marked this conversation as resolved.
Show resolved Hide resolved
* `az connectedk8s upgrade`: Added upgrade timeout parameter
* Release namespace detection bug fix in multiple commands


1.1.5
++++++
* Add custom-locations oid parameter for spn scenario
Expand Down
4 changes: 4 additions & 0 deletions src/connectedk8s/azext_connectedk8s/_help.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
text: az connectedk8s connect -g resourceGroupName -n connectedClusterName --proxy-https https://proxy-url --proxy-http http://proxy-url --proxy-skip-range excludedIP,excludedCIDR,exampleCIDRfollowed,10.0.0.0/24
- name: Onboard a connected kubernetes cluster by specifying the https proxy, http proxy, no proxy with cert settings.
text: az connectedk8s connect -g resourceGroupName -n connectedClusterName --proxy-cert /path/to/crt --proxy-https https://proxy-url --proxy-http http://proxy-url --proxy-skip-range excludedIP,excludedCIDR,exampleCIDRfollowed,10.0.0.0/24
- name: Onboard a connected kubernetes cluster with custom onboarding timeout.
text: az connectedk8s connect -g resourceGroupName -n connectedClusterName --onboarding-timeout 600

"""

Expand All @@ -49,6 +51,8 @@
text: az connectedk8s upgrade -g resourceGroupName -n connectedClusterName
- name: Upgrade the agents to a specific version
text: az connectedk8s upgrade -g resourceGroupName -n connectedClusterName --agent-version 0.2.62
- name: Upgrade the agents with custom upgrade timeout.
text: az connectedk8s upgrade -g resourceGroupName -n connectedClusterName --upgrade-timeout 600
"""

helps['connectedk8s list'] = """
Expand Down
3 changes: 3 additions & 0 deletions src/connectedk8s/azext_connectedk8s/_params.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ def load_arguments(self, _):
c.argument('infrastructure', options_list=['--infrastructure'], help='The infrastructure on which the Kubernetes cluster represented by this connected cluster will be running on.', arg_type=get_enum_type(Infrastructure_Enum_Values))
c.argument('disable_auto_upgrade', options_list=['--disable-auto-upgrade'], action='store_true', help='Flag to disable auto upgrade of arc agents.')
c.argument('cl_oid', options_list=['--custom-locations-oid'], help="OID of 'custom-locations' app")
c.argument('onboarding_timeout', options_list=['--onboarding-timeout'], arg_group='Timeout', help='Time required (in seconds) for the arc-agent pods to be installed on the kubernetes cluster. Override this value if the hardware/network constraints on your cluster requires more time for installing the arc-agent pods.')
c.argument('no_wait', options_list=['--no-wait'], arg_group='Timeout', help="Do not wait for the long-running operation to finish.")

with self.argument_context('connectedk8s update') as c:
c.argument('cluster_name', options_list=['--name', '-n'], id_part='name', help='The name of the connected cluster.')
Expand All @@ -51,6 +53,7 @@ def load_arguments(self, _):
c.argument('kube_config', options_list=['--kube-config'], help='Path to the kube config file.')
c.argument('kube_context', options_list=['--kube-context'], help='Kubconfig context from current machine.')
c.argument('arc_agent_version', options_list=['--agent-version'], help='Version of agent to update the helm charts to.')
c.argument('upgrade_timeout', options_list=['--upgrade-timeout'], help='Time required (in seconds) for the arc-agent pods to be upgraded on the kubernetes cluster. Override this value if the hardware/network constraints on your cluster requires more time for upgrading the arc-agent pods.')

with self.argument_context('connectedk8s enable-features') as c:
c.argument('cluster_name', options_list=['--name', '-n'], id_part='name', help='The name of the connected cluster.')
Expand Down
10 changes: 8 additions & 2 deletions src/connectedk8s/azext_connectedk8s/_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from azure.cli.core.commands.client_factory import get_subscription_id
from azure.cli.core.util import send_raw_request
from azure.cli.core import telemetry
from azure.core.exceptions import ResourceNotFoundError
from msrest.exceptions import AuthenticationError, HttpOperationError, TokenExpiredError, ValidationError
from msrestazure.azure_exceptions import CloudError
from kubernetes.client.rest import ApiException
Expand Down Expand Up @@ -197,6 +198,9 @@ def arm_exception_handler(ex, fault_type, summary, return_if_not_found=False):
raise AzureInternalError("Cloud error occured while making ARM request: " + str(ex) + "\nSummary: {}".format(summary))
raise AzureResponseError("Cloud error occured while making ARM request: " + str(ex) + "\nSummary: {}".format(summary))

if isinstance(ex, ResourceNotFoundError) and return_if_not_found:
return

telemetry.set_exception(exception=ex, fault_type=fault_type, summary=summary)
raise ClientRequestError("Error occured while making ARM request: " + str(ex) + "\nSummary: {}".format(summary))

Expand Down Expand Up @@ -286,7 +290,7 @@ def delete_arc_agents(release_namespace, kube_config, kube_context, configuratio

def helm_install_release(chart_path, subscription_id, kubernetes_distro, kubernetes_infra, resource_group_name, cluster_name,
location, onboarding_tenant_id, http_proxy, https_proxy, no_proxy, proxy_cert, private_key_pem,
kube_config, kube_context, no_wait, values_file_provided, values_file, cloud_name, disable_auto_upgrade, enable_custom_locations, custom_locations_oid):
kube_config, kube_context, no_wait, values_file_provided, values_file, cloud_name, disable_auto_upgrade, enable_custom_locations, custom_locations_oid, onboarding_timeout="300"):
cmd_helm_install = ["helm", "upgrade", "--install", "azure-arc", chart_path,
"--set", "global.subscriptionId={}".format(subscription_id),
"--set", "global.kubernetesDistro={}".format(kubernetes_distro),
Expand Down Expand Up @@ -324,7 +328,9 @@ def helm_install_release(chart_path, subscription_id, kubernetes_distro, kuberne
if kube_context:
cmd_helm_install.extend(["--kube-context", kube_context])
if not no_wait:
cmd_helm_install.extend(["--wait"])
# Change --timeout format for helm client to understand
onboarding_timeout = onboarding_timeout + "s"
cmd_helm_install.extend(["--wait", "--timeout", "{}".format(onboarding_timeout)])
response_helm_install = Popen(cmd_helm_install, stdout=PIPE, stderr=PIPE)
_, error_helm_install = response_helm_install.communicate()
if response_helm_install.returncode != 0:
Expand Down
23 changes: 15 additions & 8 deletions src/connectedk8s/azext_connectedk8s/custom.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
import azext_connectedk8s._utils as utils
from glob import glob
from .vendored_sdks.models import ConnectedCluster, ConnectedClusterIdentity
from .vendored_sdks.preview_2021_04_01.models import ListClusterUserCredentialsProperties
from threading import Timer, Thread
import sys
import hashlib
Expand All @@ -57,7 +58,7 @@

def create_connectedk8s(cmd, client, resource_group_name, cluster_name, https_proxy="", http_proxy="", no_proxy="", proxy_cert="", location=None,
kube_config=None, kube_context=None, no_wait=False, tags=None, distribution='auto', infrastructure='auto',
disable_auto_upgrade=False, cl_oid=None):
disable_auto_upgrade=False, cl_oid=None, onboarding_timeout="300"):
logger.warning("Ensure that you have the latest helm version installed before proceeding.")
logger.warning("This operation might take a while...\n")

Expand Down Expand Up @@ -258,7 +259,7 @@ def create_connectedk8s(cmd, client, resource_group_name, cluster_name, https_pr
# Install azure-arc agents
utils.helm_install_release(chart_path, subscription_id, kubernetes_distro, kubernetes_infra, resource_group_name, cluster_name,
location, onboarding_tenant_id, http_proxy, https_proxy, no_proxy, proxy_cert, private_key_pem, kube_config,
kube_context, no_wait, values_file_provided, values_file, azure_cloud, disable_auto_upgrade, enable_custom_locations, custom_locations_oid)
kube_context, no_wait, values_file_provided, values_file, azure_cloud, disable_auto_upgrade, enable_custom_locations, custom_locations_oid, onboarding_timeout)

return put_cc_response

Expand Down Expand Up @@ -695,15 +696,15 @@ def get_release_namespace(kube_config, kube_context):

def create_cc_resource(client, resource_group_name, cluster_name, cc, no_wait):
try:
return sdk_no_wait(no_wait, client.create, resource_group_name=resource_group_name,
return sdk_no_wait(no_wait, client.begin_create, resource_group_name=resource_group_name,
cluster_name=cluster_name, connected_cluster=cc)
except CloudError as e:
utils.arm_exception_handler(e, consts.Create_ConnectedCluster_Fault_Type, 'Unable to create connected cluster resource')


def delete_cc_resource(client, resource_group_name, cluster_name, no_wait):
try:
sdk_no_wait(no_wait, client.delete,
sdk_no_wait(no_wait, client.begin_delete,
resource_group_name=resource_group_name,
cluster_name=cluster_name)
except CloudError as e:
Expand Down Expand Up @@ -870,7 +871,7 @@ def update_agents(cmd, client, resource_group_name, cluster_name, https_proxy=""
return str.format(consts.Update_Agent_Success, connected_cluster.name)


def upgrade_agents(cmd, client, resource_group_name, cluster_name, kube_config=None, kube_context=None, arc_agent_version=None):
def upgrade_agents(cmd, client, resource_group_name, cluster_name, kube_config=None, kube_context=None, arc_agent_version=None, upgrade_timeout="300"):
logger.warning("Ensure that you have the latest helm version installed before proceeding.")
logger.warning("This operation might take a while...\n")

Expand Down Expand Up @@ -946,7 +947,7 @@ def upgrade_agents(cmd, client, resource_group_name, cluster_name, kube_config=N
telemetry.set_exception(exception='connectedk8s upgrade called when auto-update is set to true', fault_type=consts.Manual_Upgrade_Called_In_Auto_Update_Enabled,
summary='az connectedk8s upgrade to manually upgrade agents and extensions is only supported when auto-upgrade is set to false.')
raise ClientRequestError("az connectedk8s upgrade to manually upgrade agents and extensions is only supported when auto-upgrade is set to false.",
recommendation="Please run az connectedk8s update -n <connected-cluster-name> -g <resource-group-name> --auto-upgrade 'false' before performing manual upgrade")
recommendation="Please run 'az connectedk8s update -n <connected-cluster-name> -g <resource-group-name> --auto-upgrade false' before performing manual upgrade")

else:
telemetry.set_exception(exception="The azure-arc release namespace couldn't be retrieved", fault_type=consts.Release_Namespace_Not_Found,
Expand Down Expand Up @@ -1020,8 +1021,10 @@ def upgrade_agents(cmd, client, resource_group_name, cluster_name, kube_config=N
summary='Problem loading the helm existing user supplied values')
raise CLIInternalError("Problem loading the helm existing user supplied values: " + str(e))

# Change --timeout format for helm client to understand
upgrade_timeout = upgrade_timeout + "s"
cmd_helm_upgrade = ["helm", "upgrade", "azure-arc", chart_path, "--namespace", release_namespace,
"--output", "json", "--atomic"]
"--output", "json", "--atomic", "--wait", "--timeout", "{}".format(upgrade_timeout)]

proxy_enabled_param_added = False
infra_added = False
Expand Down Expand Up @@ -1818,7 +1821,11 @@ def client_side_proxy(cmd,

# Fetching hybrid connection details from Userrp
try:
response = client.list_cluster_user_credentials(resource_group_name, cluster_name, auth_method, True)
list_prop = ListClusterUserCredentialsProperties(
authentication_method=auth_method,
client_proxy=True
)
response = client.list_cluster_user_credentials(resource_group_name, cluster_name, list_prop)
except Exception as e:
if flag == 1:
clientproxy_process.terminate()
Expand Down
19 changes: 10 additions & 9 deletions src/connectedk8s/azext_connectedk8s/vendored_sdks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
#
# Licensed under the MIT License. See License.txt in the project root for license information.
# Code generated by Microsoft (R) AutoRest Code Generator.
# Changes may cause incorrect behavior and will be lost if the code is
# regenerated.
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

from .connected_kubernetes_client import ConnectedKubernetesClient
from .version import VERSION

__all__ = ['ConnectedKubernetesClient']
from ._connected_kubernetes_client import ConnectedKubernetesClient
from ._version import VERSION

__version__ = VERSION
__all__ = ['ConnectedKubernetesClient']

try:
from ._patch import patch_sdk # type: ignore
patch_sdk()
except ImportError:
pass
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# Code generated by Microsoft (R) AutoRest Code Generator.
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

from typing import TYPE_CHECKING

from azure.core.configuration import Configuration
from azure.core.pipeline import policies
from azure.mgmt.core.policies import ARMHttpLoggingPolicy

from ._version import VERSION

if TYPE_CHECKING:
# pylint: disable=unused-import,ungrouped-imports
from typing import Any

from azure.core.credentials import TokenCredential


class ConnectedKubernetesClientConfiguration(Configuration):
"""Configuration for ConnectedKubernetesClient.

Note that all parameters used to create this instance are saved as instance
attributes.

:param credential: Credential needed for the client to connect to Azure.
:type credential: ~azure.core.credentials.TokenCredential
:param subscription_id: The ID of the target subscription.
:type subscription_id: str
"""

def __init__(
self,
credential, # type: "TokenCredential"
subscription_id, # type: str
**kwargs # type: Any
):
# type: (...) -> None
if credential is None:
raise ValueError("Parameter 'credential' must not be None.")
if subscription_id is None:
raise ValueError("Parameter 'subscription_id' must not be None.")
super(ConnectedKubernetesClientConfiguration, self).__init__(**kwargs)

self.credential = credential
self.subscription_id = subscription_id
self.api_version = "2021-03-01"
self.credential_scopes = kwargs.pop('credential_scopes', ['https://management.azure.com/.default'])
kwargs.setdefault('sdk_moniker', 'mgmt-hybridkubernetes/{}'.format(VERSION))
self._configure(**kwargs)

def _configure(
self,
**kwargs # type: Any
):
# type: (...) -> None
self.user_agent_policy = kwargs.get('user_agent_policy') or policies.UserAgentPolicy(**kwargs)
self.headers_policy = kwargs.get('headers_policy') or policies.HeadersPolicy(**kwargs)
self.proxy_policy = kwargs.get('proxy_policy') or policies.ProxyPolicy(**kwargs)
self.logging_policy = kwargs.get('logging_policy') or policies.NetworkTraceLoggingPolicy(**kwargs)
self.http_logging_policy = kwargs.get('http_logging_policy') or ARMHttpLoggingPolicy(**kwargs)
self.retry_policy = kwargs.get('retry_policy') or policies.RetryPolicy(**kwargs)
self.custom_hook_policy = kwargs.get('custom_hook_policy') or policies.CustomHookPolicy(**kwargs)
self.redirect_policy = kwargs.get('redirect_policy') or policies.RedirectPolicy(**kwargs)
self.authentication_policy = kwargs.get('authentication_policy')
if self.credential and not self.authentication_policy:
self.authentication_policy = policies.BearerTokenCredentialPolicy(self.credential, *self.credential_scopes, **kwargs)
Loading