Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clusters, vlans, tm clusters , filters #327

Merged
merged 146 commits into from
Feb 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
02a5cfb
Base design for profiles creation
bhati-pradeep Oct 27, 2022
26e79f2
Spec changes
bhati-pradeep Oct 30, 2022
a92d5d2
Patch for scaling database storage
bhati-pradeep Oct 30, 2022
48645f0
create seperate module for scaling
bhati-pradeep Oct 31, 2022
adce11b
1. Design spec for snapshots 2. Minor fixes to database scaling
bhati-pradeep Oct 31, 2022
c221ce1
database restore spec design
bhati-pradeep Oct 31, 2022
4c2ca00
Minor changes
bhati-pradeep Oct 31, 2022
2ddeaaf
Spec design changes for restore and scaling
bhati-pradeep Nov 3, 2022
de6bbe9
Profiles spec design changes
bhati-pradeep Nov 3, 2022
4c2b9cc
Update database scaling backened as per spec changes
bhati-pradeep Nov 3, 2022
7cd3de2
Module for database restore and log catcup. Add snapshot create code
bhati-pradeep Nov 4, 2022
3433897
formatting
bhati-pradeep Nov 4, 2022
f725300
Add update snapshot code
bhati-pradeep Nov 8, 2022
c0e1aff
Added logic for delete of snapshots
bhati-pradeep Nov 8, 2022
7cc28da
Logic for compute profile creation
bhati-pradeep Nov 9, 2022
5ecdb20
Network profile create for multi clusters
bhati-pradeep Nov 10, 2022
2cd2b56
Database parameter type profile creation logic
bhati-pradeep Nov 11, 2022
8f0a4fd
Add software profile update and version ops logic
bhati-pradeep Nov 14, 2022
bb2fedb
Add compute profile update logic
bhati-pradeep Nov 14, 2022
2d095b4
Add logic for db parameters update and publush flow
bhati-pradeep Nov 15, 2022
23aac4c
Add network type profile update logic
bhati-pradeep Nov 15, 2022
43f8b31
Formatting
bhati-pradeep Nov 15, 2022
c771e56
Add logic for delete ops
bhati-pradeep Nov 16, 2022
6c95373
Linked databases code
bhati-pradeep Nov 22, 2022
171ac83
Minor fixes for linked databases
bhati-pradeep Nov 22, 2022
58b7b2d
Add database instance uuid in check mode
bhati-pradeep Nov 22, 2022
8f0d360
Add backened changes for Software replication to multiple clusters
bhati-pradeep Nov 30, 2022
33ef7dd
sanity fix
alaa-bish Nov 30, 2022
b3dfe98
Tags crud module with backened
bhati-pradeep Dec 12, 2022
b99ad6f
Add Tags update and idempotency checks
bhati-pradeep Dec 13, 2022
243ecad
Basic design for db profiles params
bhati-pradeep Dec 21, 2022
1368be7
Minor code changes
bhati-pradeep Dec 24, 2022
c4c8d81
Backened design changes for single instance database
bhati-pradeep Jan 1, 2023
a7d53a5
1. Backened for database registration
bhati-pradeep Jan 4, 2023
61541ca
Enable static IP assignment and Virtual IP assignment for HA or singl…
bhati-pradeep Jan 5, 2023
aa7616c
Send failover mode and backup policy as defaults in postgres payload
bhati-pradeep Jan 5, 2023
79de741
Update and delete flow for database server vms
bhati-pradeep Jan 8, 2023
64d1f3f
Minor fix
bhati-pradeep Jan 8, 2023
17bb454
Backened changes for registration of db server VM
bhati-pradeep Jan 8, 2023
cd84fe2
Maintenance Window Module
bhati-pradeep Jan 10, 2023
8b4cd30
Add automated patching to all supported modules
bhati-pradeep Jan 10, 2023
1bff463
Maintenance tasks crud
bhati-pradeep Jan 11, 2023
0405b87
Minor fixes for database server vm registration
bhati-pradeep Jan 12, 2023
13bbc9c
Fixes for getting tag uuid
bhati-pradeep Jan 12, 2023
3086de4
Authorize db server vms flow
bhati-pradeep Jan 12, 2023
e19934f
Database clone create workflow
bhati-pradeep Jan 15, 2023
919a523
Database clone delete and update flow
bhati-pradeep Jan 15, 2023
12577db
Database clone update flow
bhati-pradeep Jan 16, 2023
e6598c2
Clone database refresh and maintenance window info module
bhati-pradeep Jan 17, 2023
423c4df
Base design for profiles creation
bhati-pradeep Oct 27, 2022
2fea672
Spec changes
bhati-pradeep Oct 30, 2022
37f331b
Profiles spec design changes
bhati-pradeep Nov 3, 2022
4058349
Logic for compute profile creation
bhati-pradeep Nov 9, 2022
6f34a25
Network profile create for multi clusters
bhati-pradeep Nov 10, 2022
83acc33
Database parameter type profile creation logic
bhati-pradeep Nov 11, 2022
e4b1771
Add software profile update and version ops logic
bhati-pradeep Nov 14, 2022
f77b3d6
Add compute profile update logic
bhati-pradeep Nov 14, 2022
cb1c218
Add logic for db parameters update and publush flow
bhati-pradeep Nov 15, 2022
cfe1de1
Add network type profile update logic
bhati-pradeep Nov 15, 2022
8f36885
Formatting
bhati-pradeep Nov 15, 2022
1a75026
Add logic for delete ops
bhati-pradeep Nov 16, 2022
f9de0ec
Add backened changes for Software replication to multiple clusters
bhati-pradeep Nov 30, 2022
959f2ad
Profiles redesign
bhati-pradeep Jan 22, 2023
ee8a910
Merge branch 'feat/296-profiles-crud-module' of github.com:nutanix/nu…
bhati-pradeep Jan 22, 2023
6e45d4e
Minor fix
bhati-pradeep Jan 22, 2023
b1c3da6
Merge branch 'feat/ndb-backened-redevelop' into impr/299-ndb-day-one-ops
bhati-pradeep Jan 22, 2023
d075a86
Merge Day2 Ops
bhati-pradeep Jan 22, 2023
ffdb983
Add method for getting snapshot uuid
bhati-pradeep Jan 22, 2023
9d3256e
snapshot replication while creation to mutliple clusters
bhati-pradeep Jan 22, 2023
81b712b
Delete .DS_Store
bhati-pradeep Jan 23, 2023
9571822
Module for snapshot replication to clusters
bhati-pradeep Jan 24, 2023
c301dbe
sanity fix
alaa-bish Jan 25, 2023
1506127
Minor fixes and test automation for db server vm crud and registration
bhati-pradeep Jan 25, 2023
e00d48b
Merge branch 'feat/ndb-backened-redevelop' of github.com:nutanix/nuta…
bhati-pradeep Jan 25, 2023
c11e806
Fomatting
bhati-pradeep Jan 25, 2023
e3417aa
fix
alaa-bish Jan 30, 2023
902f2f0
Minor fixes and test automation flows for database instance
bhati-pradeep Jan 31, 2023
d60c5a2
fix
alaa-bish Feb 1, 2023
ba569e4
Merge branch 'feat/ndb-tags' into feat/ndb-backened-redevelop
bhati-pradeep Feb 2, 2023
5f7c57a
HA and databases test automation along with minor fixes
bhati-pradeep Feb 2, 2023
1995d8f
Test automation for tags
bhati-pradeep Feb 2, 2023
a2ae686
Add module args for deleting db vms along with db delete
bhati-pradeep Feb 3, 2023
d829ca0
Test automation for database clones and related backened fixes
bhati-pradeep Feb 6, 2023
ea58cfb
Minor runtime config changes
bhati-pradeep Feb 6, 2023
9ec069f
sanity fix
alaa-bish Feb 6, 2023
d9d1f26
Merge branch 'feat/ndb-backened-redevelop' of github.com:nutanix/nuta…
bhati-pradeep Feb 6, 2023
525e30a
black fix
alaa-bish Feb 6, 2023
ceff4af
isort fix
alaa-bish Feb 6, 2023
9b448b4
Create a get spec method to point towards appropriate spec builders i…
bhati-pradeep Feb 6, 2023
3e898db
Merge branch 'feat/ndb-backened-redevelop' of github.com:nutanix/nuta…
bhati-pradeep Feb 6, 2023
a4c1935
Automation of ndb day 2 actions
bhati-pradeep Feb 8, 2023
15ae7e5
clusters, vlans, tm clusters, filters
Feb 8, 2023
ae7e3df
cluster's tests changes
Feb 10, 2023
4fdda18
Automation for multi cluster snapshots
bhati-pradeep Feb 10, 2023
9ce8a5f
Merge branch 'feat/ndb-infra-and-info-filters' of github.com:nutanix/…
bhati-pradeep Feb 10, 2023
ed883af
waiting for time machine clusters operation
Feb 10, 2023
6b865eb
clusters and vlans fixes
Feb 14, 2023
1196b6d
info's fixes
Feb 15, 2023
f5a60a4
database info's fixes
Feb 15, 2023
af03205
fixes
alaa-bish Feb 15, 2023
d6e6066
Merge branch 'feat/ndb-infra-and-info-filters' of https://github.com/…
alaa-bish Feb 15, 2023
3eeee27
fix
alaa-bish Feb 15, 2023
217a598
fix meta
alaa-bish Feb 15, 2023
1d0d6ed
add ip filter
alaa-bish Feb 15, 2023
a0bd671
time machine's fixes
Feb 15, 2023
b96fe55
Merge remote-tracking branch 'origin/feat/ndb-infra-and-info-filters'…
Feb 15, 2023
74870f7
time machine's fixes
Feb 15, 2023
3325c20
Merge remote-tracking branch 'origin/feat/ndb-infra-and-info-filters'…
Feb 15, 2023
4978421
fix
alaa-bish Feb 15, 2023
5323544
Merge branch 'feat/ndb-infra-and-info-filters' of https://github.com/…
alaa-bish Feb 15, 2023
916d957
time machine's fixes
Feb 15, 2023
c428459
fix
alaa-bish Feb 15, 2023
eca5f5a
fix
alaa-bish Feb 15, 2023
5ddb252
fixes
Feb 15, 2023
c928c6a
Merge remote-tracking branch 'origin/feat/ndb-infra-and-info-filters'…
Feb 15, 2023
24750cb
cluster's requirements
Feb 15, 2023
a3b2930
msg fixes
Feb 15, 2023
41249f8
add db server vm fliter tests
bhati-pradeep Feb 16, 2023
a566191
Merge branch 'feat/ndb-infra-and-info-filters' of github.com:nutanix/…
bhati-pradeep Feb 16, 2023
a81cfe2
fix vlan
alaa-bish Feb 16, 2023
cf6a61e
Merge branch 'feat/ndb-infra-and-info-filters' of https://github.com/…
alaa-bish Feb 16, 2023
c89bee9
fix cluster create failure
bhati-pradeep Feb 16, 2023
47c2e56
Merge branch 'feat/ndb-infra-and-info-filters' of github.com:nutanix/…
bhati-pradeep Feb 16, 2023
95f965d
fix
alaa-bish Feb 16, 2023
ef17c4c
Merge branch 'feat/ndb-infra-and-info-filters' of https://github.com/…
alaa-bish Feb 16, 2023
988ef14
Fix snapshot uuid issue
bhati-pradeep Feb 16, 2023
7fa31e7
Merge branch 'feat/ndb-infra-and-info-filters' of github.com:nutanix/…
bhati-pradeep Feb 16, 2023
4651c94
Minor comment addition
bhati-pradeep Feb 16, 2023
73adc4c
Wait for replication to finish in tests
bhati-pradeep Feb 16, 2023
527b8a3
Minor comment fix
bhati-pradeep Feb 16, 2023
77290e4
stretched_vlan_uuid fixes
Feb 16, 2023
7585dda
minor response changes in cluster
bhati-pradeep Feb 16, 2023
214b3c9
Merge branch 'feat/ndb-infra-and-info-filters' of github.com:nutanix/…
bhati-pradeep Feb 16, 2023
57fc29f
fix clusters
alaa-bish Feb 16, 2023
2779722
fix
alaa-bish Feb 16, 2023
00346ba
tm clusters fixes
Feb 16, 2023
13ce724
tm clusters fixes
Feb 16, 2023
f7485ba
tm clusters fixes
Feb 16, 2023
24fdf9d
FIx errors for updating clusters in time machine
bhati-pradeep Feb 16, 2023
8ffe77d
Time machines and snapshots test fixes
bhati-pradeep Feb 16, 2023
65d4288
minore fixes
Feb 17, 2023
19c8668
minor fixes
Feb 17, 2023
507c8a7
minor fixes
Feb 17, 2023
daf04a4
Merge branch 'release/1.8.0' into feat/ndb-infra-and-info-filters
bhati-pradeep Feb 17, 2023
f54f3b1
Minor files formatting
bhati-pradeep Feb 17, 2023
0fd6ffc
Minor fixes
bhati-pradeep Feb 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions meta/runtime.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,12 @@ action_groups:
- ntnx_ndb_register_database
- ntnx_ndb_db_server_vms
- ntnx_ndb_clusters_info
- ntnx_ndb_clusters
- ntnx_ndb_snapshots_info
- ntnx_ndb_vlans
- ntnx_ndb_vlans_info
- ntnx_ndb_stretched_vlans
- ntnx_ndb_time_machine_clusters
- ntnx_ndb_tags
- ntnx_ndb_database_clones
- ntnx_ndb_database_snapshots
Expand All @@ -83,6 +89,7 @@ action_groups:
- ntnx_ndb_database_restore
- ntnx_ndb_database_scale
- ntnx_ndb_linked_databases
- ntnx_ndb_database_replicate_snapshots
- ntnx_ndb_register_db_server_vm
- ntnx_ndb_maintenance_tasks
- ntnx_ndb_maintenance_window
Expand Down
209 changes: 206 additions & 3 deletions plugins/module_utils/ndb/clusters.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,70 @@

__metaclass__ = type

from copy import deepcopy

from .nutanix_database import NutanixDatabase


class Cluster(NutanixDatabase):

def __init__(self, module):
resource_type = "/clusters"
super(Cluster, self).__init__(module, resource_type=resource_type)
self.build_spec_methods = {
"name": self._build_spec_name,
"desc": self._build_spec_desc,
"name_prefix": self._build_spec_name_prefix,
"cluster_ip": self._build_spec_cluster_ip,
"cluster_credentials": self._build_spec_cluster_credentials,
"agent_network": self._build_spec_agent_network,
"vlan_access": self._build_spec_vlan_access,
"storage_container": self._build_spec_storage_container,
}

def update(
self,
data=None,
uuid=None,
endpoint=None,
query=None,
raise_error=True,
no_response=False,
timeout=30,
method="PATCH",
):
return super().update(
data, uuid, endpoint, query, raise_error, no_response, timeout, method
)

def delete(
self,
uuid=None,
endpoint=None,
query=None,
raise_error=True,
no_response=False,
timeout=30,
data=None,
):
if not self._validate_cluster_deleting(uuid):
err = "NDB is unable to remove the Nutanix Cluster at this time. Check dependencies"
return None, err
if not data:
data = {"deleteRemoteSites": False}
return (
super().delete(
uuid, endpoint, query, raise_error, no_response, timeout, data
),
None,
)

def get_cluster_by_ip(self, cluster_ip):
clusters = self.read()
for cluster in clusters:
if cluster_ip in cluster["ipAddresses"]:
return cluster
return None

def get_uuid(
self,
Expand All @@ -28,13 +84,13 @@ def get_uuid(

def get_cluster(self, uuid=None, name=None):
if uuid:
resp = self.read(uuid=uuid)
resp = self.read(uuid=uuid, raise_error=False)
elif name:
endpoint = "{0}/{1}".format("name", name)
resp = self.read(endpoint=endpoint)
resp = self.read(endpoint=endpoint, raise_error=False)

# we fetch cluster using ID again to get complete info.
if resp and resp.get("id"):
if resp and not resp.get("errorCode") and resp.get("id"):
resp = self.read(uuid=resp["id"])

else:
Expand All @@ -43,8 +99,155 @@ def get_cluster(self, uuid=None, name=None):
"Please provide either uuid or name for fetching cluster details",
)

if isinstance(resp, dict) and resp.get("errorCode"):
self.module.fail_json(
msg="Failed fetching cluster info",
error=resp.get("message"),
response=resp,
)

return resp, None

def _get_default_spec(self):
return deepcopy(
{
"clusterName": "",
"clusterIP": "",
"storageContainer": "",
"agentVMPrefix": "",
"port": 9440,
"protocol": "https",
"clusterType": "NTNX",
"version": "v2",
"credentialsInfo": [],
"agentNetworkInfo": [],
"networksInfo": [],
}
)

def get_default_update_spec(self, override_spec=None):
spec = deepcopy(
{
"name": "",
"description": "",
"ipAddresses": [],
}
)
if override_spec:
for key in spec.keys():
if override_spec.get(key):
spec[key] = deepcopy(override_spec[key])

return spec

def _build_spec_name(self, payload, name):
if self.module.params.get("uuid"):
payload["name"] = name
else:
payload["clusterName"] = name
return payload, None

def _build_spec_name_prefix(self, payload, prefix):
payload["agentVMPrefix"] = prefix
return payload, None

def _build_spec_desc(self, payload, desc):
if self.module.params.get("uuid"):
payload["description"] = desc
else:
payload["clusterDescription"] = desc
return payload, None

def _build_spec_cluster_ip(self, payload, cluster_ip):
if self.module.params.get("uuid"):
payload["ipAddresses"] = [cluster_ip]
else:
payload["clusterIP"] = cluster_ip
return payload, None

def _build_spec_cluster_credentials(self, payload, credentials):
if self.module.params.get("uuid"):
payload["username"] = credentials["username"]
payload["password"] = credentials["password"]
else:
payload["credentialsInfo"] = [
{"name": "username", "value": credentials["username"]},
{"name": "password", "value": credentials["password"]},
]
return payload, None

def _build_spec_agent_network(self, payload, agent_network):
payload["agentNetworkInfo"] = [
{"name": "dns", "value": ",".join(agent_network["dns_servers"])},
{"name": "ntp", "value": ",".join(agent_network["ntp_servers"])},
]
return payload, None

def _build_spec_vlan_access(self, payload, vlans_config):
networks_info = []
prism_vlan = self._generate_vlan_access_spec(vlans_config["prism_vlan"])
prism_vlan["accessType"] = ["PRISM"]

if vlans_config.get("dsip_vlan"):
dsip_vlan = self._generate_vlan_access_spec(vlans_config["dsip_vlan"])
dsip_vlan["accessType"] = ["DSIP"]
networks_info.append(dsip_vlan)
else:
prism_vlan["accessType"].append("DSIP")

if vlans_config.get("dbserver_vlan"):
dbserver_vlan = self._generate_vlan_access_spec(vlans_config["dbserver_vlan"])
dbserver_vlan["accessType"] = ["DBSERVER"]
networks_info.append(dbserver_vlan)
else:
prism_vlan["accessType"].append("DBSERVER")
networks_info.append(prism_vlan)

payload["networksInfo"] = networks_info
return payload, None

@staticmethod
def _generate_vlan_access_spec(vlan):
vlan_spec = {
"type": vlan["vlan_type"],
"networkInfo": [
{
"name": "vlanName",
"value": vlan["vlan_name"],
},
{
"name": "staticIP",
"value": vlan["static_ip"],
},
{
"name": "gateway",
"value": vlan["gateway"],
},
{
"name": "subnetMask",
"value": vlan["subnet_mask"],
},
],
}
return vlan_spec

def _build_spec_storage_container(self, payload, storage_container):
payload["storageContainer"] = storage_container
return payload, None

def _validate_cluster_deleting(self, cluster_uuid):
query = {"count_entities": True}
cluster = self.read(cluster_uuid, query=query)

if cluster.get("entityCounts"):
if cluster["entityCounts"].get("dbServers") != 0:
return False
elif cluster["entityCounts"].get("engineCounts"):
for engine in cluster["entityCounts"]["engineCounts"].values():
if engine["timeMachines"] != 0:
return False
return True

def get_all_clusters_uuid_name_map(self):
resp = self.read()
uuid_name_map = {}
Expand Down
12 changes: 7 additions & 5 deletions plugins/module_utils/ndb/database_instances.py
Original file line number Diff line number Diff line change
Expand Up @@ -196,13 +196,15 @@ def get_default_restore_spec(self):
}
)

def get_database(self, name=None, uuid=None):
default_query = {"detailed": True}
def get_database(self, name=None, uuid=None, query=None):
if uuid:
resp = self.read(uuid=uuid, query=default_query)
resp = self.read(uuid=uuid, query=query, raise_error=False)
elif name:
query = {"value-type": "name", "value": name}
query.update(deepcopy(default_query))
query_params = {"value-type": "name", "value": name}
if query:
query.update(query_params)
else:
query = query_params
resp = self.read(query=query)
if not resp:
return None, "Database with name {0} not found".format(name)
Expand Down
10 changes: 7 additions & 3 deletions plugins/module_utils/ndb/db_server_vm.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,18 @@ def get_all_db_servers_name_uuid_map(self):

return name_uuid_map

def get_db_server(self, name=None, uuid=None, ip=None):
def get_db_server(self, name=None, uuid=None, ip=None, query=None):
resp = None
if uuid:
resp = self.read(uuid=uuid)
resp = self.read(uuid=uuid, query=query)
elif name or ip:
key = "name" if name else "ip"
val = name if name else ip
query = {"value-type": key, "value": val}
query_params = {"value-type": key, "value": val}
if query:
query.update(query_params)
else:
query = query_params
resp = self.read(query=query)
if not resp:
return None, "Database server with {0} {1} not found".format(key, val)
Expand Down
3 changes: 0 additions & 3 deletions plugins/module_utils/ndb/profiles/profiles.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ def __init__(self, module):
"database_type": self.build_spec_database_type,
}

def get_type(self):
return self._type

def get_profile_uuid(self, data, type=None):
uuid = ""
if data.get("name"):
Expand Down
29 changes: 24 additions & 5 deletions plugins/module_utils/ndb/snapshots.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,10 @@ def get_snapshot(self, time_machine_uuid, name):

def get_snapshot_uuid(self, time_machine_uuid, name):
query = {
"value-type": time_machine_uuid,
"detailed": False,
"load-database": False,
"load-clones": False,
"time-zone": "UTC",
"value-type": "time-machine",
"value": time_machine_uuid,
"detailed": True,
"all": True
}

snapshots = self.read(query=query)
Expand Down Expand Up @@ -175,3 +174,23 @@ def resolve_cluster_uuids(self, clusters):
specs.append(uuid)

return specs, None

def _build_query_params(self, query_params):
if query_params.get("database-ids"):
db_ids = ",".join(query_params["database-ids"])
query_params["database-ids"] = db_ids
return query_params

def get_snapshots(self, query_params=None):
if query_params:
queries = self._build_query_params(query_params)
resp = self.read(query=queries)
else:
resp = self.read()

return resp

def get_snapshot_files(self, uuid):
endpoint = "files"
resp = self.read(uuid=uuid, endpoint=endpoint)
return resp
Loading