Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Helm template validation to the build method in the CLI #136

Merged
merged 192 commits into from
Jan 23, 2024
Merged
Show file tree
Hide file tree
Changes from 191 commits
Commits
Show all changes
192 commits
Select commit Hold shift + click to select a range
260c1d7
added empty template parser
Nov 27, 2023
8f2f08d
Initial scaffolding of some new classes
PerfectChaos Nov 28, 2023
1c3e3cf
added utils, added base builder + reader + their children, added nfd …
Nov 28, 2023
17da2a3
moved utils to common folder + added local file builder
Nov 28, 2023
ee6f565
Add some basic implementations
PerfectChaos Nov 28, 2023
bfdcb80
Merge branch 'Azure:main' into cli-redesign
jddarby Nov 29, 2023
d0b2003
add new commands and restructure command groups
jddarby Nov 22, 2023
38729e5
better naming for command groups
jddarby Nov 27, 2023
e347e09
revert addition on the onboard command group
jddarby Nov 27, 2023
adf9f71
update HISTORY.rst
jddarby Nov 27, 2023
6c7b4de
fix style issue
jddarby Nov 27, 2023
e389165
fix linting issues
jddarby Nov 27, 2023
1c54a2f
added empty vhd and nfd processors; moved client factory from old repo
Nov 29, 2023
6ee1f84
Merge remote-tracking branch 'origin/cli-redesign' into jdarby/add-st…
jddarby Nov 29, 2023
7581bca
Merge pull request #123 from jddarby/jdarby/add-status-commands
jddarby Nov 29, 2023
5d31127
Add template parser interface (BaseParser)
Nov 29, 2023
39d24b5
Add parameters to BaseParser
Nov 29, 2023
0d78d45
Make defaults_path optional in base_parser
Nov 29, 2023
acc0dc0
defaults_path defaults to None in base_parser
Nov 29, 2023
a0ef8e1
add helm chart parser class and common exceptions
jddarby Nov 29, 2023
c6592ca
add missing doc strings
jddarby Nov 29, 2023
0abc904
Change parser class to input template class
Nov 29, 2023
3485ac2
added all input config params + comments; added common arm template c…
Nov 29, 2023
648524e
Merge branch 'cli-redesign' of github.com:jddarby/azure-cli-extension…
Nov 29, 2023
364217c
add typing to base processor and add copyright statement to new files
jddarby Nov 29, 2023
2307e7f
fix conflicts
jddarby Nov 29, 2023
7b224ff
fixed formatting on input configs
Nov 29, 2023
1bdb3c8
fixed nsd input config + tidied other input config files
Nov 29, 2023
e404652
Merge remote-tracking branch 'origin/cli-redesign' into jdarby/helm-c…
jddarby Nov 29, 2023
4cba6db
added comments for images; general formatting
Nov 29, 2023
d1f12b6
rename template parsers to input templates and add helm chart processor
jddarby Nov 29, 2023
89a7786
add __init__.py to folders
jddarby Nov 30, 2023
2832e82
fixed imports + added innit in folders
Nov 30, 2023
9bdf2b5
Merge branch 'cli-redesign' of github.com:jddarby/azure-cli-extension…
Nov 30, 2023
9615a95
fixed imports + added defaults for input params
Nov 30, 2023
596846d
Not quite complete first pass at definition folder handlers
PerfectChaos Nov 30, 2023
c50e9b2
added generate config log; fixed input config objects
Dec 4, 2023
5e4c447
Merge branch 'cli-redesign' of github.com:jddarby/azure-cli-extension…
Dec 4, 2023
150ec15
rename InputTemplate to InputArtifacts
jddarby Dec 4, 2023
58535e5
tidied write config + added read config; deleted json file checked in…
Dec 4, 2023
0e63ce7
added overwrite validation to generate config; added j2 templates to …
Dec 5, 2023
23fe02c
added validation to base config and nsd
Dec 5, 2023
a55d7d3
Add UTs and fix up definition folder code
PerfectChaos Dec 6, 2023
abdba4c
Merge branch 'cli-redesign' of github.com:jddarby/azure-cli-extension…
PerfectChaos Dec 6, 2023
652c63f
added validation to nsd + cnf; added default_file_name property; adde…
Dec 6, 2023
e7c1d8f
add vhd processor
jddarby Dec 7, 2023
5791ccb
merge cli-redesign branch and fix up conflicts and issues
jddarby Dec 7, 2023
d2ebe7a
remove leftover file from merge
jddarby Dec 7, 2023
d074000
added vnf config validation; improved comment spacing on input.json; …
Dec 7, 2023
e7a6760
Merge branch 'cli-redesign' of github.com:jddarby/azure-cli-extension…
Dec 7, 2023
fa44839
added optional output file name param to generate config; removed mis…
Dec 7, 2023
ae06f97
Fix up some type hinting
Dec 7, 2023
2d64c1d
added write manifest bicep and nf application to base handler; fixed …
Dec 8, 2023
fbae371
push latest changes
jddarby Dec 8, 2023
f09b779
Initial ArmInputTemplate class
Nov 29, 2023
da37d4b
Outline ArmBuildProcessor hierarchy
Nov 29, 2023
456830e
Update VNF tests to keep built output for comparison
Nov 29, 2023
79ef1cd
WIP, for sharing with Jacob
Dec 7, 2023
abc0fb5
Incremental update
Dec 7, 2023
7705b84
added vnf config validation; improved comment spacing on input.json; …
Dec 7, 2023
4d49c6c
added optional output file name param to generate config; removed mis…
Dec 7, 2023
75653a1
Approximately complete ARM processor. No tests, no testing yet.
Dec 7, 2023
55d8c11
Fix artifact store reference
Dec 8, 2023
8138a0d
WIP
Dec 8, 2023
fe6193f
fixed cnf bicep template
Dec 9, 2023
5621f01
Merge remote-tracking branch 'origin/achurchard/vnf-handler' into jl/…
Dec 9, 2023
46b9db8
fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attemp…
Dec 9, 2023
307bd7e
fixed cnfdef bicep; return correct type in cnf handler
Dec 9, 2023
c43f533
merged cli-redesign into branch
Dec 9, 2023
c41abb0
Added outputfolder name to base handler; reorganised templates folder…
Dec 13, 2023
606cf2b
renamed write manifest; updated nf templates to deploy initial resour…
Dec 13, 2023
d8b25fc
added logic to cnf handler for manifest and definition supporting fil…
Dec 13, 2023
0501506
fixed cnf params files; added render params for vnf
Dec 13, 2023
6460a31
fix HelmChartInput issues
jddarby Dec 14, 2023
167db94
merge main and fix conflicts
jddarby Dec 14, 2023
a4b4222
Merge remote-tracking branch 'origin/cli-redesign' into jdarby/helm-c…
jddarby Dec 14, 2023
2b6e17c
Merge arm-processor code into Jacob's branch (#127)
jddarby Dec 14, 2023
eddaae9
added build deploy params schema to base handler; added functionality…
Dec 14, 2023
666aaae
fix majority of linting issues
jddarby Dec 14, 2023
d2b67cf
Merge branch 'cli-redesign' into jdarby/add-build-processors
jddarby Dec 14, 2023
fc7655d
renamed write on base handler to render; added build logic for manife…
Dec 14, 2023
c8e4a21
added write manifest bicep and nf application to base handler; fixed …
Dec 8, 2023
366812a
fixed cnf bicep template
Dec 9, 2023
32e9b18
Initial ArmInputTemplate class
Nov 29, 2023
0f48856
complete helm chart processor
jddarby Dec 14, 2023
d93d0d4
almost finished NFD processor
jddarby Dec 14, 2023
6e1d28e
WIP, for sharing with Jacob
Dec 7, 2023
54745f3
Incremental update
Dec 7, 2023
b9c6f33
added vnf config validation; improved comment spacing on input.json; …
Dec 7, 2023
a8cd7e0
added optional output file name param to generate config; removed mis…
Dec 7, 2023
d06d075
Approximately complete ARM processor. No tests, no testing yet.
Dec 7, 2023
b5b1bff
WIP
Dec 8, 2023
df3e7f4
fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attemp…
Dec 9, 2023
49d9be0
fixed cnfdef bicep; return correct type in cnf handler
Dec 9, 2023
7b1d00f
Added outputfolder name to base handler; reorganised templates folder…
Dec 13, 2023
a5672ad
renamed write manifest; updated nf templates to deploy initial resour…
Dec 13, 2023
505174c
added logic to cnf handler for manifest and definition supporting fil…
Dec 13, 2023
1fd6043
fixed cnf params files; added render params for vnf
Dec 13, 2023
1870c7d
added build deploy params schema to base handler; added functionality…
Dec 14, 2023
2d21df5
renamed write on base handler to render; added build logic for manife…
Dec 14, 2023
dc46061
fixed merge conflicts
Dec 14, 2023
a4df2bf
Merge branch 'jdarby/add-build-processors' into jl/generate_manifest_…
jddarby Dec 15, 2023
3f11dad
fix up code
jddarby Dec 15, 2023
eda2d54
fix base input
jddarby Dec 15, 2023
c0796a2
run isort
jddarby Dec 15, 2023
320f3b0
implemented vnf build + removed test code; fixed arm processor
Dec 15, 2023
3194a38
fixed templates
Dec 15, 2023
0727a68
removed multiple vhds from vnf + fixed bicep; made path to mappings o…
Dec 15, 2023
17a5973
renamed write file to render contents
Dec 15, 2023
3faff68
fixed helm chart processor + small tidy up
Dec 19, 2023
ebee020
commit for build handover; not working + handover notes sent separately
Dec 19, 2023
81892fb
run isort
jddarby Dec 20, 2023
434758f
Merge branch 'cli-redesign' into jdarby/add-build-processors
jddarby Dec 20, 2023
3544877
fix issues in input classes
jddarby Dec 20, 2023
7333360
add generic schema and value mapping generation and fix arm and vhd p…
jddarby Dec 21, 2023
eff311f
push changes I forgot to push before xmas
jddarby Jan 3, 2024
020c821
checked out fixed build processors and inputs from jdarby/add-build-p…
Jan 3, 2024
ddeafdd
fix values and schema generation
jddarby Jan 3, 2024
1ba8a4c
add RET generation for ARM templates
jddarby Jan 3, 2024
884a2b1
fix linting issues
jddarby Jan 3, 2024
720a642
made input file name constant + renamed inputs to match changes
Jan 4, 2024
9c31c1d
checked out build processor changes
Jan 4, 2024
7344bb1
Moved _render_deployment_params_schema and _build_deploy_params_schem…
Jan 4, 2024
b469c0e
changed j2 template for vhd image name
Jan 4, 2024
da5c5ef
HELM CHART PROCESSOR CHANGES! fixed registry value path and depends o…
Jan 4, 2024
2da9ad0
fixed nsd artifact manifest, changed artifact type in all processers …
Jan 4, 2024
d75016d
implemented nsd build get artifact manifest and get artifact list
Jan 4, 2024
4943394
added base bicep to all handlers; edited j2 templates to note that ba…
Jan 5, 2024
316d72a
added to_dict for localfileACRartifact
Jan 5, 2024
b40f080
added source namespcae and source reigstry to artifact + in helm proc…
Jan 5, 2024
92fb1dc
finish nfd processor
jddarby Jan 8, 2024
a78e08a
add copyright statement to helm processor file
jddarby Jan 8, 2024
6bc703f
linting
jddarby Jan 8, 2024
c3b80f5
created common params folder (making new jsonbuilder), adding get par…
Jan 8, 2024
a84cf66
partially ready commit; added cmd context to base + added some logic …
Jan 8, 2024
2a0f964
added all parameters constant
Jan 9, 2024
fa9354e
tidied vnf handler; added common params logic to cnf
Jan 9, 2024
852f3cb
markups inc tidy up
Jan 9, 2024
fa0c0b3
renamed snake_case param
Jan 9, 2024
411b64c
add nsd output config
Jan 9, 2024
7a0df44
taken some input and build processor changes from jdarby/build_proces…
Jan 9, 2024
79904da
Add logging, improve doc string and fix mypy errors for inputs
jddarby Jan 9, 2024
21a9446
fix 2 bugs introduced by merging buildprocessors and adding json depl…
Jan 9, 2024
c5170fc
fixed how aosm client is passed to build; properly get nfdv object wi…
Jan 9, 2024
3ba573f
refactored vnf handler to create processors upfront
Jan 10, 2024
ad5189c
refactor cnf to instantiate processors upfront
Jan 10, 2024
f39cdbc
Improve docstrings, add logging and fix linting issues for build proc…
jddarby Jan 10, 2024
46e3606
nsd temp commit
Jan 11, 2024
5e16dcf
Merge remote-tracking branch 'origin/jdarby/add-build-processors' int…
jddarby Jan 11, 2024
79cf90d
fix missed conflict
jddarby Jan 11, 2024
4459e0e
fix linting and formatting issue
jddarby Jan 11, 2024
576c92c
rename + add constants + add new definition tempalte
Jan 11, 2024
1d8b67c
simplified render bicep definiton contents + removes all constants fo…
Jan 11, 2024
dba5ef4
build nsd bicep core working
Jan 11, 2024
70c65a6
fixed nsd bicep
Jan 11, 2024
d97d3b2
Fix two minor CLI bugs (#129)
patrykkulik-microsoft Jan 12, 2024
72498fa
small fixes; removed location from nfd input, removed comma from vnd …
Jan 12, 2024
fbfeae0
made cgs name constant to fix error in mappings
Jan 12, 2024
beb73ff
added store type to all the templates
Jan 12, 2024
be91506
tidy up + small markups
Jan 12, 2024
a61ad80
removed version state from the definition biceps
Jan 15, 2024
a096904
fixed bug where schemas and defaults were copied incorrectly
Jan 15, 2024
c9c7a03
made artifact paths absolute in all cases except nsd nfs, where paht …
Jan 15, 2024
5d1ba28
added missing comments; renamed common params + edited its format; ch…
Jan 16, 2024
a46ddf3
added existing back to all templates + descriptors of resources
Jan 16, 2024
560a3d7
First stab at doing helm template - check the changes here
patrykkulik-microsoft Jan 16, 2024
ff5f3ac
Merge publish into temp wheel branch (#131)
jordlay Jan 17, 2024
8d7d82f
Minor input improvements (#132)
jordlay Jan 17, 2024
d6b9ffb
edited input file (#133)
jordlay Jan 17, 2024
628d29f
Add skip step
patrykkulik-microsoft Jan 18, 2024
b36d7fc
fixed getparamsconfig for vnf/nsd; added fixes for building correct p…
Jan 18, 2024
9459197
Achurchard/publish (#130)
Cyclam Jan 19, 2024
896918e
bug fixes for chart with subcharts
Jan 19, 2024
6e29c0d
added ruamel
Jan 19, 2024
8fe1299
Use safe yaml loading
Jan 19, 2024
a00b844
Move code out of import list
Jan 19, 2024
267916a
Merge branch 'jl/build_markups' into temp_mavenir_wheel
Jan 19, 2024
d22bfb9
Fix errant config line bug
Jan 19, 2024
e9cf042
fixed deployparams error (#134)
jordlay Jan 19, 2024
d569bab
fixed yaml load error
Jan 19, 2024
a056a36
Merge temp_mavenir_wheel
patrykkulik-microsoft Jan 19, 2024
09f1a91
Change "path_to_mappings" to "default_values"
patrykkulik-microsoft Jan 19, 2024
04aac62
Add unit tests
patrykkulik-microsoft Jan 22, 2024
fa5cff2
Merge branch 'main' into pk5/parse_helm_images
patrykkulik-microsoft Jan 22, 2024
7b67ae8
remove the merge mistake
patrykkulik-microsoft Jan 22, 2024
b355e60
Delete a file
patrykkulik-microsoft Jan 22, 2024
a43686e
Minor error fix
patrykkulik-microsoft Jan 22, 2024
13b6cdb
First round of markups
patrykkulik-microsoft Jan 23, 2024
6e72896
Markups round 2
patrykkulik-microsoft Jan 23, 2024
dfcaf76
Comment out unit tests which are not working
patrykkulik-microsoft Jan 23, 2024
fbf1560
Updating changelog
patrykkulik-microsoft Jan 23, 2024
c62c472
Remove the temporary helm_template file
patrykkulik-microsoft Jan 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/aosm/HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ Release History
Unreleased
++++++++
* Add `publisher` command group for management of publisher resources.
* Changed the name of the `path_to_mappings` parameter in the CNF input file to `default_values`
* Added a `helm template` validation step to the `az aosm nfd build` command for the `cnf` definition type

1.0.0b4
++++++++
Expand Down
22 changes: 16 additions & 6 deletions src/aosm/azext_aosm/_params.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,14 @@
from argcomplete.completers import FilesCompleter
from azure.cli.core import AzCommandsLoader

from .common.constants import CNF, VNF, BICEP_PUBLISH, ARTIFACT_UPLOAD, IMAGE_UPLOAD
from .common.constants import (
CNF,
VNF,
BICEP_PUBLISH,
ARTIFACT_UPLOAD,
IMAGE_UPLOAD,
HELM_TEMPLATE,
)


def load_arguments(self: AzCommandsLoader, _):
Expand All @@ -17,7 +24,9 @@ def load_arguments(self: AzCommandsLoader, _):
)

definition_type = get_enum_type([VNF, CNF])
nf_skip_steps = get_enum_type([BICEP_PUBLISH, ARTIFACT_UPLOAD, IMAGE_UPLOAD])
nf_skip_steps = get_enum_type(
[BICEP_PUBLISH, ARTIFACT_UPLOAD, IMAGE_UPLOAD, HELM_TEMPLATE]
)
ns_skip_steps = get_enum_type([BICEP_PUBLISH, ARTIFACT_UPLOAD])

# Set the argument context so these options are only available when this specific command
Expand Down Expand Up @@ -53,18 +62,19 @@ def load_arguments(self: AzCommandsLoader, _):
options_list=["--build-output-folder", "-b"],
type=file_type,
completer=FilesCompleter(allowednames="*.json"),
help=(
"Path to the folder to publish, created by the build command."
),
help=("Path to the folder to publish, created by the build command."),
)
# This will only ever output one string and will fail if more than one
# skip steps are provided. It might be good to change that.
c.argument(
"skip",
arg_type=nf_skip_steps,
help=(
"Optional skip steps. 'bicep-publish' will skip deploying the bicep "
"template; 'artifact-upload' will skip uploading any artifacts; "
"'image-upload' will skip uploading the VHD image (for VNFs) or the "
"container images (for CNFs)."
"container images (for CNFs); 'helm-template' will skip templating the "
"helm charts (for CNFs)."
),
)
c.argument(
Expand Down
26 changes: 16 additions & 10 deletions src/aosm/azext_aosm/cli_handlers/onboarding_base_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,13 @@
)
from azext_aosm.definition_folder.reader.definition_folder import DefinitionFolder
from azext_aosm.common.command_context import CommandContext
from azext_aosm.configuration_models.common_parameters_config import \
BaseCommonParametersConfig
from azext_aosm.configuration_models.common_parameters_config import (
BaseCommonParametersConfig,
)
from azext_aosm.common.local_file_builder import LocalFileBuilder
from azext_aosm.vendored_sdks import HybridNetworkManagementClient
from azext_aosm.common.constants import DEPLOYMENT_PARAMETERS_FILENAME

logger = get_logger(__name__)


Expand All @@ -36,17 +38,19 @@ def __init__(
self,
provided_input_path: Optional[Path] = None,
aosm_client: Optional[HybridNetworkManagementClient] = None,
skip: str = None,
patrykkulik-microsoft marked this conversation as resolved.
Show resolved Hide resolved
):
"""Initialize the CLI handler."""
self.aosm_client = aosm_client
self.skip = skip
# If config file provided (for build, publish and delete)
if provided_input_path:
provided_input_path = Path(provided_input_path)
# If config file is the input.jsonc for build command
if provided_input_path.suffix == ".jsonc":
config_dict = self._read_input_config_from_file(provided_input_path)
self.config = self._get_input_config(config_dict)
# Validate config before getting processor list,
# Validate config before getting processor list,
# in case error with input artifacts i.e helm package
self.config.validate()
self.processors = self._get_processor_list()
Expand All @@ -59,7 +63,6 @@ def __init__(
# If no config file provided (for generate-config)
else:
self.config = self._get_input_config()

self.definition_folder_builder = DefinitionFolderBuilder(
Path.cwd() / self.output_folder_file_name
)
Expand Down Expand Up @@ -89,6 +92,7 @@ def generate_config(self, output_file: str | None = None):

def build(self):
"""Build the definition."""
self.pre_validate_build()
patrykkulik-microsoft marked this conversation as resolved.
Show resolved Hide resolved
self.definition_folder_builder.add_element(self.build_base_bicep())
self.definition_folder_builder.add_element(self.build_manifest_bicep())
self.definition_folder_builder.add_element(self.build_artifact_list())
Expand All @@ -98,7 +102,9 @@ def build(self):

def publish(self, command_context: CommandContext):
"""Publish the definition contained in the specified definition folder."""
definition_folder = DefinitionFolder(command_context.cli_options["definition_folder"])
definition_folder = DefinitionFolder(
command_context.cli_options["definition_folder"]
)
definition_folder.deploy(config=self.config, command_context=command_context)

def delete(self, command_context: CommandContext):
Expand All @@ -109,6 +115,10 @@ def delete(self, command_context: CommandContext):
# - Do element.delete()
# TODO: Implement

def pre_validate_build(self):
"""Perform all validations that need to be done before running the build command."""
pass

@abstractmethod
def build_base_bicep(self):
"""Build bicep file for underlying resources."""
Expand Down Expand Up @@ -173,11 +183,7 @@ def _render_base_bicep_contents(self, template_path):
bicep_contents: str = template.render()
return bicep_contents

def _render_definition_bicep_contents(
self,
template_path: Path,
params
):
def _render_definition_bicep_contents(self, template_path: Path, params):
"""Write the definition bicep file from given template."""
with open(template_path, "r", encoding="UTF-8") as f:
template: Template = Template(
Expand Down
109 changes: 81 additions & 28 deletions src/aosm/azext_aosm/cli_handlers/onboarding_cnf_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@
from __future__ import annotations

import json
import ruamel.yaml
from ruamel.yaml.error import ReusedAnchorWarning
import warnings
from pathlib import Path
import ruamel.yaml
from ruamel.yaml.error import ReusedAnchorWarning

from jinja2 import Template

from azure.cli.core.azclierror import UnclassifiedUserFault
from azure.cli.core.azclierror import ValidationError
from knack.log import get_logger

from azext_aosm.build_processors.helm_chart_processor import HelmChartProcessor
Expand All @@ -31,21 +33,26 @@
from azext_aosm.definition_folder.builder.json_builder import (
JSONDefinitionElementBuilder,
)
from azext_aosm.common.constants import (ARTIFACT_LIST_FILENAME,
BASE_FOLDER_NAME,
CNF_BASE_TEMPLATE_FILENAME,
CNF_TEMPLATE_FOLDER_NAME,
CNF_DEFINITION_TEMPLATE_FILENAME,
CNF_INPUT_FILENAME,
CNF_MANIFEST_TEMPLATE_FILENAME,
CNF_OUTPUT_FOLDER_FILENAME,
MANIFEST_FOLDER_NAME,
NF_DEFINITION_FOLDER_NAME,
DEPLOYMENT_PARAMETERS_FILENAME)
from azext_aosm.common.constants import (
ARTIFACT_LIST_FILENAME,
BASE_FOLDER_NAME,
CNF_BASE_TEMPLATE_FILENAME,
CNF_TEMPLATE_FOLDER_NAME,
CNF_DEFINITION_TEMPLATE_FILENAME,
CNF_HELM_VALIDATION_ERRORS_TEMPLATE_FILENAME,
CNF_INPUT_FILENAME,
CNF_MANIFEST_TEMPLATE_FILENAME,
CNF_OUTPUT_FOLDER_FILENAME,
HELM_TEMPLATE,
MANIFEST_FOLDER_NAME,
NF_DEFINITION_FOLDER_NAME,
DEPLOYMENT_PARAMETERS_FILENAME,
)

from .onboarding_nfd_base_handler import OnboardingNFDBaseCLIHandler

logger = get_logger(__name__)
yaml_processor = ruamel.yaml.YAML(typ="safe", pure=True)
warnings.simplefilter("ignore", ReusedAnchorWarning)


Expand All @@ -68,9 +75,7 @@ def _get_input_config(self, input_config: dict = None) -> OnboardingCNFInputConf
input_config = {}
return OnboardingCNFInputConfig(**input_config)

def _get_params_config(
self, config_file: Path = None
) -> CNFCommonParametersConfig:
def _get_params_config(self, config_file: Path = None) -> CNFCommonParametersConfig:
"""Get the configuration for the command."""
with open(config_file, "r", encoding="utf-8") as _file:
params_dict = json.load(_file)
Expand All @@ -82,19 +87,22 @@ def _get_processor_list(self) -> [HelmChartProcessor]:
processor_list = []
# for each helm package, instantiate helm processor
for helm_package in self.config.helm_packages:
if helm_package.path_to_mappings:
if Path(helm_package.path_to_mappings).exists():
yaml = ruamel.yaml.YAML(typ='safe', pure=True)
provided_config = yaml.load(open(helm_package.path_to_mappings))
if helm_package.default_values:
if Path(helm_package.default_values).exists():
provided_config = yaml_processor.load(
open(helm_package.default_values)
)
else:
raise UnclassifiedUserFault(
raise FileNotFoundError(
"There is no file at the path provided for the mappings file."
)
else:
provided_config = None

helm_input = HelmChartInput.from_chart_path(
Path(helm_package.path_to_chart).absolute(), default_config=provided_config
Path(helm_package.path_to_chart).absolute(),
default_config=provided_config,
default_config_path=helm_package.default_values,
)
helm_processor = HelmChartProcessor(
helm_package.name,
Expand All @@ -105,6 +113,53 @@ def _get_processor_list(self) -> [HelmChartProcessor]:
processor_list.append(helm_processor)
return processor_list

def _validate_helm_template(self):
"""Validate the helm packages."""
helm_chart_processors = self._get_processor_list()

validation_errors = {}

for helm_processor in helm_chart_processors:
validation_output = helm_processor.input_artifact.validate_template()

if validation_output:
validation_errors[
helm_processor.input_artifact.artifact_name
] = validation_output

if validation_errors:
# Create an error file using a j2 template
error_output_template_path = self._get_template_path(
CNF_TEMPLATE_FOLDER_NAME, CNF_HELM_VALIDATION_ERRORS_TEMPLATE_FILENAME
)

with open(
error_output_template_path,
"r",
encoding="utf-8",
) as file:
error_output_template = Template(file.read())

rendered_error_output_template = error_output_template.render(
errors=validation_errors
)

logger.info(rendered_error_output_template)

error_message = (
"Could not validate all the provided Helm charts. "
"Please run the CLI command again with the --verbose flag "
"to see the validation errors."
)

raise ValidationError(error_message)

def pre_validate_build(self):
"""Run all validation functions required before building the cnf."""
logger.debug("Pre-validating build")
if self.skip != HELM_TEMPLATE:
self._validate_helm_template()
patrykkulik-microsoft marked this conversation as resolved.
Show resolved Hide resolved

def build_base_bicep(self):
"""Build the base bicep file."""
# Build manifest bicep contents, with j2 template
Expand Down Expand Up @@ -203,11 +258,9 @@ def build_resource_bicep(self):

params = {
"acr_nf_applications": nf_application_list,
"deployment_parameters_file": DEPLOYMENT_PARAMETERS_FILENAME
"deployment_parameters_file": DEPLOYMENT_PARAMETERS_FILENAME,
}
bicep_contents = self._render_definition_bicep_contents(
template_path, params
)
bicep_contents = self._render_definition_bicep_contents(template_path, params)

# Create a bicep element + add its supporting mapping files
bicep_file = BicepDefinitionElementBuilder(
Expand All @@ -233,7 +286,7 @@ def build_all_parameters_json(self):
"acrArtifactStoreName": self.config.acr_artifact_store_name,
"acrManifestName": self.config.acr_artifact_store_name + "-manifest",
"nfDefinitionGroup": self.config.nf_name,
"nfDefinitionVersion": self.config.version
"nfDefinitionVersion": self.config.version,
}

base_file = JSONDefinitionElementBuilder(
Expand Down
1 change: 1 addition & 0 deletions src/aosm/azext_aosm/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from azure.cli.core import AzCommandsLoader

from ._client_factory import cf_aosm # pylint: disable=import-error

# from azext_aosm._client_factory import cf_aosm


Expand Down
3 changes: 3 additions & 0 deletions src/aosm/azext_aosm/common/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,14 @@ class DeployableResourceTypes(str, Enum):
BICEP_PUBLISH = "bicep-publish"
ARTIFACT_UPLOAD = "artifact-upload"
IMAGE_UPLOAD = "image-upload"
HELM_TEMPLATE = "helm-template"


class SkipSteps(Enum):
BICEP_PUBLISH = BICEP_PUBLISH
ARTIFACT_UPLOAD = ARTIFACT_UPLOAD
IMAGE_UPLOAD = IMAGE_UPLOAD
HELM_TEMPLATE = HELM_TEMPLATE


class ManifestsExist(str, Enum):
Expand Down Expand Up @@ -69,6 +71,7 @@ class ManifestsExist(str, Enum):
CNF_INPUT_FILENAME = "cnf-input.jsonc"
CNF_DEFINITION_TEMPLATE_FILENAME = "cnfdefinition.bicep.j2"
CNF_MANIFEST_TEMPLATE_FILENAME = "cnfartifactmanifest.bicep.j2"
CNF_HELM_VALIDATION_ERRORS_TEMPLATE_FILENAME = "cnfhelmtemplateerrors.txt.j2"
CNF_BASE_TEMPLATE_FILENAME = "cnfbase.bicep"
CNF_VALUES_SCHEMA_FILENAME = "values.schema.json"
CNF_TEMPLATE_FOLDER_NAME = "cnf"
Expand Down
7 changes: 7 additions & 0 deletions src/aosm/azext_aosm/common/exceptions.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
from azure.cli.core.azclierror import UserFault


class InvalidFileTypeError(Exception):
"""Raised when the file type is not supported by the parser"""


class MissingDependency(UserFault):
"""Raised when the required dependency is missing"""


class MissingChartDependencyError(Exception):
"""Raised when the chart dependency is missing"""

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Errors were found in the following Helm charts (see below for details). Please fix them and try again.

{% for chart_name in errors.keys() %}
- {{ chart_name }}
{% endfor %}

{% for chart_name, error_message in errors.items() %}
----------------------------------------
Error in Helm chart: {{ chart_name }}
----------------------------------------

{{ error_message }}

{% endfor %}
Loading