Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AZ AOSM Extension Refactor #7429

Merged
merged 46 commits into from
May 20, 2024
Merged

AZ AOSM Extension Refactor #7429

merged 46 commits into from
May 20, 2024

Conversation

jordlay
Copy link
Contributor

@jordlay jordlay commented Mar 28, 2024


  • History.RST and setup.py up to date for release of 2.0.0b
  • Large scale refactor of az aosm extension
  • Logic reviewed by responsible team
  • Good unit test coverage
  • Working integration tests using ScenarioTest

This checklist is used to make sure that common guidelines for a pull request are followed.

Related command

General Guidelines

  • Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
  • Have you run python scripts/ci/test_index.py -q locally? (pip install wheel==0.30.0 required)
  • My extension version conforms to the Extension version schema

For new extensions:

About Extension Publish

There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

jordlay and others added 27 commits January 19, 2024 17:10
* added empty template parser

* Initial scaffolding of some new classes

* added utils, added base builder + reader + their children, added nfd and vhd processors

* moved utils to common folder + added local file builder

* Add some basic implementations

* add new commands and restructure command groups

* better naming for command groups

* revert addition on the onboard command group

* update HISTORY.rst

* fix style issue

* fix linting issues

* added empty vhd and nfd processors; moved client factory from old repo

* Add template parser interface (BaseParser)

* Add parameters to BaseParser

* Make defaults_path optional in base_parser

* defaults_path defaults to None in base_parser

* add helm chart parser class and common exceptions

* add missing doc strings

* Change parser class to input template class

* added all input config params + comments; added common arm template config; added instructions for completing publish/delete

* add typing to base processor and add copyright statement to new files

* fixed formatting on input configs

* fixed nsd input config + tidied other input config files

* added comments for images; general formatting

* rename template parsers to input templates and add helm chart processor

* add __init__.py to folders

* fixed imports + added innit in folders

* fixed imports + added defaults for input params

* Not quite complete first pass at definition folder handlers

* added generate config log; fixed input config objects

* rename InputTemplate to InputArtifacts

* tidied write config + added read config; deleted json file checked in by mistake

* added overwrite validation to generate config; added j2 templates to common

* added validation to base config and nsd

* Add UTs and fix up definition folder code

* added validation to nsd + cnf; added default_file_name property; added tests for generate config; added example nsd-input.jsonc for testing

* add vhd processor

* remove leftover file from merge

* added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params

* added optional output file name param to generate config; removed misc print statements

* Fix up some type hinting

* added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep

* push latest changes

* Initial ArmInputTemplate class

* Outline ArmBuildProcessor hierarchy

* Update VNF tests to keep built output for comparison

* WIP, for sharing with Jacob

* Incremental update

* added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params

* added optional output file name param to generate config; removed misc print statements

* Approximately complete ARM processor. No tests, no testing yet.

* Fix artifact store reference

* WIP

* fixed cnf bicep template

* fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case

* fixed cnfdef bicep; return correct type in cnf handler

* Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder

* renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working

* added logic to cnf handler for manifest and definition supporting files + config mappings

* fixed cnf params files; added render params for vnf

* fix HelmChartInput issues

* Merge arm-processor code into Jacob's branch (#127)

* Initial ArmInputTemplate class

* Outline ArmBuildProcessor hierarchy

* Update VNF tests to keep built output for comparison

* Initial ArmInputTemplate class

* Outline ArmBuildProcessor hierarchy

* Update VNF tests to keep built output for comparison

* WIP, for sharing with Jacob

* Incremental update

* Approximately complete ARM processor. No tests, no testing yet.

* Fix artifact store reference

---------

Co-authored-by: Andy Churchard <[email protected]>

* added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd

* fix majority of linting issues

* renamed write on base handler to render; added build logic for manifest and artifact list for vnf

* added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep

* fixed cnf bicep template

* Initial ArmInputTemplate class

* complete helm chart processor

* almost finished NFD processor

* WIP, for sharing with Jacob

* Incremental update

* added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params

* added optional output file name param to generate config; removed misc print statements

* Approximately complete ARM processor. No tests, no testing yet.

* WIP

* fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case

* fixed cnfdef bicep; return correct type in cnf handler

* Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder

* renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working

* added logic to cnf handler for manifest and definition supporting files + config mappings

* fixed cnf params files; added render params for vnf

* added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd

* renamed write on base handler to render; added build logic for manifest and artifact list for vnf

* fix up code

* fix base input

* run isort

* implemented vnf build + removed test code; fixed arm processor

* fixed templates

* removed multiple vhds from vnf + fixed bicep; made path to mappings optional for cnf

* renamed write file to render contents

* fixed helm chart processor + small tidy up

* commit for build handover; not working + handover notes sent separately

* run isort

* fix issues in input classes

* add generic schema and value mapping generation and fix arm and vhd processors

* push changes I forgot to push before xmas

* checked out fixed build processors and inputs from jdarby/add-build-processors

* fix values and schema generation

* add RET generation for ARM templates

* fix linting issues

* made input file name constant + renamed inputs to match changes

* checked out build processor changes

* Moved _render_deployment_params_schema and _build_deploy_params_schema to nfd base handler; added default_config logic to cnf handler;created deploymentParamers for cnf + vnf

* changed j2 template for vhd image name

* HELM CHART PROCESSOR CHANGES! fixed registry value path and depends on profile in cnf template

* fixed nsd artifact manifest, changed artifact type in all processers (PROCESSOR CHANGE)

* implemented nsd build get artifact manifest and get artifact list

* added base bicep to all handlers; edited j2 templates to note that base resources should be deployed first on publish; made all build use constants

* added to_dict for localfileACRartifact

* added source namespcae and source reigstry to artifact + in helm processor callinh remote arc artifacts

* finish nfd processor

* add copyright statement to helm processor file

* linting

* created common params folder (making new jsonbuilder), adding get params config to base', renamed get_config to get_input_config; aded logging to vnf; chnged custom.py publish inputs

* partially ready commit; added cmd context to base + added some logic to nsds

* added all parameters constant

* tidied vnf handler; added common params logic to cnf

* markups inc tidy up

* renamed snake_case param

* add nsd output config

* taken some input and build processor changes from jdarby/build_processor branch

* Add logging, improve doc string and fix mypy errors for inputs

* fix 2 bugs introduced by merging buildprocessors and adding json deploy params

* fixed how aosm client is passed to build; properly get nfdv object with api client

* refactored vnf handler to create processors upfront

* refactor cnf to instantiate processors upfront

* Improve docstrings, add logging and fix linting issues for build processors

* nsd temp commit

* fix missed conflict

* fix linting and formatting issue

* rename + add constants + add new definition tempalte

* simplified render bicep definiton contents + removes all constants for filenames from nf handlers and templates

* build nsd bicep core working

* fixed nsd bicep

* Fix two minor CLI bugs (#129)

* Fix bug where the build directory wasn't deleted

* Fix bug where the json wasn't output correctly

* fixed incorrect merge fix

---------

Co-authored-by: Jordan <[email protected]>

* small fixes; removed location from nfd input, removed comma from vnd def template, fixed artifact version in nsds, fixed vhd validation

* made cgs name constant to fix error in mappings

* added store type to all the templates

* tidy up + small markups

* removed version state from the definition biceps

* fixed bug where schemas and defaults were copied incorrectly

* made artifact paths absolute in all cases except nsd nfs, where paht is relative to nsd output folder

* added missing comments; renamed common params + edited its format; changed custom to take path

* added existing back to all templates + descriptors of resources

* Merge publish into temp wheel branch (#131)

* Move artifact create from dict logic to instantiating code

* Create CommandContext class to hold Azure clients and CLI options, and pass instance through to the delete and publish commands.

* Mostly implemented and working publish. Lots of debug code still present.

* Publish done and mostly tested.

* Add some more logging. Remove help text referring to --order-params.

* Update src/aosm/azext_aosm/common/artifact.py

Co-authored-by: Cyclam <[email protected]>

* fix merge conflicts

---------

Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Jordan <[email protected]>
Co-authored-by: Cyclam <[email protected]>

* Minor input improvements (#132)

* fixed common params files for all

* remove prints; remove sourcelocaldockerimage + add correct validation; edit input.json comments

* fixed extract tar error

* only create oras_client once - temp solution

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>

* edited input file (#133)

Co-authored-by: Jordan <[email protected]>

* fixed getparamsconfig for vnf/nsd; added fixes for building correct python wheel

* Achurchard/publish (#130)

Markups from publish PR

---------

Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Jordan <[email protected]>
Co-authored-by: jordlay <[email protected]>

* bug fixes for chart with subcharts

* added ruamel

* Use safe yaml loading

* Move code out of import list

* Fix errant config line bug

* fixed deployparams error (#134)

Co-authored-by: Jordan <[email protected]>

* fixed yaml load error

* disabled multi anchor yaml warning for values.yaml

* delete accidental deployparams file

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Chaos Chhapi <[email protected]>
Co-authored-by: Jacob <[email protected]>
Co-authored-by: Jacob Darby <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: patrykkulik-microsoft <[email protected]>
Co-authored-by: Cyclam <[email protected]>
…ext to publish + passed paths properly in custom.py
* added empty template parser

* Initial scaffolding of some new classes

* added utils, added base builder + reader + their children, added nfd and vhd processors

* moved utils to common folder + added local file builder

* Add some basic implementations

* add new commands and restructure command groups

* better naming for command groups

* revert addition on the onboard command group

* update HISTORY.rst

* fix style issue

* fix linting issues

* added empty vhd and nfd processors; moved client factory from old repo

* Add template parser interface (BaseParser)

* Add parameters to BaseParser

* Make defaults_path optional in base_parser

* defaults_path defaults to None in base_parser

* add helm chart parser class and common exceptions

* add missing doc strings

* Change parser class to input template class

* added all input config params + comments; added common arm template config; added instructions for completing publish/delete

* add typing to base processor and add copyright statement to new files

* fixed formatting on input configs

* fixed nsd input config + tidied other input config files

* added comments for images; general formatting

* rename template parsers to input templates and add helm chart processor

* add __init__.py to folders

* fixed imports + added innit in folders

* fixed imports + added defaults for input params

* Not quite complete first pass at definition folder handlers

* added generate config log; fixed input config objects

* rename InputTemplate to InputArtifacts

* tidied write config + added read config; deleted json file checked in by mistake

* added overwrite validation to generate config; added j2 templates to common

* added validation to base config and nsd

* Add UTs and fix up definition folder code

* added validation to nsd + cnf; added default_file_name property; added tests for generate config; added example nsd-input.jsonc for testing

* add vhd processor

* remove leftover file from merge

* added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params

* added optional output file name param to generate config; removed misc print statements

* Fix up some type hinting

* added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep

* push latest changes

* Initial ArmInputTemplate class

* Outline ArmBuildProcessor hierarchy

* Update VNF tests to keep built output for comparison

* WIP, for sharing with Jacob

* Incremental update

* added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params

* added optional output file name param to generate config; removed misc print statements

* Approximately complete ARM processor. No tests, no testing yet.

* Fix artifact store reference

* WIP

* fixed cnf bicep template

* fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case

* fixed cnfdef bicep; return correct type in cnf handler

* Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder

* renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working

* added logic to cnf handler for manifest and definition supporting files + config mappings

* fixed cnf params files; added render params for vnf

* fix HelmChartInput issues

* Merge arm-processor code into Jacob's branch (#127)

* Initial ArmInputTemplate class

* Outline ArmBuildProcessor hierarchy

* Update VNF tests to keep built output for comparison

* Initial ArmInputTemplate class

* Outline ArmBuildProcessor hierarchy

* Update VNF tests to keep built output for comparison

* WIP, for sharing with Jacob

* Incremental update

* Approximately complete ARM processor. No tests, no testing yet.

* Fix artifact store reference

---------

Co-authored-by: Andy Churchard <[email protected]>

* added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd

* fix majority of linting issues

* renamed write on base handler to render; added build logic for manifest and artifact list for vnf

* added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep

* fixed cnf bicep template

* Initial ArmInputTemplate class

* complete helm chart processor

* almost finished NFD processor

* WIP, for sharing with Jacob

* Incremental update

* added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params

* added optional output file name param to generate config; removed misc print statements

* Approximately complete ARM processor. No tests, no testing yet.

* WIP

* fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case

* fixed cnfdef bicep; return correct type in cnf handler

* Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder

* renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working

* added logic to cnf handler for manifest and definition supporting files + config mappings

* fixed cnf params files; added render params for vnf

* added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd

* renamed write on base handler to render; added build logic for manifest and artifact list for vnf

* fix up code

* fix base input

* run isort

* implemented vnf build + removed test code; fixed arm processor

* fixed templates

* removed multiple vhds from vnf + fixed bicep; made path to mappings optional for cnf

* renamed write file to render contents

* fixed helm chart processor + small tidy up

* commit for build handover; not working + handover notes sent separately

* run isort

* fix issues in input classes

* add generic schema and value mapping generation and fix arm and vhd processors

* push changes I forgot to push before xmas

* checked out fixed build processors and inputs from jdarby/add-build-processors

* fix values and schema generation

* add RET generation for ARM templates

* fix linting issues

* made input file name constant + renamed inputs to match changes

* checked out build processor changes

* Moved _render_deployment_params_schema and _build_deploy_params_schema to nfd base handler; added default_config logic to cnf handler;created deploymentParamers for cnf + vnf

* changed j2 template for vhd image name

* HELM CHART PROCESSOR CHANGES! fixed registry value path and depends on profile in cnf template

* fixed nsd artifact manifest, changed artifact type in all processers (PROCESSOR CHANGE)

* implemented nsd build get artifact manifest and get artifact list

* added base bicep to all handlers; edited j2 templates to note that base resources should be deployed first on publish; made all build use constants

* added to_dict for localfileACRartifact

* added source namespcae and source reigstry to artifact + in helm processor callinh remote arc artifacts

* finish nfd processor

* add copyright statement to helm processor file

* linting

* created common params folder (making new jsonbuilder), adding get params config to base', renamed get_config to get_input_config; aded logging to vnf; chnged custom.py publish inputs

* partially ready commit; added cmd context to base + added some logic to nsds

* added all parameters constant

* tidied vnf handler; added common params logic to cnf

* markups inc tidy up

* renamed snake_case param

* add nsd output config

* taken some input and build processor changes from jdarby/build_processor branch

* Add logging, improve doc string and fix mypy errors for inputs

* fix 2 bugs introduced by merging buildprocessors and adding json deploy params

* fixed how aosm client is passed to build; properly get nfdv object with api client

* refactored vnf handler to create processors upfront

* refactor cnf to instantiate processors upfront

* Improve docstrings, add logging and fix linting issues for build processors

* nsd temp commit

* fix missed conflict

* fix linting and formatting issue

* rename + add constants + add new definition tempalte

* simplified render bicep definiton contents + removes all constants for filenames from nf handlers and templates

* build nsd bicep core working

* fixed nsd bicep

* Fix two minor CLI bugs (#129)

* Fix bug where the build directory wasn't deleted

* Fix bug where the json wasn't output correctly

* fixed incorrect merge fix

---------

Co-authored-by: Jordan <[email protected]>

* small fixes; removed location from nfd input, removed comma from vnd def template, fixed artifact version in nsds, fixed vhd validation

* made cgs name constant to fix error in mappings

* added store type to all the templates

* tidy up + small markups

* removed version state from the definition biceps

* fixed bug where schemas and defaults were copied incorrectly

* made artifact paths absolute in all cases except nsd nfs, where paht is relative to nsd output folder

* added missing comments; renamed common params + edited its format; changed custom to take path

* added existing back to all templates + descriptors of resources

* First stab at doing helm template - check the changes here

* Merge publish into temp wheel branch (#131)

* Move artifact create from dict logic to instantiating code

* Create CommandContext class to hold Azure clients and CLI options, and pass instance through to the delete and publish commands.

* Mostly implemented and working publish. Lots of debug code still present.

* Publish done and mostly tested.

* Add some more logging. Remove help text referring to --order-params.

* Update src/aosm/azext_aosm/common/artifact.py

Co-authored-by: Cyclam <[email protected]>

* fix merge conflicts

---------

Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Jordan <[email protected]>
Co-authored-by: Cyclam <[email protected]>

* Minor input improvements (#132)

* fixed common params files for all

* remove prints; remove sourcelocaldockerimage + add correct validation; edit input.json comments

* fixed extract tar error

* only create oras_client once - temp solution

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>

* edited input file (#133)

Co-authored-by: Jordan <[email protected]>

* Add skip step

* fixed getparamsconfig for vnf/nsd; added fixes for building correct python wheel

* Achurchard/publish (#130)

Markups from publish PR

---------

Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Jordan <[email protected]>
Co-authored-by: jordlay <[email protected]>

* bug fixes for chart with subcharts

* added ruamel

* Use safe yaml loading

* Move code out of import list

* Fix errant config line bug

* fixed deployparams error (#134)

Co-authored-by: Jordan <[email protected]>

* fixed yaml load error

* Change "path_to_mappings" to "default_values"

* Add unit tests

* remove the merge mistake

* Delete a file

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Chaos Chhapi <[email protected]>
Co-authored-by: Jacob <[email protected]>
Co-authored-by: Jacob Darby <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: jordlay <[email protected]>
Co-authored-by: Cyclam <[email protected]>
* Add image parsing

* Add unit tests

* Fix small log issue

* Minor markup edits

* Improve unit tests

* remove the test_utils file

* update HISTORY.rst

* update HISTORY file
* add validation of the default values for helm charts

* Add unit tests

* update HISTORY.rst

* Markups
* fix for relative path issue for nsd publish; fix for deployment parameters schema being wrong

* Add image parsing

* LocalFileACRArtifact converts bicep to ARM before upload.

* Remove TODO comment that is done. Refactor convert to bicep code.

* Add unit tests

* Fix small log issue

* Minor markup edits

* Add location to configObject passed to NF template. Value is hardcoded to that provided in the input.jsonc file.

* Add note on using NF name instead of NSD name when creating NFDInput

* Minor fixes: - check attribute exists before accessing it\n - better logging\n - reduce nesting

* input file (jsonc) comment updates.

* Fix NSD generate --output-file handling bug

* Minor markups.

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Patryk Kulik <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>
Linting and type hinting fixes.
* added nexus arm processor; added nfvi type to vnfinput.json; added condition for calling each arm processor

* pipe through nfvi type from vnf input to template; added nexus input; fixed mistakes in armprocessor

* initial commit for children vnf handlers

* added core and nexus config logic

* fixed nexus input and nexus processor; fixed templates

* generalised render manifest and render definition bicep functions to use one

* added incorrect config error handling

* added nexus flag to publish; added error handling for incorrect vnf type (if forgot to put nexus flag case); tidied nexus inputs

* added vnf nexus base bicep; fixed vnf definition template; temp fix for image versions

* moved render bicep to common/utils; replaced build base bicep with render bicep

* general tidyup: removed prints, added return types

* added nfvi type to nsds

* removed old todos

* refactored nexus handler, moved generate params code into processor

* moved logic to base vnf handler; moved more logic to processor

* fix template name in vnf j2

* fixed nexus image file

* added nfvitype to nsd nf template by making new j2 + changing how nfd processor works

* added nfvitype to nsd nf template by making new j2 + changing how nfd processor works

* minor formatting

* made vnfnexus a definition type; slight refactor of custom.py

* removed nexus param, removed prints, added commented out test file

* fixed flake8 + pylint issues

* Base handler treats command inputs separately (#143)

* refactored base handler init to treat inputs from different commands separately; added better validation

* fixed error handling in base handler

* fixed error typing

---------

Co-authored-by: Jordan <[email protected]>

* mypy fixes

* fixed customLocation id for vnf nexus

* markups from review inc moving build manifest to parent vnf handler

* added better docstrings

* updated history.rst

* Achurchard/fix helm chart upload (#144)

* Use helm push for Helm charts (not oras push)

* Logging

---------

Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Jordan <[email protected]>

* bumped version

* Bug: No type in schema (#148)

* first fix for anyOf logic error msg

* Update src/aosm/azext_aosm/build_processors/base_processor.py

Co-authored-by: Cyclam <[email protected]>

* Update src/aosm/azext_aosm/build_processors/base_processor.py

Co-authored-by: Cyclam <[email protected]>

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Cyclam <[email protected]>

* Bug: Nexus Image Version Must be Semver (#149)

* added semver checking to input config validation; moved split image path to utils

* changed semver regex; renamed function

* fixed typo

* markups

* fix typo

---------

Co-authored-by: Jordan <[email protected]>

* Create RG if it doesn't exist (#150)

* add validation resource group exists function

* tidied up code

* Update src/aosm/azext_aosm/definition_folder/reader/definition_folder.py

Co-authored-by: Cyclam <[email protected]>

* renaming from markups

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Cyclam <[email protected]>

* Bug Fix: NFD and NSD Manifest Names Clash (#147)

* added sa_manifest and acr_manifest property, added sa and acr to manifest names; added nsd to nsd manifest name

* create manifest name from nf/nsd name instead of acr/sa; fixed docs strings

---------

Co-authored-by: Jordan <[email protected]>

* Fix Nexus Linting + Add Unit Tests (#146)

* fixed pylint + flake8 errors

* fix mypy errors

* fixed artifact builder tests

* temp commit for unit testing

* temp push of broken tests

* added new mocks (not perfect) for vnfs£

* fixed artifact write failure

* More mypy fixes

* fixed style issues

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>

* Use ephemeral tempdir for generated helm package .tgz file. (#151)

* Use ephemeral tempdir for generated helm package .tgz file.

* Fix file_path bug if .tgz file was provided by user.

---------

Co-authored-by: Andy Churchard <[email protected]>

---------

Co-authored-by: Jordan <[email protected]>
Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Cyclam <[email protected]>
* converted vhd parameters to camel case to match what aosm expects

* markups + fixed apiVersion

---------

Co-authored-by: Jordan <[email protected]>
* moved working tests to new folder structure; added empty tests for processors and build

* added empty generate cofnig tests; fixed and moved bicep,artifact and definitino folder builder tests

* added nexus arm tests; removed outdated vnf mocks

* added azure core tests for every function (including base)

* fixed up core and nexus arm processor

* fixed up arm processors; added logging to tests

* added arm template tests

* added test from copilot learning day

* changed assert

* removed build folder; added json dump to artifact and definition tests; improved arm processor testing

* removed mock open and replaced with new templates

* fixed linting

---------

Co-authored-by: Jordan <[email protected]>
* check object is a dict before looking for `type` key.

---------

Co-authored-by: Jordan <[email protected]>
* Initial stab at making the multiple registry support work

* Update TODO

* update TODO

* merge fixes

* add logic to find registry credentials

* Code cleanup and adding unit tests

* updates to the registry class

* In Progress work

* Working publish

* Move _push_image_from_local_registry and copy_image into the registry class

* tidy up changes

* Namespace and style fixes

* Minor input config change

* unit tests

* Do not push image namespace to target ACR

* Make sure not to break the nexus part of the code

* Update CHANGELOG

* Fix error not being caught correctly

* Minor comment change

* Markups

* Update warning

* Change how we treat namespace and find images

* markups

* modify how we create the registry list

---------

Co-authored-by: Andy Churchard <[email protected]>
* Initial stab at making the multiple registry support work

* Update TODO

* update TODO

* merge fixes

* add logic to find registry credentials

* Code cleanup and adding unit tests

* updates to the registry class

* In Progress work

* Working publish

* Move _push_image_from_local_registry and copy_image into the registry class

* tidy up changes

* Namespace and style fixes

* Minor input config change

* unit tests

* Do not push image namespace to target ACR

* Make sure not to break the nexus part of the code

* Update CHANGELOG

* initial commit for integration tests

* Fix error not being caught correctly

* Minor comment change

* Fix integration tests

* Moving files around

* Working vnf integration test

* Comment out test that is not working

* Clean up the structure of the integration tests

* Delete unnecessary tests

* Fix CNF test and move a util function

* fix other integration tests

* Delete the broken tests folder

* Fix the vnf test

* move files around

* Improve the update_input_file function

* improve the nsd test

* Move files around and improve the nsd output

* remove unnecessary line

* Update recording processors and recordings

* Fix linting issues

* bad merge change

* Update the nsd output with Open CGVs

* Fix few unit tests broken by merge

* Markups

---------

Co-authored-by: Andy Churchard <[email protected]>
Co-authored-by: Jordan <[email protected]>
Copy link

azure-client-tools-bot-prd bot commented Mar 28, 2024

❌Azure CLI Extensions Breaking Change Test
❌aosm
rule cmd_name rule_message suggest_message
1007 - ParaRemove aosm nfd build cmd aosm nfd build removed parameter force please add back parameter force for cmd aosm nfd build
1007 - ParaRemove aosm nfd build cmd aosm nfd build removed parameter interactive please add back parameter interactive for cmd aosm nfd build
1007 - ParaRemove aosm nfd build cmd aosm nfd build removed parameter order_params please add back parameter order_params for cmd aosm nfd build
1002 - CmdRemove aosm nfd delete cmd aosm nfd delete removed please confirm cmd aosm nfd delete removed
1006 - ParaAdd aosm nfd publish cmd aosm nfd publish added parameter build_output_folder please remove parameter build_output_folder for cmd aosm nfd publish
1007 - ParaRemove aosm nfd publish cmd aosm nfd publish removed parameter config_file please add back parameter config_file for cmd aosm nfd publish
1007 - ParaRemove aosm nfd publish cmd aosm nfd publish removed parameter definition_file please add back parameter definition_file for cmd aosm nfd publish
1007 - ParaRemove aosm nfd publish cmd aosm nfd publish removed parameter manifest_file please add back parameter manifest_file for cmd aosm nfd publish
1007 - ParaRemove aosm nfd publish cmd aosm nfd publish removed parameter manifest_params_file please add back parameter manifest_params_file for cmd aosm nfd publish
1007 - ParaRemove aosm nfd publish cmd aosm nfd publish removed parameter parameters_json_file please add back parameter parameters_json_file for cmd aosm nfd publish
1007 - ParaRemove aosm nfd publish cmd aosm nfd publish removed parameter skip please add back parameter skip for cmd aosm nfd publish
1007 - ParaRemove aosm nsd build cmd aosm nsd build removed parameter force please add back parameter force for cmd aosm nsd build
1002 - CmdRemove aosm nsd delete cmd aosm nsd delete removed please confirm cmd aosm nsd delete removed
1006 - ParaAdd aosm nsd publish cmd aosm nsd publish added parameter build_output_folder please remove parameter build_output_folder for cmd aosm nsd publish
1007 - ParaRemove aosm nsd publish cmd aosm nsd publish removed parameter config_file please add back parameter config_file for cmd aosm nsd publish
1007 - ParaRemove aosm nsd publish cmd aosm nsd publish removed parameter design_file please add back parameter design_file for cmd aosm nsd publish
1007 - ParaRemove aosm nsd publish cmd aosm nsd publish removed parameter manifest_file please add back parameter manifest_file for cmd aosm nsd publish
1007 - ParaRemove aosm nsd publish cmd aosm nsd publish removed parameter manifest_params_file please add back parameter manifest_params_file for cmd aosm nsd publish
1007 - ParaRemove aosm nsd publish cmd aosm nsd publish removed parameter parameters_json_file please add back parameter parameters_json_file for cmd aosm nsd publish
1007 - ParaRemove aosm nsd publish cmd aosm nsd publish removed parameter skip please add back parameter skip for cmd aosm nsd publish
⚠️ 1006 - ParaAdd aosm nfd build cmd aosm nfd build added parameter skip
⚠️ 1010 - ParaPropUpdate aosm nfd build cmd aosm nfd build update parameter definition_type: updated property choices from ['cnf', 'vnf'] to ['cnf', 'vnf', 'vnf-nexus']
⚠️ 1010 - ParaPropUpdate aosm nfd generate-config cmd aosm nfd generate-config update parameter definition_type: updated property choices from ['cnf', 'vnf'] to ['cnf', 'vnf', 'vnf-nexus']
⚠️ 1009 - ParaPropRemove aosm nfd generate-config cmd aosm nfd generate-config update parameter output_file: removed property default=input.json
⚠️ 1010 - ParaPropUpdate aosm nfd publish cmd aosm nfd publish update parameter definition_type: updated property choices from ['cnf', 'vnf'] to ['cnf', 'vnf', 'vnf-nexus']
⚠️ 1009 - ParaPropRemove aosm nsd generate-config cmd aosm nsd generate-config update parameter output_file: removed property default=input.json
⚠️ 1006 - ParaAdd aosm nsd publish cmd aosm nsd publish added parameter no_subscription_permissions
⚠️ 1011 - SubgroupAdd aosm publisher sub group aosm publisher added

Copy link

Hi @jordlay,
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in setup.py as well.

is_preview=True,
)
class UpdateState(AAZCommand):
"""Update state for artifact manifest.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add examples for those aaz commands. You can reference this doc

Copy link
Contributor

@Cyclam Cyclam Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you and done. There's just one example per command as these commands are very limited in scope, so more than one example doesn't make sense.

@kairu-ms kairu-ms requested a review from yanzhudd April 18, 2024 04:42
jordlay and others added 3 commits April 19, 2024 18:00
* updated changelog

* markups

---------

Co-authored-by: Jordan <[email protected]>
- Add examples
- Minor rewording
- Ensure all commands are preview
@Cyclam
Copy link
Contributor

Cyclam commented May 2, 2024

@kairu-ms , we've added examples as requested, could you review again, please?

@yonzhan
Copy link
Collaborator

yonzhan commented May 14, 2024

Please fix CI issues

patrykkulik-microsoft and others added 4 commits May 16, 2024 13:45
* Add retry logic to fix integration tests

* Modify the publisher name because previous name is used up

* Regenerate the recordings
@yonzhan yonzhan assigned jsntcy and kairu-ms and unassigned yanzhudd May 16, 2024
@kairu-ms
Copy link
Contributor

/azp run

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@Cyclam
Copy link
Contributor

Cyclam commented May 17, 2024

Thanks @kairu-ms . Do we need to do anything else for this to merge, or is there another button you need to press?

@kairu-ms kairu-ms merged commit 8decf29 into Azure:main May 20, 2024
19 checks passed
@kairu-ms
Copy link
Contributor

Thanks @kairu-ms . Do we need to do anything else for this to merge, or is there another button you need to press?

PR merged

@azclibot
Copy link
Collaborator

[Release] Update index.json for extension [ aosm ] : https://dev.azure.com/azclitools/release/_build/results?buildId=159847&view=results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants