Merge publish into temp wheel branch #131
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 6 commits
December 18, 2023 17:29
…d pass instance through to the delete and publish commands.
Cyclam
reviewed
Jan 16, 2024
Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com>
jordlay
added a commit
that referenced
this pull request
Jan 19, 2024
* added empty template parser * Initial scaffolding of some new classes * added utils, added base builder + reader + their children, added nfd and vhd processors * moved utils to common folder + added local file builder * Add some basic implementations * add new commands and restructure command groups * better naming for command groups * revert addition on the onboard command group * update HISTORY.rst * fix style issue * fix linting issues * added empty vhd and nfd processors; moved client factory from old repo * Add template parser interface (BaseParser) * Add parameters to BaseParser * Make defaults_path optional in base_parser * defaults_path defaults to None in base_parser * add helm chart parser class and common exceptions * add missing doc strings * Change parser class to input template class * added all input config params + comments; added common arm template config; added instructions for completing publish/delete * add typing to base processor and add copyright statement to new files * fixed formatting on input configs * fixed nsd input config + tidied other input config files * added comments for images; general formatting * rename template parsers to input templates and add helm chart processor * add __init__.py to folders * fixed imports + added innit in folders * fixed imports + added defaults for input params * Not quite complete first pass at definition folder handlers * added generate config log; fixed input config objects * rename InputTemplate to InputArtifacts * tidied write config + added read config; deleted json file checked in by mistake * added overwrite validation to generate config; added j2 templates to common * added validation to base config and nsd * Add UTs and fix up definition folder code * added validation to nsd + cnf; added default_file_name property; added tests for generate config; added example nsd-input.jsonc for testing * add vhd processor * remove leftover file from merge * added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params * added optional output file name param to generate config; removed misc print statements * Fix up some type hinting * added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep * push latest changes * Initial ArmInputTemplate class * Outline ArmBuildProcessor hierarchy * Update VNF tests to keep built output for comparison * WIP, for sharing with Jacob * Incremental update * added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params * added optional output file name param to generate config; removed misc print statements * Approximately complete ARM processor. No tests, no testing yet. * Fix artifact store reference * WIP * fixed cnf bicep template * fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case * fixed cnfdef bicep; return correct type in cnf handler * Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder * renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working * added logic to cnf handler for manifest and definition supporting files + config mappings * fixed cnf params files; added render params for vnf * fix HelmChartInput issues * Merge arm-processor code into Jacob's branch (#127) * Initial ArmInputTemplate class * Outline ArmBuildProcessor hierarchy * Update VNF tests to keep built output for comparison * Initial ArmInputTemplate class * Outline ArmBuildProcessor hierarchy * Update VNF tests to keep built output for comparison * WIP, for sharing with Jacob * Incremental update * Approximately complete ARM processor. No tests, no testing yet. * Fix artifact store reference --------- Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> * added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd * fix majority of linting issues * renamed write on base handler to render; added build logic for manifest and artifact list for vnf * added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep * fixed cnf bicep template * Initial ArmInputTemplate class * complete helm chart processor * almost finished NFD processor * WIP, for sharing with Jacob * Incremental update * added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params * added optional output file name param to generate config; removed misc print statements * Approximately complete ARM processor. No tests, no testing yet. * WIP * fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case * fixed cnfdef bicep; return correct type in cnf handler * Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder * renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working * added logic to cnf handler for manifest and definition supporting files + config mappings * fixed cnf params files; added render params for vnf * added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd * renamed write on base handler to render; added build logic for manifest and artifact list for vnf * fix up code * fix base input * run isort * implemented vnf build + removed test code; fixed arm processor * fixed templates * removed multiple vhds from vnf + fixed bicep; made path to mappings optional for cnf * renamed write file to render contents * fixed helm chart processor + small tidy up * commit for build handover; not working + handover notes sent separately * run isort * fix issues in input classes * add generic schema and value mapping generation and fix arm and vhd processors * push changes I forgot to push before xmas * checked out fixed build processors and inputs from jdarby/add-build-processors * fix values and schema generation * add RET generation for ARM templates * fix linting issues * made input file name constant + renamed inputs to match changes * checked out build processor changes * Moved _render_deployment_params_schema and _build_deploy_params_schema to nfd base handler; added default_config logic to cnf handler;created deploymentParamers for cnf + vnf * changed j2 template for vhd image name * HELM CHART PROCESSOR CHANGES! fixed registry value path and depends on profile in cnf template * fixed nsd artifact manifest, changed artifact type in all processers (PROCESSOR CHANGE) * implemented nsd build get artifact manifest and get artifact list * added base bicep to all handlers; edited j2 templates to note that base resources should be deployed first on publish; made all build use constants * added to_dict for localfileACRartifact * added source namespcae and source reigstry to artifact + in helm processor callinh remote arc artifacts * finish nfd processor * add copyright statement to helm processor file * linting * created common params folder (making new jsonbuilder), adding get params config to base', renamed get_config to get_input_config; aded logging to vnf; chnged custom.py publish inputs * partially ready commit; added cmd context to base + added some logic to nsds * added all parameters constant * tidied vnf handler; added common params logic to cnf * markups inc tidy up * renamed snake_case param * add nsd output config * taken some input and build processor changes from jdarby/build_processor branch * Add logging, improve doc string and fix mypy errors for inputs * fix 2 bugs introduced by merging buildprocessors and adding json deploy params * fixed how aosm client is passed to build; properly get nfdv object with api client * refactored vnf handler to create processors upfront * refactor cnf to instantiate processors upfront * Improve docstrings, add logging and fix linting issues for build processors * nsd temp commit * fix missed conflict * fix linting and formatting issue * rename + add constants + add new definition tempalte * simplified render bicep definiton contents + removes all constants for filenames from nf handlers and templates * build nsd bicep core working * fixed nsd bicep * Fix two minor CLI bugs (#129) * Fix bug where the build directory wasn't deleted * Fix bug where the json wasn't output correctly * fixed incorrect merge fix --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> * small fixes; removed location from nfd input, removed comma from vnd def template, fixed artifact version in nsds, fixed vhd validation * made cgs name constant to fix error in mappings * added store type to all the templates * tidy up + small markups * removed version state from the definition biceps * fixed bug where schemas and defaults were copied incorrectly * made artifact paths absolute in all cases except nsd nfs, where paht is relative to nsd output folder * added missing comments; renamed common params + edited its format; changed custom to take path * added existing back to all templates + descriptors of resources * Merge publish into temp wheel branch (#131) * Move artifact create from dict logic to instantiating code * Create CommandContext class to hold Azure clients and CLI options, and pass instance through to the delete and publish commands. * Mostly implemented and working publish. Lots of debug code still present. * Publish done and mostly tested. * Add some more logging. Remove help text referring to --order-params. * Update src/aosm/azext_aosm/common/artifact.py Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * fix merge conflicts --------- Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * Minor input improvements (#132) * fixed common params files for all * remove prints; remove sourcelocaldockerimage + add correct validation; edit input.json comments * fixed extract tar error * only create oras_client once - temp solution --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> * edited input file (#133) Co-authored-by: Jordan <jordan.layton@metaswitch.com> * fixed getparamsconfig for vnf/nsd; added fixes for building correct python wheel * Achurchard/publish (#130) Markups from publish PR --------- Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: jordlay <72226943+jordlay@users.noreply.github.com> * bug fixes for chart with subcharts * added ruamel * Use safe yaml loading * Move code out of import list * Fix errant config line bug * fixed deployparams error (#134) Co-authored-by: Jordan <jordan.layton@metaswitch.com> * fixed yaml load error * disabled multi anchor yaml warning for values.yaml * delete accidental deployparams file --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Chaos Chhapi <chaos.chhapi@metaswitch.com> Co-authored-by: Jacob <53872686+jddarby@users.noreply.github.com> Co-authored-by: Jacob Darby <jdarby@microsoft.com> Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: patrykkulik-microsoft <116072282+patrykkulik-microsoft@users.noreply.github.com> Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com>
patrykkulik-microsoft
added a commit
that referenced
this pull request
Jan 23, 2024
* added empty template parser * Initial scaffolding of some new classes * added utils, added base builder + reader + their children, added nfd and vhd processors * moved utils to common folder + added local file builder * Add some basic implementations * add new commands and restructure command groups * better naming for command groups * revert addition on the onboard command group * update HISTORY.rst * fix style issue * fix linting issues * added empty vhd and nfd processors; moved client factory from old repo * Add template parser interface (BaseParser) * Add parameters to BaseParser * Make defaults_path optional in base_parser * defaults_path defaults to None in base_parser * add helm chart parser class and common exceptions * add missing doc strings * Change parser class to input template class * added all input config params + comments; added common arm template config; added instructions for completing publish/delete * add typing to base processor and add copyright statement to new files * fixed formatting on input configs * fixed nsd input config + tidied other input config files * added comments for images; general formatting * rename template parsers to input templates and add helm chart processor * add __init__.py to folders * fixed imports + added innit in folders * fixed imports + added defaults for input params * Not quite complete first pass at definition folder handlers * added generate config log; fixed input config objects * rename InputTemplate to InputArtifacts * tidied write config + added read config; deleted json file checked in by mistake * added overwrite validation to generate config; added j2 templates to common * added validation to base config and nsd * Add UTs and fix up definition folder code * added validation to nsd + cnf; added default_file_name property; added tests for generate config; added example nsd-input.jsonc for testing * add vhd processor * remove leftover file from merge * added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params * added optional output file name param to generate config; removed misc print statements * Fix up some type hinting * added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep * push latest changes * Initial ArmInputTemplate class * Outline ArmBuildProcessor hierarchy * Update VNF tests to keep built output for comparison * WIP, for sharing with Jacob * Incremental update * added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params * added optional output file name param to generate config; removed misc print statements * Approximately complete ARM processor. No tests, no testing yet. * Fix artifact store reference * WIP * fixed cnf bicep template * fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case * fixed cnfdef bicep; return correct type in cnf handler * Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder * renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working * added logic to cnf handler for manifest and definition supporting files + config mappings * fixed cnf params files; added render params for vnf * fix HelmChartInput issues * Merge arm-processor code into Jacob's branch (#127) * Initial ArmInputTemplate class * Outline ArmBuildProcessor hierarchy * Update VNF tests to keep built output for comparison * Initial ArmInputTemplate class * Outline ArmBuildProcessor hierarchy * Update VNF tests to keep built output for comparison * WIP, for sharing with Jacob * Incremental update * Approximately complete ARM processor. No tests, no testing yet. * Fix artifact store reference --------- Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> * added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd * fix majority of linting issues * renamed write on base handler to render; added build logic for manifest and artifact list for vnf * added write manifest bicep and nf application to base handler; fixed helm config to produce correct bicep * fixed cnf bicep template * Initial ArmInputTemplate class * complete helm chart processor * almost finished NFD processor * WIP, for sharing with Jacob * Incremental update * added vnf config validation; improved comment spacing on input.json; fixed depends on format; changes type for optional params * added optional output file name param to generate config; removed misc print statements * Approximately complete ARM processor. No tests, no testing yet. * WIP * fixed multiple helm/vhd/arm error; working vnfartifactmanifet; attempted vnfdefintiion with testing code still in; updated write bicep functions to take 2 arguments for vnf case * fixed cnfdef bicep; return correct type in cnf handler * Added outputfolder name to base handler; reorganised templates folder structure; added constants for cnf j2 templates; implemented build (tested with mocks) for cnf handler; implemented localfileACR artifact to_dict function; updated write for artifact_builder.py; added overwriting folder logic to defintion folder builder * renamed write manifest; updated nf templates to deploy initial resources; vnf templates working; vnf build commands working * added logic to cnf handler for manifest and definition supporting files + config mappings * fixed cnf params files; added render params for vnf * added build deploy params schema to base handler; added functionality for deployParams to be added as supporting file for cnd * renamed write on base handler to render; added build logic for manifest and artifact list for vnf * fix up code * fix base input * run isort * implemented vnf build + removed test code; fixed arm processor * fixed templates * removed multiple vhds from vnf + fixed bicep; made path to mappings optional for cnf * renamed write file to render contents * fixed helm chart processor + small tidy up * commit for build handover; not working + handover notes sent separately * run isort * fix issues in input classes * add generic schema and value mapping generation and fix arm and vhd processors * push changes I forgot to push before xmas * checked out fixed build processors and inputs from jdarby/add-build-processors * fix values and schema generation * add RET generation for ARM templates * fix linting issues * made input file name constant + renamed inputs to match changes * checked out build processor changes * Moved _render_deployment_params_schema and _build_deploy_params_schema to nfd base handler; added default_config logic to cnf handler;created deploymentParamers for cnf + vnf * changed j2 template for vhd image name * HELM CHART PROCESSOR CHANGES! fixed registry value path and depends on profile in cnf template * fixed nsd artifact manifest, changed artifact type in all processers (PROCESSOR CHANGE) * implemented nsd build get artifact manifest and get artifact list * added base bicep to all handlers; edited j2 templates to note that base resources should be deployed first on publish; made all build use constants * added to_dict for localfileACRartifact * added source namespcae and source reigstry to artifact + in helm processor callinh remote arc artifacts * finish nfd processor * add copyright statement to helm processor file * linting * created common params folder (making new jsonbuilder), adding get params config to base', renamed get_config to get_input_config; aded logging to vnf; chnged custom.py publish inputs * partially ready commit; added cmd context to base + added some logic to nsds * added all parameters constant * tidied vnf handler; added common params logic to cnf * markups inc tidy up * renamed snake_case param * add nsd output config * taken some input and build processor changes from jdarby/build_processor branch * Add logging, improve doc string and fix mypy errors for inputs * fix 2 bugs introduced by merging buildprocessors and adding json deploy params * fixed how aosm client is passed to build; properly get nfdv object with api client * refactored vnf handler to create processors upfront * refactor cnf to instantiate processors upfront * Improve docstrings, add logging and fix linting issues for build processors * nsd temp commit * fix missed conflict * fix linting and formatting issue * rename + add constants + add new definition tempalte * simplified render bicep definiton contents + removes all constants for filenames from nf handlers and templates * build nsd bicep core working * fixed nsd bicep * Fix two minor CLI bugs (#129) * Fix bug where the build directory wasn't deleted * Fix bug where the json wasn't output correctly * fixed incorrect merge fix --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> * small fixes; removed location from nfd input, removed comma from vnd def template, fixed artifact version in nsds, fixed vhd validation * made cgs name constant to fix error in mappings * added store type to all the templates * tidy up + small markups * removed version state from the definition biceps * fixed bug where schemas and defaults were copied incorrectly * made artifact paths absolute in all cases except nsd nfs, where paht is relative to nsd output folder * added missing comments; renamed common params + edited its format; changed custom to take path * added existing back to all templates + descriptors of resources * First stab at doing helm template - check the changes here * Merge publish into temp wheel branch (#131) * Move artifact create from dict logic to instantiating code * Create CommandContext class to hold Azure clients and CLI options, and pass instance through to the delete and publish commands. * Mostly implemented and working publish. Lots of debug code still present. * Publish done and mostly tested. * Add some more logging. Remove help text referring to --order-params. * Update src/aosm/azext_aosm/common/artifact.py Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * fix merge conflicts --------- Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * Minor input improvements (#132) * fixed common params files for all * remove prints; remove sourcelocaldockerimage + add correct validation; edit input.json comments * fixed extract tar error * only create oras_client once - temp solution --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> * edited input file (#133) Co-authored-by: Jordan <jordan.layton@metaswitch.com> * Add skip step * fixed getparamsconfig for vnf/nsd; added fixes for building correct python wheel * Achurchard/publish (#130) Markups from publish PR --------- Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: jordlay <72226943+jordlay@users.noreply.github.com> * bug fixes for chart with subcharts * added ruamel * Use safe yaml loading * Move code out of import list * Fix errant config line bug * fixed deployparams error (#134) Co-authored-by: Jordan <jordan.layton@metaswitch.com> * fixed yaml load error * Change "path_to_mappings" to "default_values" * Add unit tests * remove the merge mistake * Delete a file --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Chaos Chhapi <chaos.chhapi@metaswitch.com> Co-authored-by: Jacob <53872686+jddarby@users.noreply.github.com> Co-authored-by: Jacob Darby <jdarby@microsoft.com> Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: jordlay <72226943+jordlay@users.noreply.github.com> Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This checklist is used to make sure that common guidelines for a pull request are followed.
Related command
General Guidelines
azdev style <YOUR_EXT>locally? (pip install azdevrequired)python scripts/ci/test_index.py -qlocally? (pip install wheel==0.30.0required)For new extensions:
About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update
src/index.jsonautomatically.You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify
src/index.json.