From 649b29ab8f154006be6cc4fd2c53f242eec0a319 Mon Sep 17 00:00:00 2001 From: Jonathan Innis Date: Mon, 17 Jan 2022 22:56:19 -0800 Subject: [PATCH] [k8s-configuration] Release v1.4.1 with fixed help text (#4328) * Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update CODEOWNERS * Update azure-pipelines.yml * Create pull.yml * Update pull.yml * Update pull.yml * Update pipelines file * Update k8s-configuration name * Update test script params * Update pipeline file * Remove codeowners * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update custom pipelines files * Add publish step to k8s-configuration * Update pipeline to publish extension * Update public extension pipeline * Change condition variable * Update pipeline naming * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model * Stage Public Version of k8s-extension 0.2.0 for official release (#15) * Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update public extension pipeline * Change condition variable * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model Co-authored-by: action@github.com * Remove custom pipelines file * Update extension description, remove private const * Update pipeline file * Disable check ref docs * Disable refs docs * Update to include better create warning logs and remove update context (#20) * Update to include better create warning logs and remove update context * Remove help text for update * Fix spelling error * Update message * Fix k8s-extension conflict with private version * Fix style errors * Fix filename * add customization for microsoft.azureml.kubernetes (#23) * add customization for microsoft.azureml.kubernetes * Update release history Co-authored-by: Yue Yu Co-authored-by: jonathan-innis * Add E2E Testing from Separate branch into internal code (#26) * Add internal e2e testing * Change to testing folder * Inference CLI validation for Scoring FE (#24) * cli validation starter * added the call to the fe validation function * nodeport validation not required * test fix Co-authored-by: Jonathan Innis * legal warning added (#27) * Remove deprecated method logger.warn * Update k8s-custom-pipelines.yml for Azure Pipelines * Update k8s-custom-pipelines.yml for Azure Pipelines * Add Azure Defender to E2E testing (#28) * Add azure defender testing to e2e * Remove the debug flag * Add configuration testing * Fix pipeline failures * Make test script more intuitive * Remove parameter from testing * Add some debug * Fix wrong location for k8s config whl * Fix pip install upgrade issue * Fix pip install upgrade issue * Add Check for Provider Registration and Refactor (#19) * Add check for provider registration and refactor * Fix bug in checking registration * Add license header to utils * Update private key check and error messaging * Update based on refactoring * Fix failing tests * Add provider registration check * Create a test for uppercase url, address comments * Add blank line to fix style check * Testing increase to ubuntu-latest * Update k8s-configuration Models to Track2 (#63) * Update models to track2 * Increase k8s-configuration version number * Update kind version * Change error to warning because of DSA failure * Upgrade helm operator chart version (#75) * Pin helm version * Bump version * Migrate pipeline (#90) * Disable updates on configuration tests (#89) * Release k8s-configuration v1.2.0 for Flux v2 Public Preview (#86) * Scaffold out the k8s-config package * Base implementation of CLI commands * Add create scenario and cleanup in consts * Add help text to commands, params * Add other clients to client factory * Automatically installing the flux extension * Move flux and extension into modules * Updated the versioned sdks * Push working command for testing * Update to multi api versioned sdk * Support other extension methods * Fix nullity check * Add source control provider * Add scc commands * Add defer logic for create with cache * Use default extension with identity * Fix identity creation * Add kustomization caching * Add formatters * Add scc provider * Add help text for k8s-config fluxv1 * Add help text for extension * Allow force delete of extension and fluxconfiguration * Add location to the extension model * Update with latest from k8s-extension * Add k8s-config testing * Add license header * Fix all style issues * Update codeowners file * Validate data before checking cluster compliance * No kustomizations warning * Fix identity issue in 2020-07-01 * Fix k8s regex * Fix configuration name regex validation * Fix name length validation * Adding some validation warnings * Add protected settings to request * Exclude private test path * Add suspend functionality * Add correct values to build Kustomization * Add no_wait * Fix style issues * Use base64 encoded httpsUser * Fix formatting error and base64 encoding error * Fix style issues * Fix force * Updated help text * Style fixes * Increase namespace maximum len * Add managed cluster support to k8s-config * Custom confirmation when prune is enabled * Add flux commands to existing k8s-configuration * Remove extension provider from CLI * Fix style issues * Override extension variables * Strip newlines from known_hosts file * Update help text and validators * Strip newlines from known hosts * Add provisioning state check for flux extension * Pin helm version * Remove validation from create command * Add patch support with new SDK * Add implementation for CRUD of source and kustomization * Fix errors on patch * Fix some bugs in patching properties * Add fixes for patch in k8s-configuraiton * Change duration formatting in table output * Add validation and conversion for durations * Bump verison and fix typo * Fix bug with dependencies * Fix linter and style issues * Fix delete prune check * Add flux testing * Create separate jobs for scenarios * Update error text * Fix filepath suggestion from CLI team * Fix unneeded file edit * Add a Deployed Object List to the Flux CLI (#91) * Enable a deployed object list on the CLI * Show detail when extension install fails * Bump version * Update deployed object format (#93) * Fix help text for consistency * Enable Bucket Support in the CLI (#92) * Update vendored_sdks * Enable source kind generation factories and use kwargs to pass to provider * Add better validation logic to source generator * Move away from classes in the provider directory * Fix style issues using black auto-formatter * Fix linter failures * Update identity with api version and rp in same function * Bucket Testing for E2E Testing (#96) * Bucket testing * Add switching kind test to update CLI * Bump version * Fix help text and parameter naming for bucket (#100) * Fix default help text for parameters (#101) * Remove unneeded files * Edit history with breaking change Co-authored-by: action@github.com Co-authored-by: yuyue9284 <15863499+yuyue9284@users.noreply.github.com> Co-authored-by: Yue Yu Co-authored-by: Lia Kazakova <58274127+liakaz@users.noreply.github.com> --- src/k8s-configuration/HISTORY.rst | 7 +++ .../azext_k8s_configuration/_help.py | 7 ++- .../azext_k8s_configuration/_params.py | 12 ++--- .../azext_k8s_configuration/consts.py | 6 +-- .../providers/FluxConfigurationProvider.py | 52 ++++++++++--------- src/k8s-configuration/setup.py | 42 ++++++++------- 6 files changed, 71 insertions(+), 55 deletions(-) diff --git a/src/k8s-configuration/HISTORY.rst b/src/k8s-configuration/HISTORY.rst index 5af7a8a6f3d..4c8d242171b 100644 --- a/src/k8s-configuration/HISTORY.rst +++ b/src/k8s-configuration/HISTORY.rst @@ -3,6 +3,13 @@ Release History =============== +1.4.1 +++++++++++++++++++ +* [BREAKING CHANGE] `--access-key` changed to `--bucket-access-key` +* [BREAKING CHANGE] `--secret-key` changed to `--bucket-secret-key` +* [BREAKING CHANGE] `--insecure` changed to `--bucket-insecure` +* Fix help text for bucket parameters + 1.4.0 ++++++++++++++++++ * Add `--kind bucket` for creation of S3 bucket as source for fluxConfigurations diff --git a/src/k8s-configuration/azext_k8s_configuration/_help.py b/src/k8s-configuration/azext_k8s_configuration/_help.py index 31e14bffda4..8b6c6dc04f8 100644 --- a/src/k8s-configuration/azext_k8s_configuration/_help.py +++ b/src/k8s-configuration/azext_k8s_configuration/_help.py @@ -93,7 +93,7 @@ --name myconfig --scope cluster --namespace my-namespace \\ --kind bucket --url https://bucket-provider.minio.io \\ --bucket-name my-bucket --kustomization name=my-kustomization \\ - --access-key my-access-key --secret-key my-secret-key + --bucket-access-key my-access-key --bucket-secret-key my-secret-key """ helps[ @@ -108,6 +108,11 @@ --cluster-name mycluster --cluster-type connectedClusters --name myconfig \\ --url https://github.com/Azure/arc-k8s-demo --branch main \\ --kustomization name=my-kustomization path=./my/new-path + - name: Update a Flux v2 Kubernetse configuration with Bucket Source Kind to connect insecurely + text: |- + az k8s-configuration flux update --resource-group my-resource-group \\ + --cluster-name mycluster --cluster-type connectedClusters --name myconfig \\ + --bucket-insecure """ helps[ diff --git a/src/k8s-configuration/azext_k8s_configuration/_params.py b/src/k8s-configuration/azext_k8s_configuration/_params.py index bed4994c216..d763f1c3ee2 100644 --- a/src/k8s-configuration/azext_k8s_configuration/_params.py +++ b/src/k8s-configuration/azext_k8s_configuration/_params.py @@ -137,21 +137,21 @@ def load_arguments(self, _): help="File path to known_hosts contents containing public SSH keys required to access private Git instances", ) c.argument( - "access_key", + "bucket_access_key", arg_group="Bucket Auth", - help="Access Key Id used to authenticate with the bucket url", + help="Access Key ID used to authenticate with the bucket", ) c.argument( - "secret_key", + "bucket_secret_key", arg_group="Bucket Auth", - help="Secret Key used to authenticate with the bucket url", + help="Secret Key used to authenticate with the bucket", ) c.argument( "bucket_name", - help="Name of the S3 bucket at the given url to connect to", + help="Name of the S3 bucket to sync", ) c.argument( - "insecure", + "bucket_insecure", arg_type=get_three_state_flag(), help="Communicate with a bucket without TLS", ) diff --git a/src/k8s-configuration/azext_k8s_configuration/consts.py b/src/k8s-configuration/azext_k8s_configuration/consts.py index 7db604b9c33..192f62d6335 100644 --- a/src/k8s-configuration/azext_k8s_configuration/consts.py +++ b/src/k8s-configuration/azext_k8s_configuration/consts.py @@ -209,9 +209,9 @@ "bucket_name", "sync_interval", "timeout", - "access_key", - "secret_key", - "insecure", + "bucket_access_key", + "bucket_secret_key", + "bucket_insecure", "local_auth_ref", } diff --git a/src/k8s-configuration/azext_k8s_configuration/providers/FluxConfigurationProvider.py b/src/k8s-configuration/azext_k8s_configuration/providers/FluxConfigurationProvider.py index 1faaea717e1..7b86396501b 100644 --- a/src/k8s-configuration/azext_k8s_configuration/providers/FluxConfigurationProvider.py +++ b/src/k8s-configuration/azext_k8s_configuration/providers/FluxConfigurationProvider.py @@ -144,9 +144,9 @@ def create_config( https_ca_cert_file=None, known_hosts=None, known_hosts_file=None, - access_key=None, - secret_key=None, - insecure=False, + bucket_access_key=None, + bucket_secret_key=None, + bucket_insecure=False, suspend=False, kustomization=None, no_wait=False, @@ -175,9 +175,9 @@ def create_config( https_ca_cert_file=https_ca_cert_file, known_hosts=known_hosts, known_hosts_file=known_hosts_file, - access_key=access_key, - secret_key=secret_key, - insecure=insecure, + bucket_access_key=bucket_access_key, + bucket_secret_key=bucket_secret_key, + bucket_insecure=bucket_insecure, ) # This update func is a generated update function that modifies @@ -193,7 +193,7 @@ def create_config( # Get the protected settings and validate the private key value protected_settings = get_protected_settings( - ssh_private_key, ssh_private_key_file, https_key, secret_key + ssh_private_key, ssh_private_key_file, https_key, bucket_secret_key ) if protected_settings and consts.SSH_PRIVATE_KEY_KEY in protected_settings: validate_private_key(protected_settings["sshPrivateKey"]) @@ -256,9 +256,9 @@ def update_config( https_ca_cert_file=None, known_hosts=None, known_hosts_file=None, - access_key=None, - secret_key=None, - insecure=None, + bucket_access_key=None, + bucket_secret_key=None, + bucket_insecure=None, suspend=None, kustomization=None, no_wait=False, @@ -293,9 +293,9 @@ def update_config( https_ca_cert_file=https_ca_cert_file, known_hosts=known_hosts, known_hosts_file=known_hosts_file, - access_key=access_key, - secret_key=secret_key, - insecure=insecure, + bucket_access_key=bucket_access_key, + bucket_secret_key=bucket_secret_key, + bucket_insecure=bucket_insecure, ) # This update func is a generated update function that modifies @@ -324,7 +324,7 @@ def update_config( # Get the protected settings and validate the private key value protected_settings = get_protected_settings( - ssh_private_key, ssh_private_key_file, https_key, secret_key + ssh_private_key, ssh_private_key_file, https_key, bucket_secret_key ) if protected_settings and consts.SSH_PRIVATE_KEY_KEY in protected_settings: validate_private_key(protected_settings["sshPrivateKey"]) @@ -968,15 +968,17 @@ def __init__(self, **kwargs): self.bucket_name = kwargs.get("bucket_name") self.timeout = kwargs.get("timeout") self.sync_interval = kwargs.get("sync_interval") - self.access_key = kwargs.get("access_key") - self.secret_key = kwargs.get("secret_key") + self.bucket_access_key = kwargs.get("bucket_access_key") + self.bucket_secret_key = kwargs.get("bucket_secret_key") self.local_auth_ref = kwargs.get("local_auth_ref") - self.insecure = kwargs.get("insecure") + self.bucket_insecure = kwargs.get("bucket_insecure") def validate(self): super().validate_required_params(**self.kwargs) validate_bucket_url(self.url) - if not ((self.access_key and self.secret_key) or self.local_auth_ref): + if not ( + (self.bucket_access_key and self.bucket_secret_key) or self.local_auth_ref + ): raise RequiredArgumentMissingError( consts.REQUIRED_BUCKET_VALUES_MISSING_ERROR, consts.REQUIRED_BUCKET_VALUES_MISSING_HELP, @@ -995,9 +997,9 @@ def bucket_updater(config): bucket_name=self.bucket_name, timeout_in_seconds=parse_duration(self.timeout), sync_interval_in_seconds=parse_duration(self.sync_interval), - access_key=self.access_key, + access_key=self.bucket_access_key, local_auth_ref=self.local_auth_ref, - insecure=self.insecure, + insecure=self.bucket_insecure, ) config.source_kind = SourceKindType.BUCKET return config @@ -1018,9 +1020,9 @@ def bucket_patch_updater(config): bucket_name=self.bucket_name, timeout_in_seconds=parse_duration(self.timeout), sync_interval_in_seconds=parse_duration(self.sync_interval), - access_key=self.access_key, + access_key=self.bucket_access_key, local_auth_ref=self.local_auth_ref, - insecure=self.insecure, + insecure=self.bucket_insecure, ) if swapped_kind: self.validate() @@ -1032,7 +1034,7 @@ def bucket_patch_updater(config): def get_protected_settings( - ssh_private_key, ssh_private_key_file, https_key, secret_key + ssh_private_key, ssh_private_key_file, https_key, bucket_secret_key ): protected_settings = {} ssh_private_key_data = get_data_from_key_or_file( @@ -1046,8 +1048,8 @@ def get_protected_settings( if https_key: protected_settings[consts.HTTPS_KEY_KEY] = to_base64(https_key) - if secret_key: - protected_settings[consts.BUCKET_SECRET_KEY_KEY] = to_base64(secret_key) + if bucket_secret_key: + protected_settings[consts.BUCKET_SECRET_KEY_KEY] = to_base64(bucket_secret_key) # Return the protected settings dict if there are any values there return protected_settings if len(protected_settings) > 0 else None diff --git a/src/k8s-configuration/setup.py b/src/k8s-configuration/setup.py index 2139b4fbe1b..dfe4c09fb73 100644 --- a/src/k8s-configuration/setup.py +++ b/src/k8s-configuration/setup.py @@ -8,47 +8,49 @@ from codecs import open from setuptools import setup, find_packages + try: from azure_bdist_wheel import cmdclass except ImportError: from distutils import log as logger + logger.warn("Wheel is not available, disabling bdist_wheel hook") -VERSION = '1.4.0' +VERSION = "1.4.1" # The full list of classifiers is available at # https://pypi.python.org/pypi?%3Aaction=list_classifiers CLASSIFIERS = [ - 'Development Status :: 4 - Beta', - 'Intended Audience :: Developers', - 'Intended Audience :: System Administrators', - 'Programming Language :: Python', - 'Programming Language :: Python :: 3', - 'Programming Language :: Python :: 3.6', - 'Programming Language :: Python :: 3.7', - 'Programming Language :: Python :: 3.8', - 'License :: OSI Approved :: MIT License', + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "Intended Audience :: System Administrators", + "Programming Language :: Python", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.6", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "License :: OSI Approved :: MIT License", ] DEPENDENCIES = ["pycryptodome~=3.9.8"] -with open('README.rst', 'r', encoding='utf-8') as f: +with open("README.rst", "r", encoding="utf-8") as f: README = f.read() -with open('HISTORY.rst', 'r', encoding='utf-8') as f: +with open("HISTORY.rst", "r", encoding="utf-8") as f: HISTORY = f.read() setup( - name='k8s-configuration', + name="k8s-configuration", version=VERSION, - description='Microsoft Azure Command-Line Tools K8s-configuration Extension', + description="Microsoft Azure Command-Line Tools K8s-configuration Extension", # TODO: Update author and email, if applicable - author='Microsoft Corporation', - author_email='azpycli@microsoft.com', - url='https://github.com/Azure/azure-cli-extensions/tree/master/src/k8s-configuration', - long_description=README + '\n\n' + HISTORY, - license='MIT', + author="Microsoft Corporation", + author_email="azpycli@microsoft.com", + url="https://github.com/Azure/azure-cli-extensions/tree/master/src/k8s-configuration", + long_description=README + "\n\n" + HISTORY, + license="MIT", classifiers=CLASSIFIERS, packages=find_packages(), install_requires=DEPENDENCIES, - package_data={'azext_k8s_configuration': ['azext_metadata.json']}, + package_data={"azext_k8s_configuration": ["azext_metadata.json"]}, )