Add selectable distribution support for cloudera.cluster #51
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Enable by setting parcel_distro in definition to [el7, el8, or bionic] per dynamic OS options in cloudera.exe.infrastructure.vars Change default dynamic inventory selection strings from 'centos7' to match the distribution identity strings of el7 etc. - user can update defaults to a different el7 distro etc. in cloudera.exe.infrastructure.vars Add Ubuntu 18.04 'bionic' as option to dynamic inventory Determine preferred parcel distribution in cloudera-deploy init Add uniqueness to generated dynamic inventory VM name to reflect selection of distribution in case multiple clusters are deployed in the same account Move dynamic inventory OS selection to globals, update appropriate reference docs to reflect change Add filtering by distro to download mirror support, and ensure that manifest is still always collected Enforce no_log always when working with Paywall credentials Increase initial paywall download timeout to 7200s due to present CDN speed issues when deploying on EC2 outside of us-east-1 Modify ansible.builtin.package lock_timeout to only be used on RedHat, as it is not a Debian option Pass selected parcel distribution to repo analysis during intial deployment when target cluster OS is not yet determined by deployment Move cloudera.cluster plays which require knowledge of the cluster distribution to run on the cloudera_manager host instead of the Ansible controller so the correct distribution actions are applied Fix extract_products_from_manifests filter in cloudera.cluster to correctly reference self and process os_distribution value Fix import ordering in cloudera.cluster filters.py to not break under recent versions of Python3 Add distribution specific tasks for cloudera.cluster.deployment.repometa so it can identify the cluster distribution using the strings recognised by Cloudera Manager deployment Force refresh of apt package cache on Debian distributions during OS prereqs setup in cloudera.cluster as the package cache in the image is sometimes missing packages Signed-off-by: Daniel Chaffelson <chaffelson@gmail.com>
wmudge
approved these changes
Oct 29, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable by setting parcel_distro in definition to [el7, el8, or bionic] per dynamic OS options in cloudera.exe.infrastructure.vars
Change default dynamic inventory selection strings from 'centos7' to match the distribution identity strings of el7 etc. - user can update defaults to a different el7 distro etc. in cloudera.exe.infrastructure.vars
Add Ubuntu 18.04 'bionic' as option to dynamic inventory
Determine preferred parcel distribution in cloudera-deploy init
Add uniqueness to generated dynamic inventory VM name to reflect selection of distribution in case multiple clusters are deployed in the same account
Move dynamic inventory OS selection to globals, update appropriate reference docs to reflect change
Add filtering by distro to download mirror support, and ensure that manifest is still always collected
Enforce no_log always when working with Paywall credentials
Increase initial paywall download timeout to 7200s due to present CDN speed issues when deploying on EC2 outside of us-east-1
Modify ansible.builtin.package lock_timeout to only be used on RedHat, as it is not a Debian option
Pass selected parcel distribution to repo analysis during intial deployment when target cluster OS is not yet determined by deployment
Move cloudera.cluster plays which require knowledge of the cluster distribution to run on the cloudera_manager host instead of the Ansible controller so the correct distribution actions are applied
Fix extract_products_from_manifests filter in cloudera.cluster to correctly reference self and process os_distribution value
Fix import ordering in cloudera.cluster filters.py to not break under recent versions of Python3
Add distribution specific tasks for cloudera.cluster.deployment.repometa so it can identify the cluster distribution using the strings recognised by Cloudera Manager deployment
Force refresh of apt package cache on Debian distributions during OS prereqs setup in cloudera.cluster as the package cache in the image is sometimes missing packages
Signed-off-by: Daniel Chaffelson chaffelson@gmail.com