Version 1.4.0

Features:

• Add Level 0 (public), Level 1 (public/private), and Level 2 (private) options for both net new and existing networks for AWS
• Introduce ‘purge’ keyword for Definitions to signal that an AWS VPC and all child objects, and all unattached EBS volumes in the name_prefix, should be removed from the given infra_region
• Remove AWS DynamoDB references

Updates:

• Improve AMI owners selection to be configurable and add defaults for aws-marketplace option for Centos8 AMI
• Update to delete roles to the authz teardown
• Add 'df' key to sequence role to handle deploying to defaults when only df is requested
• Switch DF request to use Environment CRN instead of name for better determinism when user supplies unexpected values
• Use env.tunnel instead of globals.env_network_type and fix syntactic issues
• Exclude CDP CIDRs from inbound security group
• Use existing properties and introduce public endpoint access for AWS
• Add public endpoint gateway support for experiences
• Split default extra network ports into ssh, cloud, and base port groups, with appropriate dot notation keys under infra.vpc
• Add switch ‘infra.vpc.enable_ssh’, defaulted to True, to allow port 22/ssh access
• Switch including cm ports 7180/7183 by default when ‘mgmt’ or ‘clusters’ is defined for base deployment
• Switch including https/443 on when cdp public platform is deployed
• Move AWS policy documents path to common role and propagate to plat and data roles
• Add warning to Create AWS Buckets function for common error where user attempts to create a bucket that is already owned elsewhere
• Introduce wait and retry controls for handling Azure eventual consistency when negotiating between Ansible Controller, Azure Control Plane, and CDP Control Plane
• Add retry to address intermittent failure in datahub_template_info listing of datahub templates in CDP 7.2.10
• Add explicit test for Azure Storage Account being unavailable for use in this deployment
• Move default Azure minimal policy json from private gist to cloudera-labs snippets
• Correct Azure App name where sometimes referred to with http:// header and sometimes not, resulting in idempotent failures
• Introduce more robust validations that Service principals and other objects created by az CLI are populated as expected
• Ensure that Azure objects are consistently bound to the Azure namespace created from the name_prefix
• Provide more user friendly errors when Azure App and Service Principal creation doesn’t go as planned
• Add tunnel and public endpoint control support to Azure Environment creation in line with AWS offering
• Fix ML submission preparation to include nfs information following existing combination patterns
• Swap order of Runtime initialization tasks to handle provider-specific tasks before general tasks, to allow Azure-specific values to be populated
• Explicitly derive Azure NFS Mount information in Runtime deployment from earlier Infrastructure deployment steps
• Allow override of namespace unique component under globals.labels.namespace_suffix
• Improve EKS cluster discovery and removal by pinning to target region
• Discover Autoscaling groups which may have one or more vpc_zone_identifiers
• Update env setup to include passing freeipa instance count and include defaults
• Remove initialize tasks in sudoers role

Fixes:

• Fix reference to unused DWX variable in DE placeholders for new feature
• Fix filtering datahub definitions to include version and provider, not just version
• Avoid duplicate route table creation of one exists and fix error handling in NAT gateway failures
• Update to use CLI for VPC details due to boto3 bug
• Handle edge cases for preparation of Netapp NFS Mount when deploying ML automatically on Azure
• Fix purge teardown edge case where child services are not deleted if at least one child service is not present in the Definition
• Update ML Workspace setup to use definition of 1 instance group
• Correct AWS CDP IDBroker role's policy definitions
• Remove the NAT gateways before the ENI cleanup is called, when calling purge with an L1 deployment or when NAT gateways are deployed with attached ENI.