Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v3.2.1 #729

Merged
merged 19 commits into from
May 27, 2024
Merged

Release v3.2.1 #729

merged 19 commits into from
May 27, 2024

Commits on May 16, 2024

  1. Fix Jinja2 template rendering with autoescape enabled (awslabs#690)

    * add autoescape true
    
    * Update initial_commit.py with space
    
    * Update initial_commit.py
    
    Add autoescape=True to avoid XSS
    sujay0412 authored and sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    d56d1dd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a88a970 View commit details
    Browse the repository at this point in the history
  3. Fixing codebuild stage naming bug (awslabs#628)

    Co-authored-by: Christoffer Pozeus <christoffer.pozeus.2@consultant.volvo.com>
    Co-authored-by: Stewart Wallace <stewwall@amazon.com>
    3 people authored and sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    288628a View commit details
    Browse the repository at this point in the history
  4. Add delete default VPC in management account support (awslabs#699)

    **Why?**
    
    To allow the deletion of the Default VPC if one were to manage the management
    account via ADF's Account Management (adf-accounts).
    
    **What?**
    
    Added the missing permissions to delete and describe the default VPCs in the
    management account.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    245b69a View commit details
    Browse the repository at this point in the history
  5. Fix tagging of Cross Account Access role in the management account (a…

    …wslabs#700)
    
    **Why?**
    
    When ADF needs to tag or untag the cross-account access role, it fails to do
    so.
    
    **What?**
    
    * Granting the required permissions to tag and untag the roles.
    * Removal of the invalid `s3:listObjects` permission, the permissions required
      to list objects in a bucket are managed via the `s3:listBucket` permission.
      Which is already present.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    8bebc9b View commit details
    Browse the repository at this point in the history
  6. Fix CloudFormation cross-region changeset approval (awslabs#701)

    **Why?**
    
    With the CloudFormation deployment provider: It would fail to create the
    pipeline if one were to set the `change_set_approval` property to `True`,
    while the pipeline lives in another region than where it is asked to deploy to.
    
    **What?**
    
    * The approval action should be performed in the region that hosts the
      pipeline. Not the target region.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    10e476d View commit details
    Browse the repository at this point in the history
  7. New install process (awslabs#677)

    * New install process
    
    **Why?**
    
    With the upcoming release of ADF, CDK v2 support is added.
    The dependencies that go with it, unfortunately increased the deployment size
    beyond the limit that is supported by the Serverless Application Repository.
    
    Hence a new installation mechanism had to be developed.
    
    **What?**
    
    This change proposes to use make as the deployment mechanism.
    I chose make, as this eases support queries on deployments of old version and
    future versions.
    
    For example, by default, the `make` command will install the latest
    `aws-sam-cli` and `yq` dependencies. However, if these appear to be
    incompatible, we can instruct to use a specific version of the make install
    script:
    
    ```bash
    make update_makefile
    
    make UPDATE_VERSION=make/2.0 update_makefile
    ```
    
    Similarly, we can instruct using the AWS SAM CLI used at the time the ADF
    version was developed by running:
    
    ```bash
    make clean deps docker version_number git_ignore sam_build post_build
    ```
    
    In case someone runs into an issue while building or deploying ADF, they can
    generate a report of their environment with:
    
    ```bash
    make version_report
    ```
    
    This generates a report of all the versions of the build environment.
    
    _Long story short:_
    The proposed Makefile enables various paths to resolve and investigate issues.
    
    **Changes**
    
    * Updated the Makefile to act as the new deployment mechanism.
    * Moved tox related tasks to test and lint the code to Makefile.tox.
      As these need to run in the virtual environment managed by Tox, not the
      one managed by the new Makefile.
    * Updated the docs to support ADF administrators updating and installing
      using the new Makefile.
    * Tested with v3.2.0 deployments and the upcoming release of ADF.
    * Added support to generate version numbers on customized ADF installations.
    * Updated the contribution docs to clarify how to deploy and test.
    * Added support to update the Makefile in the future, such that a deployment
      issue in a specific ADF version can be fixed by updating the Makefile.
    * Updated GitHub workflows accordingly, plus added new tests to ensure
      the dependencies installed by ADF are compatible with each other.
    * Added a new step in the GitHub workflow to test a full build of ADF.
    * Updated the Bug Issue template to include the version report and to allow
      the reporter to indicate that they are working on a fix already.
    
    * Add macOS support to generate a version report
    
    * Update contribution guide to include upstream tags fetch step
    
    * Fix contributing guide typo
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    f39a6e1 View commit details
    Browse the repository at this point in the history
  8. Install: Add checks to ensure installer dependencies are available (a…

    …wslabs#702)
    
    **Why?**
    
    When running the make / install process of ADF, we need to ensure it will not
    continue if some of the tools are not installed yet.
    
    For example, in the past, if you did not install jq, it would still continue
    with the build process. Resulting in a broken template at deployment time.
    
    As reported in awslabs#696, the Python version in the environment was set to v3.8,
    which resulted in a broken installation process too.
    
    **What?**
    
    * Updated the documentation to indicate that `jq` needs to be available too.
    * Added checks in the build process to exit and warn the user about missing
      dependencies.
    * Updated the version number of the Makefile.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    0c2d137 View commit details
    Browse the repository at this point in the history
  9. Fix clean bootstrap of the deployment account (awslabs#703)

    **Why?**
    
    When you attempt to install ADF the first time, it will bootstrap the
    deployment account via the account bootstrap state machine.
    This, however, happens just before executing the bootstrap pipeline the first
    time. Resulting in missing parameters in the deployment account at the time the
    `regional` stack is deployed.
    
    This stack requires the `extensions/terraform/enabled` parameter.
    If that is not there yet, which is the case in the situation above, it will
    fail to bootstrap the account. Hereby the bootstrap state machine and pipeline
    both fail initially.
    
    **What?**
    
    When the bootstrap state machine determines the event details, it should try
    to retrieve the parameter in the management account to see if Terraform is
    enabled or not. If this is not configured yet, it should default to False.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    d0dffc8 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    e2374f7 View commit details
    Browse the repository at this point in the history
  11. Fix missing deployment_account_id and initial deployment global IAM b…

    …ootstrap
    
    **Why?**
    
    Issues: awslabs#659 and awslabs#594.
    
    When installing ADF the first time, the global IAM bootstrap stack that gets
    deployed is sourced from the `adf-bootstrap/global-iam.yml`.
    
    The reason for this behaviour is the absence of the `global-iam.yml` file
    in the deployment OU bootstrap folder
    (`adf-bootstrap/deployment/global-iam.yml`).
    
    It iterates to the parent directory until it finds a `global-iam.yml` to
    deploy. Hence, when the `adf-bootstrap/global-iam.yml` gets deployed in the
    deployment account, it was looking for the `deployment_account_id` SSM
    parameter. That did not get deployed in the deployment account.
    
    **What?**
    
    * Add the creation of the `deployment_account_id` in the deployment account,
      so if the global IAM bootstrap stack failed to deploy before, it will work
      in the next release. This would be the case if the previous deployment failed
      but the same `aws-deployment-framework-bootstrap` repository is used in the
      upgrade.
    * When installing the first time, it creates the bootstrap repository.
      At the time of creation, it will copy the
      `adf-bootstrap/deployment/example-global-iam.yml` to
      `adf-bootstrap/deployment/global-iam.yml`.
      The same logic as how ADF creates the initial `adf-bootstrap/global-iam.yml`.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    6478778 View commit details
    Browse the repository at this point in the history
  12. Ensure tox fails at first pytest failure

    **Why?**
    
    At the moment, pytest failures were ignored due to a change in the Makefile
    used to execute tests. The ADF CI GitHub Workflow would result in a
    success, even when a test case failed.
    
    **What?**
    
    Fixed by exiting on the first failure using Makefile foreach instead.
    sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    8c0bcd5 View commit details
    Browse the repository at this point in the history
  13. Bump jinja2 from 3.1.3 to 3.1.4 in /src/lambda_codebase/initial_commit (

    awslabs#721)
    
    * Bump jinja2 from 3.1.3 to 3.1.4 in /src/lambda_codebase/initial_commit
    
    Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
    - [Release notes](https://github.com/pallets/jinja/releases)
    - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
    - [Commits](pallets/jinja@3.1.3...3.1.4)
    
    ---
    updated-dependencies:
    - dependency-name: jinja2
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    
    * Move PR 720 into 721
    
    **Why?**
    
    Both requirements need to be changed in synchrony.
    Otherwise the pipeline will fail. Hence updating this PR to include the changes from awslabs#720.
    
    ---------
    
    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: Simon Kok <sbkok@users.noreply.github.com>
    dependabot[bot] and sbkok committed May 16, 2024
    Configuration menu
    Copy the full SHA
    5e5dc61 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. Install: Add version checks and pre-deploy warnings (awslabs#726)

    **Why?**
    
    When you clone the ADF Github repository, it will not fetch the git tags.
    The installation docs describe that you should run `git fetch --tags` to
    retrieve those. However, if someone forgets to run that command, you run
    into a weird error that does not describe what is going wrong in a resolvable
    manner:
    
    > Error: Failed to create changeset for the stack: aws-deployment-framework,
    > Parameter validation failed:
    > Invalid length for parameter Tags[0].Value, value: 0, valid min length: 1
    
    Additionally, if you installed ADF with the updated installer before. Then
    forget to fetch the latest git tags when you are about to deploy a new version
    of ADF, it will return a version number that is based on the old version
    instead of the new one. Making it hard to determine what version of ADF was
    installed. Long story short, the version in the `src/template.yml` should never
    be newer than the version determined via git tags.
    
    **What?**
    
    Added check to verify that the version number can be determined correctly.
    
    * If it did not return the version number, but an empty string instead:
      * it will test whether there is a .git folder.
        * If not, the user should clone a repository instead.
        * If the .git directory exists, it will instruct the user to fetch the
          git tags.
    * If a version number is returned, it will check whether the version number
      matches with the tag.
      * If it does, this means that it is building a released version of ADF.
      * If it does not, the user might not have switched to the right version tag
        yet. Hence, we warn the user to proceed with caution.
    
    Additionally, this change will add a warning when ADF gets deployed that the
    specific version and changes should be reviewed and tested in a non-production
    environment.
    
    Lastly, a check is added to ensure that the ADF version of the
    `src/template.yml` file should not be newer than the version resolved via git
    tags. If this is the case, it should instruct the user to fetch the latest tags
    from the ADF repository.
    sbkok committed May 17, 2024
    Configuration menu
    Copy the full SHA
    3220ce9 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. Install: Add uncommitted changes check (awslabs#733)

    **Why?**
    
    In case one were to checkout a tagged release, then make changes, ADF will
    receive the tagged release number. Instead of tracking the change that was
    made in the release number as well.
    
    By preventing these uncommitted changes, it makes it easier to detect what
    code is running in a specific version of ADF. So in case someone were to report
    an issue we can assume the number indicates which version of the code failed.
    
    **What?**
    
    * Added check for uncommitted changes, these include files that are tracked
      by the git repository.
    * Added check for newly created files, these would not be tracked by git yet.
    sbkok committed May 23, 2024
    Configuration menu
    Copy the full SHA
    d8005c0 View commit details
    Browse the repository at this point in the history
  2. Fix parameter lookups to be compatible with v3.2

    **Why?**
    
    In the next v4 release, the parameters automatically add the required prefixes.
    With v3.2, this was not the case yet. Since we are porting changes back into
    a new release of v3.2 we need to make sure they are compatible.
    
    **What?**
    
    Fixing the lookup and put path references as required to install and bootstrap
    ADF.
    sbkok committed May 23, 2024
    Configuration menu
    Copy the full SHA
    0d3a7b2 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Fix account management lambdas in v3.2

    **Why?**
    
    Issue: awslabs#728
    
    With the latest Python runtime update provided by AWS Lambda, the boto3
    dependencies were upgraded. However, with the account management Lambda
    functions, the `aws-xray-sdk` requires botocore v1.29.54. This version
    is incompatible with the latest boto3 package that was added to the Lambda
    function runtime. Hence, it returned an error as described in 728.
    
    **What?**
    
    To ensure compatible versions are installed, ADF's lambda functions should
    set the specific version of boto3 in its requirements.txt file if they rely on
    that.
    sbkok committed May 24, 2024
    Configuration menu
    Copy the full SHA
    a82658f View commit details
    Browse the repository at this point in the history
  2. Fix missing tag role permission

    **Why?**
    
    When deploying ADF as a fresh installation, it requires `iam:TagRole`
    permission to create the cross-account management role. This was not required
    in the past, so existing IAM roles were not impacted.
    
    **What?**
    
    Added the required permission and limited it to the two roles it needs to
    create. Additionally, removal of `iam:PassRole` as that was not used anywhere
    in the management account.
    sbkok committed May 24, 2024
    Configuration menu
    Copy the full SHA
    c278910 View commit details
    Browse the repository at this point in the history
  3. v3.2.1

    The upcoming v4 release will introduce breaking changes. As always, it is
    recommended to thoroughly review and test the upgrade procedure in a
    non-production environment.
    
    Unfortunately, a few issues popped up in ADF v3.2.0 that block a clean
    installation in a new environment. Making it harder to test the upgrade
    process. This release resolves those issues and includes an updated installer
    for ADF to simplify installation.
    
    We hope this shortens the time required to prepare for the v4 upgrade.
    
    ---
    
    **Fixes 🐞**
    
    * Fix management account config alias through ADF account management (awslabs#596) by @sbkok.
    * Fix CodeBuild stage naming bug (awslabs#628) by @pozeus.
    * Fix Jinja2 template rendering with autoescape enabled (awslabs#690) by @sujay0412.
    * Fix missing deployment_account_id and initial deployment global IAM bootstrap
      (awslabs#686) by @sbkok, resolves awslabs#594 and awslabs#659.
    * Fix permissions to enable delete default VPC in management account (awslabs#699) by @sbkok.
    * Fix tagging of Cross Account Access role in the management account (awslabs#700) by @sbkok.
    * Fix CloudFormation cross-region changeset approval (awslabs#701) by @sbkok.
    * Fix clean bootstrap of the deployment account (awslabs#703) by @sbkok, resolves awslabs#696.
    * Bump Jinja2 from 3.1.3 to 3.1.4 (awslabs#720 and awslabs#721) by @dependabot.
    * Fix account management lambdas in v3.2 (awslabs#729) by @sbkok.
    * Fix management account missing required IAM Tag Role permission in v3.2
      (awslabs#729) by @sbkok.
    
    ---
    
    **Installation enhancements 🚀**
    
    This release is the first release with the new installation process baked in.
    Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md)
    how to install ADF. In case you are upgrading, please follow [the admin guide
    on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions)
    instead.
    
    Changes baked into this release to support the new installation process:
    
    * New install process (awslabs#677) by @sbkok.
    * Ensure tox fails at first pytest failure (awslabs#686) by @sbkok.
    * Install: Add checks to ensure installer dependencies are available (awslabs#702) by
      @sbkok.
    * Install: Add version checks and pre-deploy warnings (awslabs#726) by @sbkok.
    * Install: Add uncommitted changes check (awslabs#733)
    sbkok committed May 24, 2024
    Configuration menu
    Copy the full SHA
    a70bb12 View commit details
    Browse the repository at this point in the history