Updated to version 3.3.3, see CHANGELOG.md for details.

aws-solutions · Jan 30, 2024 · e903bdf · e903bdf
1 parent 60cb447
commit e903bdf
Show file tree

Hide file tree

Showing 93 changed files with 3,117 additions and 1,997 deletions.
diff --git a/.idea/.gitignore b/.idea/.gitignore
diff --git a/.idea/cloud-migration-factory-on-aws.iml b/.idea/cloud-migration-factory-on-aws.iml
diff --git a/.scannerwork/report-task.txt b/.scannerwork/report-task.txt
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.3.3] - 2024-01-30
+### Security
+- JWT VALIDATION: Replaced of python-jose with PyJWT to resolve CVE https://nvd.nist.gov/vuln/detail/CVE-2024-23342 which is caused by a dependency on the vulnerable python-ecdsa module. All JWT verification is now performed using PyJWT.
+- MGN TARGET IAM ROLE: Restricted inline policy for MGN Role in target accounts to not allow iam:PassRole and sts:AssumeRole on all resources. This is now restricted to PassRole for the MGN service to EC2 service only.
+### Changed
+- MIGRATION TRACKER: Updated deployment to use Glue v4.0 as Glue v2.0 has now been removed from support. Existing deployments will need to be updated before January, 31 2024 with this version, or a manual update of glue jobs is required.
+- MGN AGENT INSTALL: Removed default creation of IAMUser from target account CFT and updated installation scripts to use temporary credentials by default, with the option of allowing a secret to be used to supply IAMUser credentials if required.
+- CODE QUALITY: Increased unit test coverage, and refactored code base.
+- AWS LAMBDA: Runtimes moved to Python version 3.11 for all functions.
+### Fixed
+- SUBMIT JOB UI: Resolved issue with changing scripts after inputting attribute values, this caused any previous values to be sent to the newly created job.
+### Added
+- AUTOMATION: Tools API ID is now exposed to automation scripts, allowing calls to the tools api from a script.
 ## [3.3.2] - 2023-12-18
 ### Changed
 - MIGRATION TRACKER: By default migration tracker now contains database and wave data in general view, this allows for more detailed customer dashboards to be created in QuickSight.

diff --git a/NOTICE.txt b/NOTICE.txt
@@ -12,66 +12,67 @@ THIRD PARTY COMPONENTS
 **********************
 This software includes third party software subject to the following copyrights:
 
-AWS SDK/boto3 under the Apache License 2.0
-aws-amplify under the Apache License 2.0
+@aws-amplify/api under the Apache License 2.0
+@aws-amplify/auth under the Apache License 2.0
+@aws-amplify/core under the Apache License 2.0
 @awsui/components-react under the Apache License 2.0
 @awsui/global-styles under the Apache License 2.0
 @awsui/jest-preset under the Apache License 2.0
-requests under the Apache License 2.0
-simplejson under the Massachusetts Institute of Technology (MIT) license
-uuid under the Massachusetts Institute of Technology (MIT) license
-aws-amplify under the Apache License 2.0
-react under the Massachusetts Institute of Technology (MIT) license
-react-dom under the Massachusetts Institute of Technology (MIT) license
-react-router-dom under the Massachusetts Institute of Technology (MIT) license
-react-scripts under the Massachusetts Institute of Technology (MIT) license
-paramiko under the GNU Lesser General Public License v2.1
-troposphere is licensed under the BSD 2-Clause license.
-ace is licensed under the BSD 2-Clause license.
-moto under the Apache License 2.0
-coverage under the Apache License 2.0
-python-jose under the Massachusetts Institute of Technology (MIT) license
-jquery under the Massachusetts Institute of Technology (MIT) license
-jquery-csv under the Massachusetts Institute of Technology (MIT) license
-uuid under the Massachusetts Institute of Technology (MIT) license
-xlsx under the Apache License 2.0
 @babel/preset-env under the Massachusetts Institute of Technology (MIT) license
 @babel/preset-react under the Massachusetts Institute of Technology (MIT) license
 @testing-library/react under the Massachusetts Institute of Technology (MIT) license
+ace is licensed under the BSD 2-Clause license.
+ace-builds under the BSD-3-Clause license
+AWS SDK/boto3 under the Apache License 2.0
+aws-amplify under the Apache License 2.0
+aws-amplify under the Apache License 2.0
+aws-lambda-powertools under the Massachusetts Institute of Technology (MIT) license
 babel-core under the Massachusetts Institute of Technology (MIT) license
 babel-jest under the Massachusetts Institute of Technology (MIT) license
-jest under the Massachusetts Institute of Technology (MIT) license
-jest-sonar-reporter under the Massachusetts Institute of Technology (MIT) license
-jest-environment-jsdom under the Massachusetts Institute of Technology (MIT) license
-jsdoc under the Apache License 2.0
-react-test-renderer under the Massachusetts Institute of Technology (MIT) license
-source-map-explorer under the Apache License 2.0
-ts-jest under the Massachusetts Institute of Technology (MIT) license
-PyNaCl under the Apache License 2.0
-Jinja2 under the BSD-3-Clause license
-MarkupSafe under the BSD-3-Clause license
-Werkzeug under the BSD License
-aws-lambda-powertools under the Massachusetts Institute of Technology (MIT) license
 bcrypt under the Apache License 2.0
 botocore under the Apache License 2.0
 cffi under the Massachusetts Institute of Technology (MIT) license
+cfn-flip under the Apache License 2.0
 click under the BSD License
+coverage under the Apache License 2.0
 cryptography under the Apache License 2.0
 docker under the Apache License 2.0
 ecdsa under the Massachusetts Institute of Technology (MIT) license
 freezegun under the Apache License 2.0
+jest under the Massachusetts Institute of Technology (MIT) license
+jest-environment-jsdom under the Massachusetts Institute of Technology (MIT) license
+jest-sonar-reporter under the Massachusetts Institute of Technology (MIT) license
+Jinja2 under the BSD-3-Clause license
 jmespath under the Massachusetts Institute of Technology (MIT) license
+jquery under the Massachusetts Institute of Technology (MIT) license
+jquery-csv under the Massachusetts Institute of Technology (MIT) license
+jsdoc under the Apache License 2.0
+MarkupSafe under the BSD-3-Clause license
+moto under the Apache License 2.0
+paramiko under the GNU Lesser General Public License v2.1
 pyasn1 under the BSD-2-Clause License
 pycparser under the BSD License
+PyJWT under the Massachusetts Institute of Technology (MIT) license
+PyNaCl under the Apache License 2.0
 python-dateutil under the Apache License 2.0 and BSD License
+python-jose under the Massachusetts Institute of Technology (MIT) license
+react under the Massachusetts Institute of Technology (MIT) license
+react-dom under the Massachusetts Institute of Technology (MIT) license
+react-router-dom under the Massachusetts Institute of Technology (MIT) license
+react-scripts under the Massachusetts Institute of Technology (MIT) license
+react-test-renderer under the Massachusetts Institute of Technology (MIT) license
+requests under the Apache License 2.0
 responses under the Apache License 2.0
 rsa under the Apache License 2.0
 s3transfer under the Apache License 2.0
+simplejson under the Massachusetts Institute of Technology (MIT) license
+source-map-explorer under the Apache License 2.0
+troposphere is licensed under the BSD 2-Clause license.
+ts-jest under the Massachusetts Institute of Technology (MIT) license
 typing_extensions under Python Software Foundation License Version 2
+uuid under the Massachusetts Institute of Technology (MIT) license
+uuid under the Massachusetts Institute of Technology (MIT) license
 websocket-client under the Apache License 2.0
+Werkzeug under the BSD License
+xlsx under the Apache License 2.0
 xmltodict under the Massachusetts Institute of Technology (MIT) license
-cfn-flip under the Apache License 2.0
-@aws-amplify/api under the Apache License 2.0
-@aws-amplify/auth under the Apache License 2.0
-@aws-amplify/core under the Apache License 2.0
-ace-builds under the BSD-3-Clause license
diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution-automation.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution-automation.template
@@ -73,7 +73,7 @@ Parameters:
 
   LambdaRuntimePython:
     Type: String
-    Default: python3.10
+    Default: python3.11
 
   IsDeploymentPrivate:
     Type: String
@@ -750,6 +750,7 @@ Resources:
           ssm_bucket: !Ref SSMBucket
           ssm_automation_document: !Ref RunCMFAutomationPackageSSMDocument
           mf_userapi: !Ref UserAPI
+          mf_toolsapi: !Ref ToolsAPI
           mf_loginapi: !Ref LoginAPI
           mf_vpce_id: !Ref VPCEID
           mf_cognitouserpoolid: !Ref CognitoUserPoolId
@@ -1270,6 +1271,10 @@ Resources:
                     Selector: $.Payload.mf_endpoints.UserApi
                     Type: String
 
+                  - Name: mf_endpoints_ToolsApi
+                    Selector: $.Payload.mf_endpoints.ToolsApi
+                    Type: String
+
                   - Name: mf_endpoints_LoginApi
                     Selector: $.Payload.mf_endpoints.LoginApi
                     Type: String
@@ -1343,7 +1348,7 @@ Resources:
                       - $file = 'c:\migrations\scripts\downloads\{{package_download.script_key}}.zip'
                       - "Try {Read-S3Object -BucketName '{{package_download.bucket_name}}' -File $file -Key 'scripts/{{package_download.script_key}}.zip' -Version '{{package_download.script_version}}' | Out-Null} Catch {$_ | Out-File C:\\migrations\\Scripts\\downloads\\logs.txt; Write-Host '[{{ package_download.ssm_id }}]' Error downloading script from S3 bucket ('{{package_download.bucket_name}}') to automation server: $_; Write-Host '[{{ package_download.ssm_id }}] JOB_FAILED'; exit 255}"
                       - $dt = (Get-Date).ToString('MM-dd-yyyy-hh.mm.sstt')
-                      - $json = @{VpceId='{{package_download.mf_endpoints_VpceId}}'; LoginApi='{{package_download.mf_endpoints_LoginApi}}'; UserApi='{{package_download.mf_endpoints_UserApi}}'; UserPoolId='{{package_download.mf_endpoints_UserPoolId}}'; UserPoolClientId='{{package_download.mf_endpoints_UserPoolClientId}}'; Region='{{package_download.mf_endpoints_Region}}'}
+                      - $json = @{VpceId='{{package_download.mf_endpoints_VpceId}}'; LoginApi='{{package_download.mf_endpoints_LoginApi}}'; UserApi='{{package_download.mf_endpoints_UserApi}}'; ToolsApi='{{package_download.mf_endpoints_ToolsApi}}'; UserPoolId='{{package_download.mf_endpoints_UserPoolId}}'; UserPoolClientId='{{package_download.mf_endpoints_UserPoolClientId}}'; Region='{{package_download.mf_endpoints_Region}}'}
                       - $target_folder = 'c:\migrations\scripts\history\{{package_download.script_key}}-' + $dt
                       - $target_folder | Out-File -FilePath 'c:\migrations\scripts\downloads\script_path.txt' -NoNewline
                       - "Try{Expand-Archive -LiteralPath $file -DestinationPath $target_folder | Out-Null} Catch {Write-Host '[{{ package_download.ssm_id }}] Error extracting script archive to automation server: ' $_; Write-Host '[{{ package_download.ssm_id }}] JOB_FAILED'; exit 255}"

diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution-credentialmanager.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution-credentialmanager.template
@@ -51,7 +51,7 @@ Parameters:
     Type: String
   LambdaRuntimePython:
     Type: String
-    Default: python3.10
+    Default: python3.11
 
 Mappings:
   Solution:

diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution-mgn.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution-mgn.template
@@ -91,7 +91,7 @@ Parameters:
 
   LambdaRuntimePython:
     Type: String
-    Default: python3.10
+    Default: python3.11
 
 Mappings:
   Solution:

diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution-replatform.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution-replatform.template
@@ -103,7 +103,7 @@ Parameters:
 
   LambdaRuntimePython:
     Type: String
-    Default: python3.10
+    Default: python3.11
 
 Mappings:
   Solution:

diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution-target-account.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution-target-account.template
@@ -47,49 +47,6 @@ Conditions:
   DeployRehostMGNRole: !Equals [!Ref RehostMGN, true]
 
 Resources:
-  MGNAgentInstallUser:
-    Condition: DeployRehostMGNRole
-    Type: AWS::IAM::User
-    Properties:
-      UserName: !Sub "MGNAgentInstallUser-${AWS::AccountId}"
-      Policies:
-        -
-          PolicyName: LambdaRolePolicy
-          PolicyDocument:
-            Version: 2012-10-17
-            Statement:
-              - Effect: Allow
-                Action:
-                  - 'mgn:SendAgentMetricsForMgn'
-                  - 'mgn:SendAgentLogsForMgn'
-                  - 'mgn:SendClientLogsForMgn'
-                Resource: '*'
-              - Effect: Allow
-                Action:
-                  - 'mgn:RegisterAgentForMgn'
-                  - 'mgn:UpdateAgentSourcePropertiesForMgn'
-                  - 'mgn:UpdateAgentReplicationInfoForMgn'
-                  - 'mgn:UpdateAgentConversionInfoForMgn'
-                  - 'mgn:GetAgentInstallationAssetsForMgn'
-                  - 'mgn:GetAgentCommandForMgn'
-                  - 'mgn:GetAgentConfirmedResumeInfoForMgn'
-                  - 'mgn:GetAgentRuntimeConfigurationForMgn'
-                  - 'mgn:UpdateAgentBacklogForMgn'
-                  - 'mgn:GetAgentReplicationInfoForMgn'
-                Resource: '*'
-              - Effect: Allow
-                Action: 'mgn:TagResource'
-                Resource: 'arn:aws:mgn:*:*:source-server/*'
-    Metadata:
-        cfn_nag:
-          rules_to_suppress:
-            - id: F2000
-              reason: "user is for agent install only, does not need to be assigned to a group"
-            - id: F10
-              reason: "user is for agent install only, does not need to be assigned to a group"
-            - id: W11
-              reason: "The resources ARN is unknown, because it is based on user's input"
-
   CMFMGNAutomationRole:
     Condition: DeployRehostMGNRole
     Type: 'AWS::IAM::Role'
@@ -142,12 +99,6 @@ Resources:
           PolicyDocument:
             Version: '2012-10-17'
             Statement:
-              -
-                Effect: Allow
-                Action:
-                  - 'iam:PassRole'
-                  - 'sts:AssumeRole'
-                Resource: "*"
               - Effect: Allow
                 Action:
                   - 'mgn:ChangeServerLifeCycleState'
@@ -209,9 +160,7 @@ Resources:
                 Resource: '*'
               - Effect: Allow
                 Action: 'iam:PassRole'
-                Resource:
-                  - >-
-                    arn:aws:iam::*:role/service-role/AWSApplicationMigrationConversionServerRole
+                Resource: '*'
                 Condition:
                   StringEquals:
                     'iam:PassedToService': ec2.amazonaws.com
@@ -450,12 +399,12 @@ Resources:
           PolicyDocument:
             Version: '2012-10-17'
             Statement:
-              -
-                Effect: Allow
-                Action:
-                  - 'iam:PassRole'
-                  - 'sts:AssumeRole'
+              - Effect: Allow
+                Action: 'iam:PassRole'
                 Resource: "*"
+                Condition:
+                  StringEquals:
+                    'iam:PassedToService': ec2.amazonaws.com
               -
                 Effect: Allow
                 Action:
@@ -572,23 +521,3 @@ Resources:
             reason: "Replacement of this resource is not required, and explicit name of this resource is easy for user to identify the table"
           - id: W76
             reason: "The policy is required for managing target EC2 instances"
-
-
-  MGNAccessKeyId:
-    Condition: DeployRehostMGNRole
-    Type: 'AWS::IAM::AccessKey'
-    Properties:
-      UserName: !Ref MGNAgentInstallUser
-  MGNUserSecrets:
-    Condition: DeployRehostMGNRole
-    Type: AWS::SecretsManager::Secret
-    Properties: 
-      Description: String
-      Name: MGNAgentInstallUser
-      SecretString: !Sub '{"AccessKeyId":"${MGNAccessKeyId}", "SecretAccessKey":"${MGNAccessKeyId.SecretAccessKey}"}'
-    Metadata:
-      cfn_nag:
-        rules_to_suppress:
-          - id: W77
-            reason: "cross-account sharing KMS key is not required"
-
diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution-tracker.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution-tracker.template
@@ -35,7 +35,7 @@ Parameters:
 
   LambdaRuntimePython:
     Type: String
-    Default: python3.10
+    Default: python3.11
 
 Mappings:
   Solution:
@@ -619,7 +619,7 @@ Resources:
       Command:
         Name: glueetl
         ScriptLocation: !Sub "s3://${MigrationTrackerBucket}/GlueScript/Migration_Tracker_App_Extract_Script.py"
-      GlueVersion: '2.0'
+      GlueVersion: '4.0'
       MaxRetries: 2
       DefaultArguments:
         "--job-bookmark-option": "job-bookmark-disable"
@@ -638,7 +638,7 @@ Resources:
       Command:
         Name: glueetl
         ScriptLocation: !Sub "s3://${MigrationTrackerBucket}/GlueScript/Migration_Tracker_Server_Extract_Script.py"
-      GlueVersion: '2.0'
+      GlueVersion: '4.0'
       MaxRetries: 2
       DefaultArguments:
         "--job-bookmark-option": "job-bookmark-disable"
@@ -657,7 +657,7 @@ Resources:
       Command:
         Name: glueetl
         ScriptLocation: !Sub "s3://${MigrationTrackerBucket}/GlueScript/Migration_Tracker_Wave_Extract_Script.py"
-      GlueVersion: '2.0'
+      GlueVersion: '4.0'
       MaxRetries: 2
       DefaultArguments:
         "--job-bookmark-option": "job-bookmark-disable"
@@ -676,7 +676,7 @@ Resources:
       Command:
         Name: glueetl
         ScriptLocation: !Sub "s3://${MigrationTrackerBucket}/GlueScript/Migration_Tracker_Database_Extract_Script.py"
-      GlueVersion: '2.0'
+      GlueVersion: '4.0'
       MaxRetries: 2
       DefaultArguments:
         "--job-bookmark-option": "job-bookmark-disable"

diff --git a/deployment/CFN-templates/aws-cloud-migration-factory-solution.template b/deployment/CFN-templates/aws-cloud-migration-factory-solution.template
@@ -139,7 +139,7 @@ Mappings:
     AppRegistry:
       SolutionName: 'Cloud Migration Factory on AWS'
     LambdaRuntime:
-      Python: python3.10
+      Python: python3.11
 
 Conditions:
   DeployTracker: !Equals [!Ref Tracker, true]