SPLAT-1160: AWS - Support Wavelength Zones with edge pool #7369

mtulio · 2023-07-25T22:48:35Z

This PR introduces the support of automating creating network resources in Wavelength Zones when the zone names are defined in the edge compute pool of install-config.yaml.

Goal: Full IPI in Wavelength Zones ( SPLAT-1160 )
Feature: OCPSTRAT-736
Required by: BYO VPC (SPLAT-1218)

Enhancement: openshift/enhancements#1510

mtulio · 2023-07-25T22:48:45Z

/test all

openshift-ci · 2023-07-25T22:48:56Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

mtulio · 2023-07-25T22:52:14Z

/test all

Local tests full IPI is working, getting CI signals

$ cat lz-p2-318/install-config.yaml-bkp 
apiVersion: v1
publish: External
baseDomain: devcluster.openshift.com
metadata:
  name: "lz-p2-318"
platform:
  aws:
    region: us-east-1
networking:
  machineNetwork:
  - cidr: 10.0.0.0/19
controlPlane:   
  hyperthreading: Enabled 
  name: master
  platform:
    aws:
      zones: [us-east-1a,us-east-1b]
compute:
- name: worker
  platform:
    aws:
      zones: [us-east-1c,us-east-1d]
- name: edge
  platform:
    aws:
      zones:
      - us-east-1-scl-1a
      - us-east-1-wl1-was-wlz-1

$ oc get machines -n openshift-machine-api
NAME                                                 PHASE     TYPE         REGION      ZONE                      AGE
lz-p2-318-z6fst-edge-us-east-1-scl-1a-t9nvg          Running   c5.2xlarge   us-east-1   us-east-1-scl-1a          48m
lz-p2-318-z6fst-edge-us-east-1-wl1-was-wlz-1-j99x9   Running   r5.2xlarge   us-east-1   us-east-1-wl1-was-wlz-1   48m
lz-p2-318-z6fst-master-0                             Running   m6i.xlarge   us-east-1   us-east-1a                54m
lz-p2-318-z6fst-master-1                             Running   m6i.xlarge   us-east-1   us-east-1b                54m
lz-p2-318-z6fst-master-2                             Running   m6i.xlarge   us-east-1   us-east-1a                54m
lz-p2-318-z6fst-worker-us-east-1c-kgs2p              Running   m6i.xlarge   us-east-1   us-east-1c                48m
lz-p2-318-z6fst-worker-us-east-1c-tcc4g              Running   m6i.xlarge   us-east-1   us-east-1c                48m
lz-p2-318-z6fst-worker-us-east-1d-5n7bd              Running   m6i.xlarge   us-east-1   us-east-1d                48m

$ oc get nodes
NAME                          STATUS   ROLES                  AGE   VERSION
ip-10-0-10-194.ec2.internal   Ready    worker                 44m   v1.27.3+e8b13aa
ip-10-0-11-216.ec2.internal   Ready    worker                 45m   v1.27.3+e8b13aa
ip-10-0-14-227.ec2.internal   Ready    worker                 46m   v1.27.3+e8b13aa
ip-10-0-2-45.ec2.internal     Ready    control-plane,master   54m   v1.27.3+e8b13aa
ip-10-0-25-67.ec2.internal    Ready    edge,worker            10m   v1.27.3+e8b13aa
ip-10-0-27-51.ec2.internal    Ready    edge,worker            35m   v1.27.3+e8b13aa
ip-10-0-3-95.ec2.internal     Ready    control-plane,master   55m   v1.27.3+e8b13aa
ip-10-0-4-232.ec2.internal    Ready    control-plane,master   55m   v1.27.3+e8b13aa

mtulio · 2023-07-26T16:38:07Z

/test tf-fmt
/test golint
/test unit
/test e2e-aws-ovn
/test e2e-aws-ovn-localzones

openshift-ci-robot · 2023-07-26T18:06:57Z

@mtulio: This pull request references SPLAT-1045 which is a valid jira issue.

In response to this:

https://issues.redhat.com/browse/SPLAT-1045

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mtulio · 2023-07-27T00:20:32Z

/test tf-fmt
/test e2e-aws-ovn-localzones

openshift-ci-robot · 2023-10-27T20:07:32Z

@mtulio: This pull request references SPLAT-1160 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

https://issues.redhat.com/browse/SPLAT-1045

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-10-27T20:10:16Z

@mtulio: This pull request references SPLAT-1160 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

Goal: Full IPI in Wavelength Zones ( SPLAT-1160 )

Feature: OCPSTRAT-736

Unblocks?: BYO VPC (SPLAT-1218)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-10-27T20:10:33Z

@mtulio: This pull request references SPLAT-1160 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

Goal: Full IPI in Wavelength Zones ( SPLAT-1160 )

Feature: OCPSTRAT-736

Required by?: BYO VPC (SPLAT-1218)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mtulio · 2023-10-27T20:17:59Z

/test e2e-aws-ovn-localzones
/test e2e-aws-ovn-shared-vpc-localzones
/test e2e-aws-ovn

mtulio · 2023-10-28T04:43:57Z

Both Local Zone jobs have perm failing for a long time for a similar reason: pod monitor tests.

The initial investigation points out that those tests are not tolerating edge nodes. I opened an investigation into it:

cc @rvanderp3 @patrickdillon

mtulio · 2023-10-30T18:33:43Z

Both Local Zone jobs have perm failing for a long time for a similar reason: pod monitor tests.

The initial investigation points out that those tests are not tolerating edge nodes. I opened an investigation into it:

https://issues.redhat.com/browse/SPLAT-1225

tolerate network disruption on edge nodes origin#28363

cc @rvanderp3 @patrickdillon

The Bug has been opened to fix the Local Zone e2e tests: https://issues.redhat.com/browse/OCPBUGS-22703

mtulio · 2023-11-03T04:04:21Z

Fix in the monitor tests was merged and perm failure must be fixed:

/test e2e-aws-ovn-localzones

The installer discover the offerings by zone from a supported set of instance types, when none of those are available in the zone it emits a warning. Edge compute pool diferentiates from regular zones when selecting the EC2 Types, it may not share the same type by edge zone, instead every zone type could have a different EC2 type, depending of the offerings in the zone. The warning for edge zone shows which zone does not have any in the supported list, to help the user to update the machine set manifest in later stage.

Adding terraform automation to create resources subnet and route table associations in AWS Wavelength zones. The AWS Wavelength Zones are identified as edge zones by installer. The Wavelength Zones does not support Nat Gateway, for that reason the terraform will create only subnet and associations to the route table from the parent region, when exists, otherwise the first private route table will be used in the association. The subnets in Wavelength Zones will be created only when the zone names are supplied in the install-config.yaml in the edge compute pool. AWS Wavelength requires an different type of gateway when ingress/egress traffic from the zone: Carrier Gateway. When installer creates the VPC, the terraform creates: - the Carrier Gateway associating to the VPC - public edge route table - public subnet in Wavelength Zone, associating to the public edge route table The installer odes not create Machine Set configuration to launch edge nodes to public subnets, but the user can do it in install time, for that reason an different feature is required to support MAPI AWS Provider, not covered and not blockes the full automation delivered by installer.

mtulio · 2023-11-22T22:00:33Z

@r4f4 Thanks for the suggestion handling the carrier gateway resource is not found in the destroy flow. Fixed.

pkg/destroy/aws/ec2helpers.go

Destroying the Carrier Gateway, handling the NotFound when the resource is not present.

r4f4

/lgtm

mtulio · 2023-11-23T05:55:53Z

/test e2e-aws-ovn-localzones

mtulio · 2023-11-23T05:58:54Z

@yunjiang29 would you mind taking a look for pre-merge tests (SPLAT-1243) ?
/assign @yunjiang29

patrickdillon · 2023-11-28T20:26:51Z

/approve

openshift-ci · 2023-11-28T20:28:06Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: patrickdillon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [patrickdillon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mtulio · 2023-11-29T02:13:31Z

/test govet
/test golint
/test gofmt
/test e2e-aws-ovn

mtulio · 2023-11-29T02:14:11Z

/test images
/test aro-unit

mtulio · 2023-11-29T03:36:03Z

/test okd-images
/test okd-scos-images
/test okd-verify-codegen

mtulio · 2023-11-29T03:37:13Z

/test okd-unit

mtulio · 2023-11-29T03:37:16Z

/test okd-unit

mtulio · 2023-11-29T03:51:27Z

/test okd-unit

openshift-ci-robot · 2023-11-29T06:41:06Z

/retest-required

Remaining retests: 0 against base HEAD 8ee36b7 and 2 for PR HEAD 8612574 in total

openshift-ci · 2023-11-29T08:27:48Z

@mtulio: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-e2e-aws-ovn	2f399f9ce19959ae9be4cdcfd51dbe6697cf2b3e	link	false	`/test okd-e2e-aws-ovn`
ci/prow/okd-scos-e2e-aws-ovn	2f399f9ce19959ae9be4cdcfd51dbe6697cf2b3e	link	false	`/test okd-scos-e2e-aws-ovn`
ci/prow/e2e-aws-ovn-shared-vpc-wavelengthzones	fd1c008400865f4041cd54db2448c0a5e11491b5	link	false	`/test e2e-aws-ovn-shared-vpc-wavelengthzones`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2023-11-29T09:02:07Z

/retest-required

Remaining retests: 0 against base HEAD cfadd7c and 1 for PR HEAD 8612574 in total

openshift-bot · 2023-11-29T23:20:02Z

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-installer-altinfra-container-v4.15.0-202311292108.p0.g153837a.assembly.stream for distgit ose-installer-altinfra.
All builds following this will include this PR.

Implement Wavelength zone support as it was done for the terraform provisioning in openshift#7369.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 25, 2023

mtulio mentioned this pull request Jul 25, 2023

SPLAT-1220: Assign Carrier IP to launch EC2 in AWS Wavelength Zones openshift/machine-api-provider-aws#78

Merged

mtulio force-pushed the edge-aws-wavelength-zone branch from 2f399f9 to 0a3e543 Compare July 26, 2023 16:36

mtulio changed the title ~~WIP|Spike: Supporting AWS Wavelength Zones in the edge compute pool~~ SPLAT-1045: spike: Supporting AWS Wavelength Zones in the edge compute pool Jul 26, 2023

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 26, 2023

mtulio force-pushed the edge-aws-wavelength-zone branch 2 times, most recently from 9f66e51 to 47ec84f Compare July 27, 2023 00:20

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 19, 2023

mtulio force-pushed the edge-aws-wavelength-zone branch from 47ec84f to aef6033 Compare October 27, 2023 19:43

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 27, 2023

mtulio changed the title ~~SPLAT-1045: spike: Supporting AWS Wavelength Zones in the edge compute pool~~ SPLAT-1160: spike: Supporting AWS Wavelength Zones in the edge compute pool Oct 27, 2023

mtulio changed the title ~~SPLAT-1160: spike: Supporting AWS Wavelength Zones in the edge compute pool~~ SPLAT-1160: Supporting AWS Wavelength Zones edge compute pool Oct 28, 2023

mtulio changed the title ~~SPLAT-1160: Supporting AWS Wavelength Zones edge compute pool~~ SPLAT-1160: Supporting AWS Wavelength Zones with edge pool Oct 28, 2023

mtulio changed the title ~~SPLAT-1160: Supporting AWS Wavelength Zones with edge pool~~ SPLAT-1160: AWS - Support Wavelength Zones with edge pool Oct 28, 2023

mtulio mentioned this pull request Oct 31, 2023

SPLAT-1218: AWS BYO VPC support for Wavelength Zones #7652

Merged

mtulio force-pushed the edge-aws-wavelength-zone branch from aef6033 to ab21b4e Compare October 31, 2023 14:44

mtulio mentioned this pull request Nov 1, 2023

SPLAT-1227: AWS Wavelength - Added jobs for BYO VPC openshift/release#45158

Merged

mtulio added 2 commits November 22, 2023 18:58

mtulio force-pushed the edge-aws-wavelength-zone branch from fd1c008 to 3594add Compare November 22, 2023 21:58

r4f4 reviewed Nov 22, 2023

View reviewed changes

pkg/destroy/aws/ec2helpers.go Outdated Show resolved Hide resolved

aws destroy: added carrier-gateway resource type

8612574

Destroying the Carrier Gateway, handling the NotFound when the resource is not present.

mtulio force-pushed the edge-aws-wavelength-zone branch from 3594add to 8612574 Compare November 22, 2023 23:05

r4f4 reviewed Nov 22, 2023

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 22, 2023

openshift-ci bot assigned yunjiang29 Nov 23, 2023

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 28, 2023

patrickdillon merged commit 153837a into openshift:master Nov 29, 2023

mtulio deleted the edge-aws-wavelength-zone branch November 29, 2023 17:59

r4f4 mentioned this pull request Nov 30, 2023

CORS-3021: infra: aws: add wavelength zone support #7786

Closed

r4f4 added a commit to r4f4/installer that referenced this pull request Dec 15, 2023

CORS-3021: infra: aws: add wavelength zone support

1709379

Implement Wavelength zone support as it was done for the terraform provisioning in openshift#7369.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SPLAT-1160: AWS - Support Wavelength Zones with edge pool #7369

SPLAT-1160: AWS - Support Wavelength Zones with edge pool #7369

mtulio commented Jul 25, 2023 •

edited

Loading

mtulio commented Jul 25, 2023

openshift-ci bot commented Jul 25, 2023

mtulio commented Jul 25, 2023 •

edited

Loading

mtulio commented Jul 26, 2023

openshift-ci-robot commented Jul 26, 2023 •

edited by openshift-ci bot

Loading

mtulio commented Jul 27, 2023

openshift-ci-robot commented Oct 27, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Oct 27, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Oct 27, 2023 •

edited by openshift-ci bot

Loading

mtulio commented Oct 27, 2023

mtulio commented Oct 28, 2023

mtulio commented Oct 30, 2023

mtulio commented Nov 3, 2023

mtulio commented Nov 22, 2023

r4f4 left a comment

mtulio commented Nov 23, 2023

mtulio commented Nov 23, 2023

patrickdillon commented Nov 28, 2023

openshift-ci bot commented Nov 28, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

openshift-ci-robot commented Nov 29, 2023

openshift-ci bot commented Nov 29, 2023 •

edited

Loading

openshift-ci-robot commented Nov 29, 2023

openshift-bot commented Nov 29, 2023

SPLAT-1160: AWS - Support Wavelength Zones with edge pool #7369

SPLAT-1160: AWS - Support Wavelength Zones with edge pool #7369

Conversation

mtulio commented Jul 25, 2023 • edited Loading

mtulio commented Jul 25, 2023

openshift-ci bot commented Jul 25, 2023

mtulio commented Jul 25, 2023 • edited Loading

mtulio commented Jul 26, 2023

openshift-ci-robot commented Jul 26, 2023 • edited by openshift-ci bot Loading

mtulio commented Jul 27, 2023

openshift-ci-robot commented Oct 27, 2023 • edited by openshift-ci bot Loading

openshift-ci-robot commented Oct 27, 2023 • edited by openshift-ci bot Loading

openshift-ci-robot commented Oct 27, 2023 • edited by openshift-ci bot Loading

mtulio commented Oct 27, 2023

mtulio commented Oct 28, 2023

mtulio commented Oct 30, 2023

mtulio commented Nov 3, 2023

mtulio commented Nov 22, 2023

r4f4 left a comment

Choose a reason for hiding this comment

mtulio commented Nov 23, 2023

mtulio commented Nov 23, 2023

patrickdillon commented Nov 28, 2023

openshift-ci bot commented Nov 28, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

mtulio commented Nov 29, 2023

openshift-ci-robot commented Nov 29, 2023

openshift-ci bot commented Nov 29, 2023 • edited Loading

openshift-ci-robot commented Nov 29, 2023

openshift-bot commented Nov 29, 2023

mtulio commented Jul 25, 2023 •

edited

Loading

mtulio commented Jul 25, 2023 •

edited

Loading

openshift-ci-robot commented Jul 26, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Oct 27, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Oct 27, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Oct 27, 2023 •

edited by openshift-ci bot

Loading

openshift-ci bot commented Nov 29, 2023 •

edited

Loading