Allow testcases to run on s390x architecture

[rposts]

Description

This PR attempts to address following issues observed on s390x arch - In particular:
o Calicoctl component replaces “amd64” with ARCH to make is applicable to “s390x” – I have used ARCH here making it consistent with earlier uses of ARCHs in the Makefile
o Felix testcase removes hardcoded “amd64” declaration with Go runtime built-in
o Confd corrects ARCH linking for s390x
o Kube-controller Dockerfile is changed to produce working version of the image (also brings it closer to amd64)
o Node/calico_test Dockerfile is changed to produce working version of the image (also brings it closer to amd64)

Related issues/PRs

This issue #7654 should clarify the status of calico/bpftool:v5.3-s390x image
This PR projectcalico/toolchain#442 enhances calico/go-build image for s390x which will be needed to build and run testcases on s390x arch

Todos

• Tests
• Documentation
• Release note

Release Note

TBD

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

• docs-pr-required: This change requires a change to the documentation that has not been completed yet.
• docs-completed: This change has all necessary documentation completed.
• docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

• release-note-required: This PR has user-facing changes. Most PRs should have this label.
• release-note-not-required: This PR has no user-facing changes.

Other optional labels:

• cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
• needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

[rposts]

@marvin-tigera Please let me know how we can proceed with this PR - Thanks!

[caseydavenport]

/sem-approve

[caseydavenport]

Sorry for the delay! Running CI, and LGTM

[marvin-tigera]

Removing "merge-when-ready" label due to new commits

[caseydavenport]

@rposts sorry for yet another delay - I was coming to merge this now, but it looks like there are some conflicts that need to be resolved first.

I think this one has been here for a while, so let me know if you'd rather I resolve them myself and submit a new PR on top of your commits.

[rposts]

Hi @caseydavenport - sure, if it makes things easier for you then go ahead with resolving them yourself. I am here for any s390x problems encountered. Also note that there are couple of inter-reflated Related issues/PRs in the section above that will be required to make it more complete.

Thanks for all your help!

[caseydavenport]

Merged it here: #8073

I haven't been able to actually run it on s390x, so if you find any issues with it please open any follow on PRs we need!