Incorporate API compatibility checks into CI

[joelanford]

When reviewing PRs and deciding on a release version, it would be helpful to know if branches of controller-runtime contain any breaking changes:

• For PRs: the diff between master and the PR branch
• For releases: the diff between the previous semver release tag and master

For the recent bumps for Kubernetes 1.15 and 1.16, we used a tool I wrote (go-apidiff) based on Go's experimental apidiff package that checks:

• The backwards compatibility of controller-runtime itself.
• The backwards compatibility of controller-runtime's dependency changes.

In it's current state, the tool is still a proof-of-concept and probably needs improvements. I haven't had a ton of time to keep up with this, so I'm opening this up to the community to ask for help improving it, finding/fixing bugs, and to potentially incorporate it into controller-runtime's CI to give reviewers a signal of breaking public API changes.

[DirectXMan12]

/kind feature
/priority important-soon
/help

[joelanford]

From @vincepri:

We're adding go-apidiff prow job in cluster api, we should do the same in controller-runtime at some point kubernetes-sigs/cluster-api#2923

We can use this as an example for incorporating into controller-runtime. Thanks @vincepri!

[alexeldeib]

I was looking at this yesterday, missed the open issue. If no one has taken it I can.

/assign

[alexeldeib]

/remove-help

[joelanford]

This is closed by #905, #910, and kubernetes/test-infra#17240.