[Backend] pH - How to update dependency?

[Bobgy]

#3965 has been blocked for nearly a month, because no one knows how to update backend dependencies properly.
We need clear documentation for instructions.

/assign @jingzhang36 @IronPan
/cc @NikeNano

[Bobgy]

TODO:

• instructions that can unblock feat(backend): workflow validation. Fixes #3526. #3965
• add documentation to backend/README.md

[jingzhang36]

First of all, I believe the way we manage BE dependencies is no different from other golang projects who use bazel. In other words, there is nothing special to our dependency management and any online tutorial on managing golang projs with bazel applies to our project (the same as it applies to other golang projects with bazel)

Second, if the dependency is broken or goes wrong, we'll have to follow the build errors and try to fix it. Personally, I don't know of any hack way that can quickly do that. But maybe @IronPan and @neuromage know of some quick fixes.

Regarding #3965, I'll try to figure out why.

[Bobgy]

If there are any existing doc that will be great, we can just put a link in the README.md

[Ark-kun]

First of all, I believe the way we manage BE dependencies is no different from other golang projects who use bazel. In other words, there is nothing special to our dependency management and any online tutorial on managing golang projs with bazel applies to our project (the same as it applies to other golang projects with bazel)

The backend uses a really outdated version of Bazel. The build files are incompatible with stable Bazel versions and the build files of new versions of the dependencies are incompatible with the old Bazel version that we use.

We might improve the developer productivity here by scaling back the usage of Bazel and going back to using Go modules.

[Bobgy]

Yes, I totally agree with @Ark-kun.

I feel the extra steps to use bazel is just unnecessary when go modules is already becoming golang standard. People contributing can possibly have good knowledge of go modules, but I don't feel enough people know how to use bazel. As Ajay mentioned, we may still keep bazel for building the proto for the moment.

[jingzhang36]

There are two issues here (let's try solving the one described in this issue's description first, without unnecessarily letting the other to be blocking).

1. One issue of the PR #3965 mentioned in the description is that the author needs to follow our instruction of updating dependencies in WORKSPACE. feat(backend): workflow validation. Fixes #3526. #3965 (comment)

2. Related but not blocking the above PR #3965 is KFP backend dependency on bazel. It has been a known issue for a long time. Several attempts had been done trying to increase the bazel version KFP backend uses. Various blockers had been ran into, e.g., some of the libs used by KFP backend require this bazel version. Hopefully, that might have been changed and we can now increase the bazel version used by KFP. An alternative is that we give up bazel, which seems ok too. I personally don't have a strong preference for this topic.

[Bobgy]

Thanks for clarification!

With https://github.com/kubeflow/pipelines/tree/master/backend#updating-build-files, I think this issue's purpose is already met. We can continue discussion of cleaning up bazel dep by upgrading/stopping to use it in #3461.