Support for Multiple Python Versions in VDK #1739
Description
What is the feature request? What problem does it solve?
At the moment, if a user deploys a data job with python 3.8 in a k8s cluster, and decides they want to deploy another job with python 3.10, the only way to do it, is to re-configure and re-deploy the Control Service. This, however, creates issues because the moment the first job (the one that uses python 3.8) is re-deployed, it would automatically be switched to python 3.10 with unforeseeable consequences.
A new mechanism is needed to make it possible to dynamically configure what python version a data job uses when it is deployed in a k8s cluster.
Suggested solution
A possible solution is to introduce a new property in the data job's config.ini file, indicating what python version is supposed to be used, when the data job is deployed. This way, when the vdk deploy command is invoked, the Control Service would deploy the data job with the specified python version.
For this to be possible, the Control Service would need to store configuration what python versions are supported, and to dynamically pass the user-defined python version to the data job builder.
Additional context
A VEP with more details will be created for this enhancement.