[ENHANCEMENT] Gradle build clean up

[andy-k-improving]

Is your feature request related to a problem?
Throughout the codebase across different sub-module there are multiple to-do in pace for Gradle file, also certain block of code can be refactor to enhance the readability.
Hence this is a place-holder issue to track all the enhancement being made against the Gradle build, in order to make the build script more concise.
Given the fact that this is refactor change, NO feature should be introduced or dropped.

What solution would you like?

• Fix todo
  • // TODO: dependency to spark should be eliminated
  • // TODO: dependency to datasources should be eliminated
  • // TODO: need to fix all java doc format
  • // TODO: fix compiler warnings
• Rewrite return provider(new Callable<RegularFile>() { wrapper to enhance readability. CI: Eliminate redundant file wrapper  #3275
• Centralise dep version to parent Gradle file. Build: Centralise dependencies version - Pt1 #3294
• Cache prometheus binary download Build: Cache Prometheus binary  #3261
• Read dep versions from OpenSearch build plugin
• Gradle version catalog enablement.

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

[acarbonetto]

@andy-k-improving can you add a link to the TODO that you wanted to fix?

[andy-k-improving]

@andy-k-improving can you add a link to the TODO that you wanted to fix?

Updated the list for the specific to-do planned to fix.

[andy-k-improving]

Added two more points which aim centralise dependencies || read directly from OpenSearch build Gradle plugin, along with the Gradle version catalogs enablement.