[WIP][Flyteadmin] Add variablemap in dataproxy for dataclass/pydantic

[mao3267]

Current Problem

There are two main components to manage when retrieving the variable_map in the Dataproxy's GetData function: GetDataFromNodeExecution and GetDataFromTaskExecution. Since the variable_map resides in the TaskClosure, it is logical to retrieve it from the Task Repository using the Task's identification details (ID, project, domain, and version).

However, I am unable to determine how to access the output from node execution. Is there a way to retrieve the TaskClosure (or Task) directly from the NodeExecution?

Tracking issue

#6081

Why are the changes needed?

While the task input/output is represented as a dataclass or Pydantic model, using the get function to fetch the FlyteRemote execution output will fail due to the absence of the variable_map information. To address this issue, we aim to provide the input/output variable_map through the Dataproxy as a solution.

What changes were proposed in this pull request?

1. Provide variable_map information in GetData from dataproxy service.
2. Unit tests will be added later.

How was this patch tested?

TODO

Setup process

git clone https://github.com/flyteorg/flyte.git
gh pr checkout 

Screenshots

TODO

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

flyteorg/flytekit#3031

Docs link

TODO

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at eduardo@union.ai.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.

Project coverage is 33.22%. Comparing base (fd9a378) to head (07878e0).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...eidl/gen/pb-go/flyteidl/admin/node_execution.pb.go	0.00%	5 Missing ⚠️
...eidl/gen/pb-go/flyteidl/admin/task_execution.pb.go	0.00%	5 Missing ⚠️
...lyteidl/gen/pb-go/flyteidl/service/dataproxy.pb.go	0.00%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6136      +/-   ##
==========================================
- Coverage   37.01%   33.22%   -3.80%     
==========================================
  Files        1318     1020     -298     
  Lines      132525   108613   -23912     
==========================================
- Hits        49058    36084   -12974     
+ Misses      79222    69332    -9890     
+ Partials     4245     3197    -1048     

Flag	Coverage Δ
unittests-datacatalog	51.58% <ø> (ø)
unittests-flyteadmin	?
unittests-flytecopilot	30.99% <ø> (ø)
unittests-flytectl	62.29% <ø> (-0.05%)	⬇️
unittests-flyteidl	7.23% <0.00%> (-0.01%)	⬇️
unittests-flyteplugins	53.85% <ø> (-0.01%)	⬇️
unittests-flytepropeller	42.64% <ø> (+0.04%)	⬆️
unittests-flytestdlib	55.17% <ø> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.