Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(v2): Fetch cache result from MLMD and put cache result into argo output artifacts/parameters #5957

Merged
merged 40 commits into from
Jul 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
a4798be
added model and storage layer for task
capri-xiyue Jun 12, 2021
b5805e2
added create task api
capri-xiyue Jun 14, 2021
3594e89
added api to list tasks
capri-xiyue Jun 14, 2021
7f1f7a9
modified task proto and fixed nits
capri-xiyue Jun 16, 2021
ad8f16c
renamed variable
capri-xiyue Jun 16, 2021
ee1e6d7
fixed ut
capri-xiyue Jun 16, 2021
7b29f2d
fixed UT
capri-xiyue Jun 16, 2021
155d50c
added UT for api_converter and resource manager
capri-xiyue Jun 17, 2021
5a1ec6d
added UT for api_converter and resource manager
capri-xiyue Jun 17, 2021
a68b0e2
fixed BE UT
capri-xiyue Jun 17, 2021
41d6cfe
added task storage layer UT
capri-xiyue Jun 19, 2021
7776460
merge with master and resolve conflicts
capri-xiyue Jun 29, 2021
9f06ff5
changed UT
capri-xiyue Jun 29, 2021
ca9d83a
fixed foreign key typo
capri-xiyue Jun 29, 2021
16851db
added some draft code for replay argo result
capri-xiyue Jun 30, 2021
fe323b7
added kfp client in v2
capri-xiyue Jun 30, 2021
23c06e4
merge with master and resolve conflicts
capri-xiyue Jun 30, 2021
634c028
deleted unused code
capri-xiyue Jun 30, 2021
8bcda3a
run go mode
capri-xiyue Jun 30, 2021
6d34968
upgraded go version in go v2 test
capri-xiyue Jun 30, 2021
db68df1
run go mod tidy
capri-xiyue Jun 30, 2021
4842007
added cache client
capri-xiyue Jun 30, 2021
8fa2da7
fixed nits
capri-xiyue Jul 1, 2021
8d493d6
fetched MLMD output parameter
capri-xiyue Jul 2, 2021
f92b3f0
store output artifacts metadata from cache
capri-xiyue Jul 3, 2021
71b060e
added mlmd pipeline context, exuection and output event
capri-xiyue Jul 5, 2021
1c97f1e
fixed grpc endpoint of mlpipeline
capri-xiyue Jul 6, 2021
3457037
fixed task backend bug and fixed launcher connect bug
capri-xiyue Jul 6, 2021
e5613e3
Merge branch 'master' into issue_5819
capri-xiyue Jul 6, 2021
885ecc5
resolve conflicts with master
capri-xiyue Jul 7, 2021
46b809c
Merge branch 'issue_5819' of https://github.com/capri-xiyue/pipelines…
capri-xiyue Jul 7, 2021
b9090c6
fixed cache bug
capri-xiyue Jul 8, 2021
d734aaa
added cache info
capri-xiyue Jul 8, 2021
411042f
fixed unused test
capri-xiyue Jul 8, 2021
22d485c
fixed backend test
capri-xiyue Jul 8, 2021
f2f3e66
fixed v2 build
capri-xiyue Jul 8, 2021
60c181b
refactored launcher
capri-xiyue Jul 8, 2021
b6186d1
removed unused code
capri-xiyue Jul 8, 2021
bb088a1
merge with master
capri-xiyue Jul 9, 2021
f67004f
fixed typo
capri-xiyue Jul 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion api/Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
PROTO_PATH=/usr/local/include/

gen_proto: v2alpha1/pipeline_spec.proto
gen_pipeline_spec_proto: v2alpha1/pipeline_spec.proto
cd v2alpha1 && protoc -I=$(PROTO_PATH) -I=. \
--go_out=go \
--go_opt=paths=source_relative \
pipeline_spec.proto

gen_cache_key_proto: v2alpha1/cache_key.proto
cd v2alpha1 && protoc -I=$(PROTO_PATH) -I=. \
--go_out=go \
--go_opt=paths=source_relative \
cache_key.proto
38 changes: 38 additions & 0 deletions api/v2alpha1/cache_key.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// Copyright 2021 The Kubeflow Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

option go_package = "github.com/kubeflow/pipelines/api/v2alpha1/go";
package ml_pipelines;

import "google/protobuf/any.proto";
import "pipeline_spec.proto";

message CacheKey {
map<string, ArtifactNameList> inputArtifactNames = 1;
map<string, Value> inputParameters = 2;
map<string, RuntimeArtifact> outputArtifactsSpec = 3;
map<string, string> outputParametersSpec=4;
ContainerSpec containerSpec=5;
}

message ContainerSpec {
string image = 1;
repeated string cmdArgs = 2;
}

message ArtifactNameList {
repeated string artifactNames = 1;
}
400 changes: 400 additions & 0 deletions api/v2alpha1/go/cache_key.pb.go

Large diffs are not rendered by default.

Loading