feat: Added docling and pytorch as add on #5089
Conversation
ntkathole
force-pushed
the
docling_extra
branch
2 times, most recently
from
a71074f to
cbe4e99
Compare
ntkathole
force-pushed
the
docling_extra
branch
4 times, most recently
from
d98454b to
c6176b1
Compare
|
It seems this is causing increase in an image size. Do we have large runners available under our github org? |
|
@ntkathole this is what I see available. Anything here work?
|
ntkathole
force-pushed
the
docling_extra
branch
3 times, most recently
from
5239ef1 to
4be9a30
Compare
| @@ -162,7 +162,7 @@ docker-build: ## Build docker image with the manager. | |||
| ## Build feast docker image. | |||
| .PHONY: feast-ci-dev-docker-img | |||
| feast-ci-dev-docker-img: | |||
| cd ./../.. && make build-feature-server-dev | |||
| cd ./../.. && make build-feature-server-dev-minimal | |||
There was a problem hiding this comment.
@franciscojavierarceo Since we do not have larger runner to run kind + feast with all dependencies, either we need self-hosted runners or we need to use image with minimal dependencies installed in CI.
Currently moving from multicloud/Dockerfile.dev to multicloud/Dockerfile works in CI to run operator e2e test.
There was a problem hiding this comment.
My concern is that we miss potential breaks by using this minimal image. That's the risk here, right?
There was a problem hiding this comment.
We will be using this image just for operator e2e test, feast functionality will be tested via integration and unit tests with all dependencies installed.
@tchughesiv or someone more familiar with operator work can confirm if it's a big risk in operator testing or better way to handle this.
There was a problem hiding this comment.
this change needs to be reverted
There was a problem hiding this comment.
@ntkathole @franciscojavierarceo this must remain make build-feature-server-dev
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
ntkathole
force-pushed
the
docling_extra
branch
2 times, most recently
from
01d60df to
b2e1769
Compare
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
ntkathole
force-pushed
the
docling_extra
branch
from
b2e1769 to
adee3e0
Compare
|
looks like that worked! |
| build-feature-server-dev-minimal: | ||
| docker buildx build \ | ||
| -t feastdev/feature-server:dev \ | ||
| -f sdk/python/feast/infra/feature_servers/multicloud/Dockerfile \ | ||
| --load sdk/python/feast/infra/feature_servers/multicloud | ||
|
|
There was a problem hiding this comment.
this is just a release build... its not a dev image build
There was a problem hiding this comment.
we'll need to remove this
What this PR does / why we need it:
Adds docling and torch as add on to feast.
Integrating Docling would allow users to efficiently chunk text data within Online Feature Views (ODFVs) during the write process and torch will allow conversion of feature vectors into pytorch tensors.
Which issue(s) this PR fixes:
#5037 #4890