Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature]: Adding streaming serialize writer #31679

Closed
1 task done
tedxu opened this issue Mar 28, 2024 · 0 comments
Closed
1 task done

[Feature]: Adding streaming serialize writer #31679

tedxu opened this issue Mar 28, 2024 · 0 comments
Assignees
@tedxu
Labels
kind/feature Issues related to feature request from users

Comments

@tedxu
Copy link
Contributor

tedxu commented Mar 28, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Is your feature request related to a problem? Please describe.

This issue is focusing the streaming I/O support on the writer part, with respect to the reader part introduced already in #30860 .

Describe the solution you'd like.

No response

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response
@tedxu tedxu added the kind/feature Issues related to feature request from users label Mar 28, 2024
@tedxu tedxu mentioned this issue Mar 28, 2024
Merged
@tedxu tedxu assigned tedxu and unassigned xiaofan-luan Mar 28, 2024
@tedxu tedxu changed the title [Feature]: Adding streaming deserialize writer [Feature]: Adding streaming serialize writer Mar 29, 2024
sre-ci-robot pushed a commit that referenced this issue Apr 11, 2024
@tedxu
feat: adding binlog streaming writer (#31537)
dc5ea6f 
See #31679

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
@tedxu tedxu mentioned this issue Apr 25, 2024
Merged
sre-ci-robot pushed a commit that referenced this issue May 17, 2024
@tedxu
enhance: enable stream writer in compactions (#32612)
a9c7ce7 
See #31679

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
@tedxu tedxu mentioned this issue Jun 17, 2024
Merged
sre-ci-robot pushed a commit that referenced this issue Jun 19, 2024
@tedxu
feat: adding deltalog stream reader and writer (#33844)
6d5747c 
See #31679

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
@tedxu tedxu mentioned this issue Jun 21, 2024
Merged
sre-ci-robot pushed a commit that referenced this issue Jun 24, 2024
@tedxu
fix: turn on compression on stream writers (#34067)
78885a4 
See #31679

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
tedxu added a commit to tedxu/milvus that referenced this issue Jun 24, 2024
@tedxu
fix: turn on compression on stream writers (milvus-io#34067)
94c06e7 
See milvus-io#31679

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
@tedxu tedxu mentioned this issue Jun 24, 2024
Closed
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Jun 24, 2024
@XuanYang-cn
enhance: [cherry-pick] Move compactor into sub package
d5c2dbb 
This PR consists of the following commits:

- enhance: Tidy compactor and remove dup codes (milvus-io#32198)
- fix: Fix l0 compactor may cause DN from OOM (milvus-io#33554)
- enhance: Add deltaRowCount in l0 compaction (milvus-io#33997)
- enhance: enable stream writer in compactions (milvus-io#32612)

See also: milvus-io#32451, milvus-io#33547, milvus-io#33998, milvus-io#31679
pr: milvus-io#32198, milvus-io#33554, milvus-io#33997, milvus-io#32612

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
@XuanYang-cn XuanYang-cn mentioned this issue Jun 24, 2024
Merged
XuanYang-cn pushed a commit to XuanYang-cn/milvus that referenced this issue Jun 24, 2024
@tedxu @XuanYang-cn
fix: turn on compression on stream writers (milvus-io#34067)
b1dbbd4 
See milvus-io#31679

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue Jun 25, 2024
@XuanYang-cn @tedxu
enhance: [cherry-pick] Move compactor into sub package (#34098)
a33b686 
This PR consists of the following commits:

- enhance: Tidy compactor and remove dup codes (#32198)
- fix: Fix l0 compactor may cause DN from OOM (#33554)
- enhance: Add deltaRowCount in l0 compaction (#33997)
- enhance: enable stream writer in compactions (#32612)
- fix: turn on compression on stream writers (#34067)
- fix: adding blob memory size in binlog serde (#33324)

See also: #32451, #33547, #33998, #31679
pr: #32198, #33554, #33997, #32612

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
yellow-shine pushed a commit to yellow-shine/milvus that referenced this issue Jul 2, 2024
@tedxu @yellow-shine
feat: adding deltalog stream reader and writer (milvus-io#33844)
3dd5263 
See milvus-io#31679

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
yellow-shine pushed a commit to yellow-shine/milvus that referenced this issue Jul 2, 2024
@tedxu @yellow-shine
fix: turn on compression on stream writers (milvus-io#34067)
dd06e18 
See milvus-io#31679

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
@tedxu tedxu closed this as completed Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tedxu tedxu

Labels
kind/feature Issues related to feature request from users
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tedxu @xiaofan-luan