-
Notifications
You must be signed in to change notification settings - Fork 255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
queue: able to set max disk used bytes for diskQueue #706
queue: able to set max disk used bytes for diskQueue #706
Conversation
queue/disk.go
Outdated
opts.MemoryQueueSize = 100 | ||
} | ||
|
||
if opts.EnableDiskUsedLimit && opts.MaxDiskUsedBytes <= 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
磁盘大小限制默认开启,默认的限制给到32GB
lgtm |
maxMemoryLength = 0 | ||
} else if enableMemory && maxMemoryLength <= 0 { | ||
maxMemoryLength = 100 | ||
func NewDiskQueue(opts NewDiskQueueOptions) BackendQueue { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这样改这个包就不兼容了,到时候smartelf和logkit-pro升级要注意下
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
本来就要多个参数编译就过不了的,一次性改成这种形式的以后更加方便添加更多选项
queue/disk.go
Outdated
d.writeOne(msg) | ||
err := d.writeOne(msg) | ||
if err != nil { | ||
log.Errorf("DISKQUEUE(%s): drop one msg from memory chan - %v", d.name, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
理论上这个错误永远不会出现
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个可以再放进memory queue里面吗
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
目前唯一调用 saveToDisk 的地方是在 ioLoop 退出的时候:
Lines 806 to 808 in d044334
if d.enableMemoryQueue { | |
d.saveToDisk() | |
} |
就目前的实现来讲,放回去也是没有意义的
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
那感觉这种情况应该强行放进去,否则丢数据了
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
如果 writeOne 一直失败就会无限循环退出不来
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
如果 writeOne 一直失败就会无限循环退不出来
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
不过之前就一直在丢数据... 这要怎么操作
Lines 793 to 800 in d044334
case <-d.exitChan: | |
if origin == FromMemory { | |
err = d.writeOne(dataRead) | |
if err != nil { | |
log.Errorf("DISKQUEUE(%s): drop one msg - %v", d.name, err) | |
} | |
} | |
break DONE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
你先加个FIXME吧,然后修一下CI
LGTM |
JIRA: https://jira.qiniu.io/browse/PDR-7002