Double buffer for cpp reader #8841
Conversation
| write_pos_(0), | ||
| read_pos_(0) { | ||
| std::thread prefetch( | ||
| std::bind(&DoubleBufferReader::PrefetchThreadFunc, this)); |
There was a problem hiding this comment.
delete the bind is ok here.
| if (read_pos_ >= kDoubleBufferSize) { | ||
| read_pos_ = 0; | ||
| } | ||
| buffer_not_full_.notify_all(); |
There was a problem hiding this comment.
For there is only one producer and one consumer here, notify_all() and notify_one() is same.
| private: | ||
| void PrefetchThreadFunc(); | ||
|
|
||
| std::vector<std::vector<framework::LoDTensor>> buffer_; |
There was a problem hiding this comment.
Maybe a thread safe channel (queue) is better here.
There was a problem hiding this comment.
I'll take a look at that. Thanks.
dzhwinter
left a comment
dzhwinter
left a comment
There was a problem hiding this comment.
Looks Great. Only one comment, now we have two runtime type(the real type is decided in runtime -- framework::VarDesc::Raw, ReaderHolder. Could we figure out a better way to unify the runtime type?
|
|
||
| prog = fluid.framework.Program() | ||
| block = prog.current_block() | ||
| startup_prog = fluid.framework.Program() |
There was a problem hiding this comment.
Why make the reader operator as a part of startup program?
There was a problem hiding this comment.
Because a reader should be created only once.
JiayiFeng
changed the title
2 participants
fixes #8847
Please review #8913 first.