operators needed by recommender system

[QiJune]

• fully connected
• lookup table, sparse gradient
• sequence conv pool
• cos sim
• sparse optimizers, adam
• square error cost

[mkliegl]

I was looking a little at what is needed to finish sequence conv pool.

Option 1

The current implementation seems to be a composite layer consisting of context projection + fully connected + sequence pool:

https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer_config_helpers/networks.py#L40

It looks like we already have C++ ops for fully connected and sequence pool, so only context projection seems to be missing for this route.

Option 2

context projection + fully connected is a special case of sequence convolution, as the name "sequence conv pool" already implies.

If we go this route, it would make more sense to prioritize implementing a sequence conv op than a context projection op.

Questions

• Do others have a preference between Options 1 and 2?
• Is somebody else already working on any of sequence pool / sequence conv / context projection / sequence conv pool layer?

[chengduoZH]

I'm developing context_project_op code, and it's probably going to be done this week.

[qingqing01]

@mkliegl @chengduoZH discussed with me a few days ago. Now I prefer to the second option.

[luotao1]

I'm developing sequence pool code, and it's probably going to be done this week.

• max_seq (doing) : add Max strategy for sequence_pool op #4864
• last_seq/first_seq/sqrt_seq (done) : add SQRT/LAST/FIRST strategy for Seqpool #4788
• sum_seq/average_seq (done) : implement framework of seq_pool_op and its unitest #4193

[chengduoZH]

sequence conv has been merged. @QiJune