Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

add text CNN model for text classification problem #1271

Merged
merged 1 commit into from
Dec 5, 2018
Merged

add text CNN model for text classification problem #1271

merged 1 commit into from
Dec 5, 2018

Conversation

yuwen-yan
Copy link
Contributor

@yuwen-yan yuwen-yan commented Dec 4, 2018

I'd like to add textCNN model for text classification problem.

  • It's my first time to add new model to tensor2tensor, so please feel free to let me know if you have any concerns.
  • Most code in this PR comes from this repo, I just did some adaptation work to follow tensor2tensor interface, again please let me know if you have any concerns. @dennybritz
  • I've done some experiments based on existing problem(sentiment_imdb), see below table in detail, again please let me know if we need more experiment result to add such model.
Problem Model Parameters Metrics
sentiment_imdb transformer_encoder hparam = transformer_tiny, train_steps = 2000 accuracy=0.7675
sentiment_imdb transformer_encoder hparam = transformer_tiny, train_steps = 10000 accuracy=0.8259
sentiment_imdb transformer_encoder hparam = transformer_tiny, train_steps = 20000 accuracy=0.8302
sentiment_imdb text_cnn hparam = text_cnn_base, train_steps = 20000 accuracy=0.8357

@googlebot googlebot added the cla: yes PR author has signed CLA label Dec 4, 2018
@yuwen-yan
Copy link
Contributor Author

Looks unittest failed with unrelated error, would appreciate any reviews or comments.

Copy link
Contributor

@lukaszkaiser lukaszkaiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the model!

@lukaszkaiser lukaszkaiser merged commit 19dcfcf into tensorflow:master Dec 5, 2018
tensorflow-copybara pushed a commit that referenced this pull request Dec 5, 2018
PiperOrigin-RevId: 224227181
@lukaszkaiser
Copy link
Contributor

Thanks for the model, good to know it can get reasonable results :).

theorm added a commit to theorm/tensor2tensor that referenced this pull request Dec 7, 2018
* master:
  New mesh_tensorflow transformer now handles encoder-decoder models and beam search. Fix bug in variable stacking code (operations in while loops did not get their inputs redirected) Fix datatype bug in simd_mesh_impl.py
  Minor cleanup. Moving one gym related function into its own file and added test. Unify naming across gym's method and ours and add "rl_env_" to the hparam, to give the hint where this gets used.
  internal merge of PR tensorflow#1277
  Temperature (tensorflow#1277)
  internal merge of PR tensorflow#1271
  add text CNN model for text classification problem (tensorflow#1271)
  Have a few shards for en-ro translation.
@p16i
Copy link

p16i commented Jan 29, 2019

Hi,

I'm trying to play with text_cnn, and it turns out that the model hasn't been added to __init__.py yet.

@ybbaigo would you like to do it? If not, I can do it for you.

@yuwen-yan
Copy link
Contributor Author

Sure, thanks for pointing out, I just added it with PR #1421 , please help review @afrozenator @lukaszkaiser

BTW, Please feel free to let me know if you have any other problem about text_cnn model in the future. @heytitle

@p16i
Copy link

p16i commented Jan 30, 2019

@ybbaigo you're welcome.

By the way, do you happen to have benchmarks for other problems, such as sentiment_sst_binary? I'm trying the model on the sst problem, but I always get very low accuracy.

kpe pushed a commit to kpe/tensor2tensor that referenced this pull request Mar 2, 2019
kpe pushed a commit to kpe/tensor2tensor that referenced this pull request Mar 2, 2019
PiperOrigin-RevId: 224227181
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla: yes PR author has signed CLA
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants