Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GPU Training #2

Closed
alexrisman opened this issue Jan 7, 2016 · 3 comments
Closed

GPU Training #2

alexrisman opened this issue Jan 7, 2016 · 3 comments

Comments

@alexrisman
Copy link

Hi, does this library support training on GPU?
@zhongkaifu
Copy link
Owner

Currently, RNNSharp doesn't support training on GPU, but I do have a plan for that. :)

zhongkaifu added a commit that referenced this issue Jan 9, 2016
@zhongkaifu
#1. Bug fix: output layer is not cleaned before calculating new values
1d2b3be 
#2. Add dropout for LSTM
@BackT0TheFuture
Copy link

Great to hear that. There's one project named "CUDAFY.NET", it allows easy development of high performance CUDA applications in C#. (CUDAFY.NET)[http://cudafy.codeplex.com/] <_>

@zhongkaifu
Copy link
Owner

Thanks for pointing out. I will look at CUDAFY.NET to see if we can use it.

zhongkaifu added a commit that referenced this issue Jan 27, 2016
@zhongkaifu
#1. Fix CRF training bug: learning network should not be called when …
f251876 
…running validation

#2. Support model vector quantization reduce model size to 1/4 original
#3. Refactoring code and speed up training
#4. Fixing feature extracting bug
@zhongkaifu zhongkaifu mentioned this issue Feb 12, 2016
Closed
zhongkaifu added a commit that referenced this issue Feb 15, 2016
@zhongkaifu
#1. Fix RNNSharp crashing bug when dense feature isn't used
1dc9759 
#2. Improve BiRNN learning process
#3. Support to train model without validated corpus
zhongkaifu added a commit that referenced this issue Feb 24, 2016
@zhongkaifu
#1. Support SIMD instruction to speed up encoding and decoding
e390b85 
#2. Optimize LSTM encoding to improve performance significantly
#3. Apply dynamtic learning rate
zhongkaifu added a commit that referenced this issue Feb 25, 2016
@zhongkaifu
#1. Fix Forward-LSTM crash bug
1a3070c 
#2. Improve encoding performance by SIMD instructions
zhongkaifu added a commit that referenced this issue Feb 25, 2016
@zhongkaifu
Revert "#1. Fix Forward-LSTM crash bug #2. Improve encoding performan…
513cb0c 
…ce by SIMD instructions"

This reverts commit 1a3070c.
zhongkaifu added a commit that referenced this issue Mar 9, 2016
@zhongkaifu
#1. LSTM weights vectorization for SIMD instruction
8df8dd9 
#2. Execute CRF forward-backward in parallel
zhongkaifu added a commit that referenced this issue Mar 9, 2016
@zhongkaifu
#1. Code refactoring
e699d62 
#2. Update readme file
zhongkaifu added a commit that referenced this issue Mar 9, 2016
@zhongkaifu
#1. Code refactoring
2d58c7a 
#2. Normalize LSTM cell value in weights updating
@zhongkaifu zhongkaifu mentioned this issue Apr 27, 2016
Open
@zhongkaifu zhongkaifu mentioned this issue May 26, 2016
Closed
zhongkaifu added a commit that referenced this issue Jul 8, 2016
@zhongkaifu
#1. Provide sparse features for each layer. The sparse feature is fro…
b277d06 
…m input layer.

#2. Refactoring dropout layer and output layer
#3. Refactoring layer initialization
@zhongkaifu zhongkaifu mentioned this issue Jul 15, 2016
Closed
zhongkaifu added a commit that referenced this issue Nov 30, 2016
@zhongkaifu
#1. Support sequence-to-sequence model
eb1cc30 
#2. Fix bug in softmax output layer when computing hidden layer value
#3. Refactoring code
zhongkaifu added a commit that referenced this issue Dec 22, 2016
@zhongkaifu
#1. Convert numeric type from double to float
0d6259f 
#2. Refactor configuration file and command line parameter
#3. use SIMD for backward pass in output layer
zhongkaifu added a commit that referenced this issue Jan 6, 2017
@zhongkaifu
#1. Improve LSTM layer performance ~ 25% by SIMD instruction.
a0d3aa6 
#2. Refactoring code
zhongkaifu added a commit that referenced this issue Jan 24, 2017
@zhongkaifu
#1. Fix the bug that RNNSharp is crashed if neither embedding nor aut…
b538b07 
…o encoder is used.

#2. For seq2seq autoencoder, concatenate first top hidden layer and last top hidden layer as final encoder output for decoder.
zhongkaifu added a commit that referenced this issue Feb 5, 2017
@zhongkaifu
#1. Retire BPTTLayer, since it's an old algorithm and its performance…
d80f402 
… is worse than LSTM

#2. Fix backward bug in Dropout layer
#3. Refactoring code
zhongkaifu added a commit that referenced this issue Feb 5, 2017
@zhongkaifu
#1. Fix error computing bug in bi-directional RNN when the number of …
1d08c95 
…hidden layer is more than 1

#2. Improve training part of bi-directional RNN. We don't re-run forward before updating weights
#3. Fix bugs in Dropout layer
#4. Change hidden layer settings in configuration file.
#5. Refactoring code
ericxsun pushed a commit to ericxsun/RNNSharp that referenced this issue Feb 9, 2017
@zhongkaifu
zhongkaifu#1. Improve SimpleRNN training performance
5b0de99 
zhongkaifu#2. Fix BPTT initialization bug
ericxsun pushed a commit to ericxsun/RNNSharp that referenced this issue Feb 9, 2017
@zhongkaifu
zhongkaifu#1. Support dropout for SimpleRNN (both forward and bi-dire…
debffdb 
…ctional)

zhongkaifu#2. Speed up training performance
ericxsun pushed a commit to ericxsun/RNNSharp that referenced this issue Feb 9, 2017
@zhongkaifu
zhongkaifu#1. Bug fix: Failed to load bi-directional model from file …
c18b865 
…in test model

zhongkaifu#2. Update: Train can be ended early if current PPL is larger than the previous one

Signed-off-by: Zhongkai Fu <fuzhongkai@gmail.com>
ericxsun pushed a commit to ericxsun/RNNSharp that referenced this issue Feb 9, 2017
@zhongkaifu
zhongkaifu#1. Improve logger
0838a69 
zhongkaifu#2. Using error token ratio to verify validated set performance
zhongkaifu added a commit that referenced this issue Feb 19, 2017
@zhongkaifu
#1.Fixed Dropout crashed bug in ForwardRNN
fe3bb58 
#2. Add incremental training
zhongkaifu added a commit that referenced this issue Mar 8, 2017
@zhongkaifu
#1. Improve encoder and decoder performance ~ 100% up
6778c27 
#2. Refactoring code
#3. Make RNNDecoder thread-safe
zhongkaifu added a commit that referenced this issue Mar 21, 2017
@zhongkaifu
#1. Fix the bug that incremental training is crashed.
df4406b 
#2. Delete useless code
zhongkaifu added a commit that referenced this issue May 3, 2017
@zhongkaifu
#1. Support mini batch for training
5043eb3 
#2. Improve training performnce ~ 300% up
#3. Fix learning rate update bug
#4. Apply SIMD instruction to update error in layers
#5. Code refactoring
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexrisman @zhongkaifu @BackT0TheFuture