seqToseq translation demo #243
Description
This is a strange bug where the model has to be saved under a subdirectory within Paddle/demo/seqToseq/translation and cannot be saved onto the Paddle/demo/seqToseq/ dynamically:
E.g. if i want the save the model directory to translation/model, then train.sh from the demo works.
Also, if i use a static full path, e.g. /path/to/model/ the model saves properly too, i.e.
paddle train \
--config='translation/train.conf' \
--save_dir='/path/to/model/' \
--use_gpu=false \
--num_passes=16 \
--show_parameter_stats_period=100 \
--trainer_count=4 \
--log_period=10 \
--dot_period=5 \
2>&1 | tee 'translation/train.log'
But if i use the same path as Paddle/demo/seqToseq/, Paddle will throw the error after the first epoch:
I1025 15:11:11.968819 3359 TrainerInternal.cpp:163] Batch=200 samples=10000 AvgCost=54.3925 CurrentCost=47.7486 Eval: classification_error_evaluator=0.822107 CurrentEval: classification_error_evaluator=0.756779
I1025 15:11:11.968868 3359 TrainerInternal.cpp:180] Pass=0 Batch=200 samples=10000 AvgCost=54.3925 Eval: classification_error_evaluator=0.822107
I1025 15:11:37.395536 3359 Tester.cpp:111] Test samples=50000 cost=48.2543 Eval: classification_error_evaluator=0.758948
/home/ltan/Paddle/binary/bin/paddle: line 81: 3359 Segmentation fault (core dumped) ${DEBUGGER} $MYDIR/../opt/paddle/bin/paddle_trainer ${@:2}
To replicate the error above, change this line in train.sh to :
--save_dir='model' \
It's a rather odd bug, although it can be simply solved by using fullpath, i.e.:
--save_dir='/home/ltan/Paddle/demo/seqToseq/model' \