Training with Sejong Treebank corpus

[xtknight]

Hello,

I was trying to train using the sejong_treebank.sample file, so I ran the following commands:
$ ./sejong/split.sh
$ ./sejong/c2d.sh
$ ./train_sejong.sh

But had an error (same as one below -- "Assign requires shapes of both tensors to match").
So then, I tried downloading a larger treebank corpus from sejong.or.kr (it seems to be the full version of the sejong_treebank.sample in your repository, but then again I'm not sure...) But, the same thing happened.

My input file (tried both sample and full) is just an long stream of the following in UTF-8, just like your sample Sejong file. Is there somewhere else I need to put this? Or is there something else I need other than saving this as sejong/sejong_treebank.txt.v1 and running the scripts?

; 1993/06/08 19 
(NP     (NP 1993/SN + //SP + 06/SN + //SP + 08/SN)
        (NP 19/SN))
; 엠마누엘 웅가로 / 
(NP     (NP     (NP 엠마누엘/NNP)
                (NP 웅가로/NNP))
        (X //SP))
; 의상서 실내 장식품으로… 
(NP_AJT         (NP_AJT 의상/NNG + 서/JKB)
        (NP_AJT         (NP 실내/NNG)
                (NP_AJT 장식품/NNG + 으로/JKB + …/SE)))
; 디자인 세계 넓혀 
(VP     (NP_OBJ         (NP 디자인/NNG)
                (NP_OBJ 세계/NNG))
        (VP 넓히/VV + 어/EC))
; 프랑스의 세계적인 의상 디자이너 엠마누엘 웅가로가 실내 장식용 직물 디자이너로 나섰다. 
(S      (NP_SBJ         (NP     (NP_MOD 프랑스/NNP + 의/JKG)
                        (NP     (VNP_MOD 세계/NNG + 적/XSN + 이/VCP + ᆫ/ETM)
                                (NP     (NP 의상/NNG)
                                        (NP 디자이너/NNG))))
                (NP_SBJ         (NP 엠마누엘/NNP)
                        (NP_SBJ 웅가로/NNP + 가/JKS)))
        (VP     (NP_AJT         (NP     (NP     (NP 실내/NNG)
                                        (NP 장식/NNG + 용/XSN))
                                (NP 직물/NNG))
                        (NP_AJT 디자이너/NNG + 로/JKB))
                (VP 나서/VV + 었/EP + 다/EF + ./SF)))

Here's the logs with all the verbose options.

andy@andy ~/Downloads/syntaxnet/models/syntaxnet/work $ ./sejong/split.sh  -v -v
+ '[' 0 '!=' 0 ']'
++++ readlink -f ./sejong/split.sh
+++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/split.sh
++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
+ CDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
+ [[ -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh ]]
+ . /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
++ set -o errexit
++ export LC_ALL=ko_KR.UTF-8
++ LC_ALL=ko_KR.UTF-8
++ export LANG=ko_KR.UTF-8
++ LANG=ko_KR.UTF-8
+++++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
++++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
+++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
++ CDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
+++++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
++++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
+++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/..
++ PDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work
++ python=/usr/bin/python
+ make_calmness
+ exec
+ exec
+ child_verbose='-v -v'
+ '[' '!' -e /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/wdir ']'
+ WDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/wdir
+ '[' '!' -e /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/log ']'
+ LDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/log
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/split.py --mode=0
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/split.py --mode=1
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/split.py --mode=2
+ close_fd
+ exec


andy@andy ~/Downloads/syntaxnet/models/syntaxnet/work $ ./sejong/c2d.sh  -v -v
+ '[' 0 '!=' 0 ']'
++++ readlink -f ./sejong/c2d.sh
+++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.sh
++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
+ CDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
+ [[ -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh ]]
+ . /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
++ set -o errexit
++ export LC_ALL=ko_KR.UTF-8
++ LC_ALL=ko_KR.UTF-8
++ export LANG=ko_KR.UTF-8
++ LANG=ko_KR.UTF-8
+++++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
++++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
+++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
++ CDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong
+++++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
++++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/env.sh
+++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/..
++ PDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work
++ python=/usr/bin/python
+ make_calmness
+ exec
+ exec
+ child_verbose='-v -v'
+ '[' '!' -e /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/wdir ']'
+ WDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/wdir
+ '[' '!' -e /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/log ']'
+ LDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/log
+ for SET in training tuning test
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.py --mode=0
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.py --mode=1
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/align.py
number_of_sent = 0, number_of_sent_skip = 0
+ for SET in training tuning test
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.py --mode=0
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.py --mode=1
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/align.py
number_of_sent = 0, number_of_sent_skip = 0
+ for SET in training tuning test
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.py --mode=0
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/c2d.py --mode=1
+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/align.py
number_of_sent = 0, number_of_sent_skip = 0
+ close_fd
+ exec


andy@andy ~/Downloads/syntaxnet/models/syntaxnet/work $ ./train_sejong.sh  -v -v
+ '[' 0 '!=' 0 ']'
++++ readlink -f ./train_sejong.sh
+++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/train_sejong.sh
++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work
+ CDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work
++++ readlink -f ./train_sejong.sh
+++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/train_sejong.sh
++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/..
+ PDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet
+ make_calmness
+ exec
+ exec
+ cd /home/andy/Downloads/syntaxnet/models/syntaxnet
+ python=/usr/bin/python
+ SYNTAXNET_HOME=/home/andy/Downloads/syntaxnet/models/syntaxnet
+ BINDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet
+ CONTEXT=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/context.pbtxt_p
+ TMP_DIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output
+ mkdir -p /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output
+ cat /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/context.pbtxt_p
+ sed s=OUTPATH=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output=
+ MODEL_DIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/models
+ HIDDEN_LAYER_SIZES=512,512
+ HIDDEN_LAYER_PARAMS=512,512
+ BATCH_SIZE=256
+ BEAM_SIZE=16
+ LP_PARAMS=512,512-0.08-4400-0.85
+ GP_PARAMS=512,512-0.02-100-0.9
+ pretrain_parser
+ /home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_trainer --arg_prefix=brain_parser --batch_size=256 --compute_lexicon --decay_steps=4400 --graph_builder=greedy --hidden_layer_sizes=512,512 --learning_rate=0.08 --momentum=0.85 --beam_size=1 --output_path=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output --task_context=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/context --projectivize_training_set --training_corpus=tagged-training-corpus --tuning_corpus=tagged-tuning-corpus --params=512,512-0.08-4400-0.85 --num_epochs=20 --report_every=100 --checkpoint_every=1000 --logtostderr
INFO:tensorflow:Computing lexicon...
I syntaxnet/lexicon_builder.cc:124] Term maps collected over 0 tokens from 0 documents
I syntaxnet/term_frequency_map.cc:137] Saved 0 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:137] Saved 0 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/lcword-map.
I syntaxnet/term_frequency_map.cc:137] Saved 0 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
I syntaxnet/term_frequency_map.cc:137] Saved 0 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/category-map.
I syntaxnet/term_frequency_map.cc:137] Saved 0 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
INFO:tensorflow:Preprocessing...
INFO:tensorflow:Training...
INFO:tensorflow:Building training network with parameters: feature_sizes: [20 20 12] domain_sizes: [3 3 3]
INFO:tensorflow:Initializing...
INFO:tensorflow:Training...
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/reader_ops.cc:141] Starting epoch 1
I syntaxnet/reader_ops.cc:141] Starting epoch 2
I syntaxnet/reader_ops.cc:141] Starting epoch 3
I syntaxnet/reader_ops.cc:141] Starting epoch 4
I syntaxnet/reader_ops.cc:141] Starting epoch 5
I syntaxnet/reader_ops.cc:141] Starting epoch 6
I syntaxnet/reader_ops.cc:141] Starting epoch 7
I syntaxnet/reader_ops.cc:141] Starting epoch 8
I syntaxnet/reader_ops.cc:141] Starting epoch 9
I syntaxnet/reader_ops.cc:141] Starting epoch 10
I syntaxnet/reader_ops.cc:141] Starting epoch 11
I syntaxnet/reader_ops.cc:141] Starting epoch 12
I syntaxnet/reader_ops.cc:141] Starting epoch 13
I syntaxnet/reader_ops.cc:141] Starting epoch 14
I syntaxnet/reader_ops.cc:141] Starting epoch 15
I syntaxnet/reader_ops.cc:141] Starting epoch 16
I syntaxnet/reader_ops.cc:141] Starting epoch 17
I syntaxnet/reader_ops.cc:141] Starting epoch 18
I syntaxnet/reader_ops.cc:141] Starting epoch 19
I syntaxnet/reader_ops.cc:141] Starting epoch 20
+ evaluate_pretrained_parser
+ for SET in training tuning test
+ /home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval --task_context=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/brain_parser/greedy/512,512-0.08-4400-0.85/context --batch_size=256 --hidden_layer_sizes=512,512 --beam_size=1 --input=tagged-training-corpus --output=parsed-training-corpus --arg_prefix=brain_parser --graph_builder=greedy --model_path=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/brain_parser/greedy/512,512-0.08-4400-0.85/model
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 0 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
INFO:tensorflow:Building training network with parameters: feature_sizes: [20 20 12] domain_sizes: [3 3 3]
Traceback (most recent call last):
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 149, in <module>
    tf.app.run()
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 145, in main
    Eval(sess, num_actions, feature_sizes, domain_sizes, embedding_dims)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 98, in Eval
    parser.saver.restore(sess, FLAGS.model_path)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 1104, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 333, in run
    run_metadata_ptr)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 573, in _run
    feed_dict_string, options, run_metadata)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 653, in _do_run
    target_list, options, run_metadata)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 673, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [3,64] rhs shape= [485,64]
     [[Node: save/Assign_5 = Assign[T=DT_FLOAT, _class=["loc:@embedding_matrix_0"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](params/embedding_matrix_0/ExponentialMovingAverage, save/restore_slice_5)]]
Caused by op u'save/Assign_5', defined at:
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 149, in <module>
    tf.app.run()
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 145, in main
    Eval(sess, num_actions, feature_sizes, domain_sizes, embedding_dims)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 96, in Eval
    parser.AddSaver(FLAGS.slim_model)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/graph_builder.py", line 568, in AddSaver
    self.saver = tf.train.Saver(variables_to_save)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 845, in __init__
    restore_sequentially=restore_sequentially)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 515, in build
    filename_tensor, vars_to_save, restore_sequentially, reshape)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 281, in _AddRestoreOps
    validate_shape=validate_shape))
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/ops/gen_state_ops.py", line 45, in assign
    use_locking=use_locking, name=name)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/ops/op_def_library.py", line 693, in apply_op
    op_def=op_def)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/framework/ops.py", line 2186, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/framework/ops.py", line 1170, in __init__
    self._traceback = _extract_stack()

[xtknight]

Okay, for the full corpus it seems like c2d was failing to create a deptree file in build_tree()...because of an unaligned sentence. Not sure what's going on there. So when I manually copied some files around and ran just with the sample corpus, I finally got something going again, but running into what seems like the same error.

andy@andy ~/Downloads/syntaxnet/models/syntaxnet/work $ ./train_sejong.sh -v -v 
+ '[' 0 '!=' 0 ']'
++++ readlink -f ./train_sejong.sh
+++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/train_sejong.sh
++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work
+ CDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work
++++ readlink -f ./train_sejong.sh
+++ dirname /home/andy/Downloads/syntaxnet/models/syntaxnet/work/train_sejong.sh
++ readlink -f /home/andy/Downloads/syntaxnet/models/syntaxnet/work/..
+ PDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet
+ make_calmness
+ exec
+ exec
+ cd /home/andy/Downloads/syntaxnet/models/syntaxnet
+ python=/usr/bin/python
+ SYNTAXNET_HOME=/home/andy/Downloads/syntaxnet/models/syntaxnet
+ BINDIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet
+ CONTEXT=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/context.pbtxt_p
+ TMP_DIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output
+ mkdir -p /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output
+ cat /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/context.pbtxt_p
+ sed s=OUTPATH=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output=
+ MODEL_DIR=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/models
+ HIDDEN_LAYER_SIZES=512,512
+ HIDDEN_LAYER_PARAMS=512,512
+ BATCH_SIZE=256
+ BEAM_SIZE=16
+ LP_PARAMS=512,512-0.08-4400-0.85
+ GP_PARAMS=512,512-0.02-100-0.9
+ pretrain_parser
+ /home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_trainer --arg_prefix=brain_parser --batch_size=256 --compute_lexicon --decay_steps=4400 --graph_builder=greedy --hidden_layer_sizes=512,512 --learning_rate=0.08 --momentum=0.85 --beam_size=1 --output_path=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output --task_context=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/context --projectivize_training_set --training_corpus=tagged-training-corpus --tuning_corpus=tagged-tuning-corpus --params=512,512-0.08-4400-0.85 --num_epochs=20 --report_every=100 --checkpoint_every=1000 --logtostderr
INFO:tensorflow:Computing lexicon...
I syntaxnet/lexicon_builder.cc:124] Term maps collected over 1745 tokens from 71 documents
I syntaxnet/term_frequency_map.cc:137] Saved 534 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:137] Saved 534 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/lcword-map.
I syntaxnet/term_frequency_map.cc:137] Saved 36 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
I syntaxnet/term_frequency_map.cc:137] Saved 36 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/category-map.
I syntaxnet/term_frequency_map.cc:137] Saved 27 terms to /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 27 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/term_frequency_map.cc:101] Loaded 534 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 36 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
INFO:tensorflow:Preprocessing...
INFO:tensorflow:Training...
INFO:tensorflow:Building training network with parameters: feature_sizes: [20 20 12] domain_sizes: [537  39  30]
INFO:tensorflow:Initializing...
INFO:tensorflow:Training...
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/term_frequency_map.cc:101] Loaded 534 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 36 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 27 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/reader_ops.cc:141] Starting epoch 1
INFO:tensorflow:Epochs: 1, num steps: 100, seconds elapsed: 2.40, avg cost: 0.26, 
I syntaxnet/reader_ops.cc:141] Starting epoch 2
INFO:tensorflow:Epochs: 2, num steps: 200, seconds elapsed: 3.87, avg cost: 0.14, 
INFO:tensorflow:Epochs: 2, num steps: 300, seconds elapsed: 5.26, avg cost: 0.11, 
I syntaxnet/reader_ops.cc:141] Starting epoch 3
INFO:tensorflow:Epochs: 3, num steps: 400, seconds elapsed: 6.80, avg cost: 0.19, 
I syntaxnet/reader_ops.cc:141] Starting epoch 4
INFO:tensorflow:Epochs: 4, num steps: 500, seconds elapsed: 8.13, avg cost: 0.06, 
INFO:tensorflow:Epochs: 4, num steps: 600, seconds elapsed: 9.68, avg cost: 0.21, 
I syntaxnet/reader_ops.cc:141] Starting epoch 5
INFO:tensorflow:Epochs: 5, num steps: 700, seconds elapsed: 11.13, avg cost: 0.14, 
INFO:tensorflow:Epochs: 5, num steps: 800, seconds elapsed: 12.51, avg cost: 0.11, 
I syntaxnet/reader_ops.cc:141] Starting epoch 6
INFO:tensorflow:Epochs: 6, num steps: 900, seconds elapsed: 14.05, avg cost: 0.19, 
I syntaxnet/reader_ops.cc:141] Starting epoch 7
INFO:tensorflow:Epochs: 7, num steps: 1000, seconds elapsed: 15.39, avg cost: 0.06, 
INFO:tensorflow:Evaluating training network.
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/reader_ops.cc:141] Starting epoch 1
I syntaxnet/reader_ops.cc:141] Starting epoch 2
INFO:tensorflow:Seconds elapsed in evaluation: 0.50, eval metric: 8.42%
INFO:tensorflow:Writing out trained parameters.
INFO:tensorflow:Epochs: 7, num steps: 1100, seconds elapsed: 18.31, avg cost: 0.19, 
I syntaxnet/reader_ops.cc:141] Starting epoch 8
INFO:tensorflow:Epochs: 8, num steps: 1200, seconds elapsed: 19.78, avg cost: 0.13, 
INFO:tensorflow:Epochs: 8, num steps: 1300, seconds elapsed: 21.21, avg cost: 0.09, 
I syntaxnet/reader_ops.cc:141] Starting epoch 9
INFO:tensorflow:Epochs: 9, num steps: 1400, seconds elapsed: 22.86, avg cost: 0.16, 
I syntaxnet/reader_ops.cc:141] Starting epoch 10
INFO:tensorflow:Epochs: 10, num steps: 1500, seconds elapsed: 24.29, avg cost: 0.05, 
INFO:tensorflow:Epochs: 10, num steps: 1600, seconds elapsed: 25.93, avg cost: 0.15, 
I syntaxnet/reader_ops.cc:141] Starting epoch 11
INFO:tensorflow:Epochs: 11, num steps: 1700, seconds elapsed: 27.49, avg cost: 0.10, 
INFO:tensorflow:Epochs: 11, num steps: 1800, seconds elapsed: 28.93, avg cost: 0.07, 
I syntaxnet/reader_ops.cc:141] Starting epoch 12
INFO:tensorflow:Epochs: 12, num steps: 1900, seconds elapsed: 30.58, avg cost: 0.13, 
I syntaxnet/reader_ops.cc:141] Starting epoch 13
INFO:tensorflow:Epochs: 13, num steps: 2000, seconds elapsed: 31.98, avg cost: 0.05, 
INFO:tensorflow:Evaluating training network.
I syntaxnet/reader_ops.cc:141] Starting epoch 3
INFO:tensorflow:Seconds elapsed in evaluation: 0.46, eval metric: 80.29%
INFO:tensorflow:Writing out trained parameters.
INFO:tensorflow:Epochs: 13, num steps: 2100, seconds elapsed: 34.92, avg cost: 0.11, 
I syntaxnet/reader_ops.cc:141] Starting epoch 14
INFO:tensorflow:Epochs: 14, num steps: 2200, seconds elapsed: 36.47, avg cost: 0.09, 
INFO:tensorflow:Epochs: 14, num steps: 2300, seconds elapsed: 37.92, avg cost: 0.06, 
I syntaxnet/reader_ops.cc:141] Starting epoch 15
INFO:tensorflow:Epochs: 15, num steps: 2400, seconds elapsed: 39.51, avg cost: 0.11, 
I syntaxnet/reader_ops.cc:141] Starting epoch 16
INFO:tensorflow:Epochs: 16, num steps: 2500, seconds elapsed: 40.94, avg cost: 0.04, 
INFO:tensorflow:Epochs: 16, num steps: 2600, seconds elapsed: 42.58, avg cost: 0.09, 
I syntaxnet/reader_ops.cc:141] Starting epoch 17
INFO:tensorflow:Epochs: 17, num steps: 2700, seconds elapsed: 44.21, avg cost: 0.07, 
INFO:tensorflow:Epochs: 17, num steps: 2800, seconds elapsed: 45.59, avg cost: 0.05, 
I syntaxnet/reader_ops.cc:141] Starting epoch 18
INFO:tensorflow:Epochs: 18, num steps: 2900, seconds elapsed: 47.28, avg cost: 0.09, 
I syntaxnet/reader_ops.cc:141] Starting epoch 19
INFO:tensorflow:Epochs: 19, num steps: 3000, seconds elapsed: 48.66, avg cost: 0.04, 
INFO:tensorflow:Evaluating training network.
I syntaxnet/reader_ops.cc:141] Starting epoch 4
INFO:tensorflow:Seconds elapsed in evaluation: 0.50, eval metric: 78.45%
INFO:tensorflow:Writing out trained parameters.
INFO:tensorflow:Epochs: 19, num steps: 3100, seconds elapsed: 51.34, avg cost: 0.08, 
I syntaxnet/reader_ops.cc:141] Starting epoch 20
+ evaluate_pretrained_parser
+ for SET in training tuning test
+ /home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval --task_context=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/brain_parser/greedy/512,512-0.08-4400-0.85/context --batch_size=256 --hidden_layer_sizes=512,512 --beam_size=1 --input=tagged-training-corpus --output=parsed-training-corpus --arg_prefix=brain_parser --graph_builder=greedy --model_path=/home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/brain_parser/greedy/512,512-0.08-4400-0.85/model
I syntaxnet/term_frequency_map.cc:101] Loaded 27 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/label-map.
I syntaxnet/embedding_feature_extractor.cc:35] Features: input.word input(1).word input(2).word input(3).word stack.word stack(1).word stack(2).word stack(3).word stack.child(1).word stack.child(1).sibling(-1).word stack.child(-1).word stack.child(-1).sibling(1).word stack(1).child(1).word stack(1).child(1).sibling(-1).word stack(1).child(-1).word stack(1).child(-1).sibling(1).word stack.child(2).word stack.child(-2).word stack(1).child(2).word stack(1).child(-2).word; input.tag input(1).tag input(2).tag input(3).tag stack.tag stack(1).tag stack(2).tag stack(3).tag stack.child(1).tag stack.child(1).sibling(-1).tag stack.child(-1).tag stack.child(-1).sibling(1).tag stack(1).child(1).tag stack(1).child(1).sibling(-1).tag stack(1).child(-1).tag stack(1).child(-1).sibling(1).tag stack.child(2).tag stack.child(-2).tag stack(1).child(2).tag stack(1).child(-2).tag; stack.child(1).label stack.child(1).sibling(-1).label stack.child(-1).label stack.child(-1).sibling(1).label stack(1).child(1).label stack(1).child(1).sibling(-1).label stack(1).child(-1).label stack(1).child(-1).sibling(1).label stack.child(2).label stack.child(-2).label stack(1).child(2).label stack(1).child(-2).label 
I syntaxnet/embedding_feature_extractor.cc:36] Embedding names: words;tags;labels
I syntaxnet/embedding_feature_extractor.cc:37] Embedding dims: 64;32;32
I syntaxnet/term_frequency_map.cc:101] Loaded 534 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/word-map.
I syntaxnet/term_frequency_map.cc:101] Loaded 36 terms from /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/tag-map.
INFO:tensorflow:Building training network with parameters: feature_sizes: [20 20 12] domain_sizes: [537  39  30]
Traceback (most recent call last):
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 149, in <module>
    tf.app.run()
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 145, in main
    Eval(sess, num_actions, feature_sizes, domain_sizes, embedding_dims)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 98, in Eval
    parser.saver.restore(sess, FLAGS.model_path)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 1104, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 333, in run
    run_metadata_ptr)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 573, in _run
    feed_dict_string, options, run_metadata)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 653, in _do_run
    target_list, options, run_metadata)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/client/session.py", line 673, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,55] rhs shape= [71,55]
     [[Node: save/Assign_15 = Assign[T=DT_FLOAT, _class=["loc:@transition_scores"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](transition_scores, save/restore_slice_15)]]
Caused by op u'save/Assign_15', defined at:
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 149, in <module>
    tf.app.run()
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 145, in main
    Eval(sess, num_actions, feature_sizes, domain_sizes, embedding_dims)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/parser_eval.py", line 96, in Eval
    parser.AddSaver(FLAGS.slim_model)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__/syntaxnet/graph_builder.py", line 568, in AddSaver
    self.saver = tf.train.Saver(variables_to_save)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 845, in __init__
    restore_sequentially=restore_sequentially)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 515, in build
    filename_tensor, vars_to_save, restore_sequentially, reshape)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/training/saver.py", line 281, in _AddRestoreOps
    validate_shape=validate_shape))
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/ops/gen_state_ops.py", line 45, in assign
    use_locking=use_locking, name=name)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/ops/op_def_library.py", line 693, in apply_op
    op_def=op_def)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/framework/ops.py", line 2186, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/andy/Downloads/syntaxnet/models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/tf/tensorflow/python/framework/ops.py", line 1170, in __init__
    self._traceback = _extract_stack()

[dsindex]

@xtknight

before executing train_sejong.sh

• place your large corpus to sejong/

12:14 $ ls sejong/sejong*
sejong/sejong_treebank.sample  sejong/sejong_treebank.txt.v1

• sejong_treebank.txt.v1 has same format to sejong_treebank.sample
• and then

$ cd sejong
$ ./split.sh -v -v
$ ls wdir/sejong_treebank.txt.v1.*
wdir/sejong_treebank.txt.v1.test  wdir/sejong_treebank.txt.v1.training  wdir/sejong_treebank.txt.v1.tuning
$ ./c2d.sh -v -v
$ ls wdir/deptree.txt.v3*
wdir/deptree.txt.v3.test  wdir/deptree.txt.v3.training  wdir/deptree.txt.v3.tuning

• check your context.pbtxt_p

$ cat sejong/context.pbtxt_p
...
input {
  name: 'tagged-training-corpus'
  record_format: 'conll-sentence'
  Part {
    file_pattern: 'work/sejong/wdir/deptree.txt.v3.training'
  }
}
input {
  name: 'tagged-tuning-corpus'
  record_format: 'conll-sentence'
  Part {
    file_pattern: 'work/sejong/wdir/deptree.txt.v3.tuning'
  }
}
input {
  name: 'tagged-test-corpus'
  record_format: 'conll-sentence'
  Part {
    file_pattern: 'work/sejong/wdir/deptree.txt.v3.test'
  }
}
...

• finaly run train_sejon.sh -v -v
• short test result : https://github.com/dsindex/blog/wiki/%5Bparsing%5D-SyntaxNet

[xtknight]

Thanks for your quick response!
I was able to train it yesterday after copying around some files again (making the v1.training, v1.tuning, v1.test files). And also, I realized my full corpus file was missing LF(line feeds) between training data entries so I fixed that and it worked. That also fixed split.sh so now I don't have to manually create the training, tuning, test files.

When I use the SyntaxNet for English, the result is a root word. In the training data the root of each training sentence is clearly specified. However in the training data for the Korean text for Sejong TreeBank I don't see how the root is specified. How does that work?

English SyntaxNet training data (the root is clearly indicated on node 3)

1   Al  Al  PROPN   NNP Number=Sing 3   name    _   SpaceAfter=No
2   -   -   PUNCT   HYPH    _   3   punct   _   SpaceAfter=No
3   Zaman   Zaman   PROPN   NNP Number=Sing 0   **root**    _   _
4   :   :   PUNCT   :   _   3   punct   _   _
5   American    american    ADJ JJ  Degree=Pos  6   amod    _   _
6   forces  force   NOUN    NNS Number=Plur 7   nsubj   _   _
7   killed  kill    VERB    VBD Mood=Ind|Tense=Past|VerbForm=Fin    3   parataxis   _   _
8   Shaikh  Shaikh  PROPN   NNP Number=Sing 12  name    _   _
9   Abdullah    Abdullah    PROPN   NNP Number=Sing 12  name    _   _
10  al  al  PROPN   NNP Number=Sing 12  name    _   SpaceAfter=No
11  -   -   PUNCT   HYPH    _   12  punct   _   SpaceAfter=No
12  Ani Ani PROPN   NNP Number=Sing 7   dobj    _   SpaceAfter=No
13  ,   ,   PUNCT   ,   _   12  punct   _   _
14  the the DET DT  Definite=Def|PronType=Art   15  det _   _

Where is root indicated in training data here and what is the difference between v2 and v3 deptree? I guess v3 is going into SyntaxNet for training but I'm not sure about v2, and your sample output seems closer to v2. (10 nodes)

deptree.txt.v2.training

1       그      그/MM   DP      2
2       날      날/NNG  NP_AJT  10
3       지회에서는      지회/NNG + 에서/JKB + 는/JX     NP_AJT  10
4       해직교사        해직/NNG + 교사/NNG     NP      5
5       비상대책회의인가        비상/NNG + 대책/NNG + 회의/NNG + 이/VCP + ᆫ가/EC        VNP     6
6       하는    하/VV + 는/ETM  VP_MOD  9
7       조금은  조금/NNG + 은/JX        NP_AJT  8
8       살벌한  살벌/NNG + 하/XSA + ᆫ/ETM       VP_MOD  9
9       회의가  회의/NNG + 가/JKS       NP_SBJ  10
10      열렸다. 열리/VV + 었/EP + 다/EF + ./SF  VP      0

deptree.txt.v3.training

1       그      그      MM      MM      _       2       DP      _       _
2       날      날      NNG     NNG     _       22      NP_AJT  _       _
3       지회    지회    NNG     NNG     _       4       MOD     _       _
4       에서    에서    JKB     JKB     _       5       MOD     _       _
5       는      는      JX      JX      _       22      NP_AJT  _       _
6       해직    해직    NNG     NNG     _       7       MOD     _       _
7       교사    교사    NNG     NNG     _       8       NP      _       _
8       비상    비상    NNG     NNG     _       9       MOD     _       _
9       대책    대책    NNG     NNG     _       10      MOD     _       _
10      회의    회의    NNG     NNG     _       11      MOD     _       _
11      이      이      VCP     VCP     _       12      MOD     _       _
12      ᆫ가     ᆫ가     EC      EC      _       13      VNP     _       _
13      하      하      VV      VV      _       14      MOD     _       _
14      는      는      ETM     ETM     _       20      VP_MOD  _       _
15      조금    조금    NNG     NNG     _       16      MOD     _       _
16      은      은      JX      JX      _       17      NP_AJT  _       _
17      살벌    살벌    NNG     NNG     _       18      MOD     _       _
18      하      하      XSA     XSA     _       19      MOD     _       _
19      ᆫ       ᆫ       ETM     ETM     _       20      VP_MOD  _       _
20      회의    회의    NNG     NNG     _       21      MOD     _       _
21      가      가      JKS     JKS     _       22      NP_SBJ  _       _
22      열리    열리    VV      VV      _       23      MOD     _       _
23      었      었      EP      EP      _       24      MOD     _       _
24      다      다      EF      EF      _       25      MOD     _       _
25      .       .       SF      SF      _       0       VP      _       _

sample output

10  열렸다. 열리/VV + 었/EP + 다/EF + ./SF  VP  0   ROOT    0   SUCCESS

How is 열렸다 chosen as the root when SyntaxNet trains on deptree v3? Is VV just considered automatically the root or is there some other mechanism?

Thanks for your hard work,
Andrew

[dsindex]

@xtknight

original sejong constituent tree is filtered by tree2con() in c2d.py
: sejong_treebank.txt.v1.* -> sejong_treebank.txt.v2.*

filtered sejong constituent tree is converted to dependency tree by tree2dep() in c2d.py
: sejong_treebank.txt.v2.* -> deptree.txt.v2.*

then, make eoj-based dependency tree to morph-based dependency tree via align.py
: deptree.txt.v2.* -> deptree.txt.v3.*

now, we have training data(deptree.txt.v3.*) for syntaxnet.

i thought your question is 'where ROOT comes from?' the answer is from the tree2dep() function.
there are a lot of rules in tree2dep(). but most important rule is 'head final'.
because korean language is head final, we can choose governor(head) of a word as its parent's most-right child. where parent should have right child(not equal to a word itself).

https://github.com/dsindex/blog/wiki/%5Bparsing%5D-visualizer-for-the-Sejong-Tree-Bank

; 가계부의 틀이 달라지고 있다.
(S  (NP_SBJ (NP_MOD 가계부/NNG + 의/JKG)
        (NP_SBJ 틀/NNG + 이/JKS))
    (VP (VP 달라지/VV + 고/EC)
        (VP 있/VX + 다/EF + ./SF)))

in this case, head of '틀이' is '있다'. but check_vx_rule() determines '있다' as auxiliary verb.
so '달라지고' is selected.

[xtknight]

Thanks for your response! Your explanation makes sense.
It seems like I trained the model (the copy_model part of train_sejong.sh succeeded).

However, I don't understand how to use the model. test.sh requires tagger-params(?) which I don't have, and I would like to enter a sentence like the English SyntaxNet and see a tree. I am wondering if this is possible with this treebank model. Do I need to use some other model first to get the part of speech tags, and then plug it into this model? (How did you test your TreeBank model?)

[dsindex]

@xtknight

unfortunately, it is not possible to tag sentences without having a Korean morphological analyzer.
when we want to analyze '가계부의 틀이 달라지고 있다', we should have a segmentation like '가계부/NNG+의/JKG 틀/NNG+이/JKO 달라지다/VV+고/EC 있다/VX+다/EP'.

if we have '먹은' with several possible segmentation '먹다+은', '먹+은' in the corpus, it can't be directly trained from Sejong tagged corpus using SyntaxNet.

so, i recommend you to use an available morphological analyzer :

http://eunjeon.blogspot.sg/
http://kkma.snu.ac.kr/documents/
http://konlpy.org/ko/v0.4.3/morph/

it worth to use.

by the way, i thought you have sejong corpus and it may differ from mine.
what is the eoj-based UAS? in my case, it is around 88%

[xtknight]

@dsindex
Sorry for my late response, I was gone for awhile. Plus, I had to train the model for a couple days.

I got Sejong TreeBank corpus (구문분석말뭉치?) from the sejong.or.kr miscellaneous files section by combining 15 different files BGAA0001.txt, BGAA0164.txt into a single UTF-8 file and then I wrote a script to try to remove invalid or weird things in the files.

I think the number you are asking for is accuracy 0.886990.

+ /usr/bin/python /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/eval.py -a /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/wdir/deptree.txt.v2.test -b /home/andy/Downloads/syntaxnet/models/syntaxnet/work/sejong/tmp_p/syntaxnet-output/brain_parser/structured/512,512-0.02-100-0.9/beam-parsed-test-corpus-eoj
can't compare different sentences
skip_sentences, total_sentences = 1, 3777
accuracy(UAS) = 0.886990
+ copy_model

I have used KoNLPy before. I am going to try to develop something to connect the morphological analyzer to this structured syntactic analyzer.

If I have the input tagged with POS tags like you mentioned, how do I run the model or which command should I use for creating the tree?
'가계부/NNG+의/JKG 틀/NNG+이/JKO 달라지다/VV+고/EC 있다/VX+다/EP'

I was experimenting with the following command, but I'm confused about what the input needs to be if I have '가계부/NNG+의/JKG 틀/NNG+이/JKO 달라지다/VV+고/EC 있다/VX+다/EP'

${BINDIR}/parser_eval \
--task_context=${TMP_DIR}/brain_parser/structured/${GP_PARAMS}/context \
--batch_size=${BATCH_SIZE} \
--hidden_layer_sizes=${HIDDEN_LAYER_SIZES} \
--beam_size=${BEAM_SIZE} \
--input=stdin \
--output=beam-parsed-${SET}-corpus \
--arg_prefix=brain_parser \
--graph_builder=structured \
--model_path=${TMP_DIR}/brain_parser/structured/${GP_PARAMS}/model \
--alsologtostderr

From analyzing English SyntaxNet, I thought I should input something like this:

1   프랑스   프랑스   NNP NNP _   0       _   _
2   의 의 JKG JKG _   0       _   _
3   세계  세계  NNG NNG _   0       _   _
4   적 적 XSN XSN _   0       _   _

But then, the resulting beam-parsed-test-corpus has some weird output like:

1       1       프랑스  프랑스  NNP     NNP     _       0               _       _       _                       _       0       ROOT    _       _

1       2       의      의      JKG     JKG     _       0               _       _       _                       _       0       ROOT    _       _

1       3       세계    세계    NNG     NNG     _       0               _       _       _                       _       0       ROOT    _       _

1       4       적      적      XSN     XSN     _       0               _       _       _                       _       0       ROOT    _       _

[dsindex]

@xtknight

i have made a modification for testing korean parser.
read https://github.com/dsindex/syntaxnet

if you make input as the sejong/tagged_input.sample, then it will work fine.

and for convenience, using Komoran tagger in konlpy, you can input raw sentence.

$ echo "나는 학교에 간다." | python sejong/tagger.py | ./test_sejong.sh
Input: 나 는 학교 에 가 ㄴ다 .
Parse:
. SF ROOT
 +-- ㄴ다 EF MOD
     +-- 가 VV MOD
         +-- 는 JX NP_SBJ
         |   +-- 나 NP MOD
         +-- 에 JKB NP_AJT
             +-- 학교 NNG MOD

[xtknight]

Thanks! I eventually figured this out. And, I ended up using Komoran too.

I have another question for you actually. After running on a custom corpus, I made a program to recombine parts of each eojeol, but as you can see, there is a problem with the word "위해" --> (위하아 VP). Komoran splits it into

10  위하  _   VV  VV  _   11  MOD _   _
11  아 _   EC  EC  _   12  VP  _   _

I can't figure out why it wants to do this. 위해 may be 위하+ㅏ or 위하+ㅓ, but python-jamo doesn't want to make 해 anyway. All I want to do is change this into "위해" in a consistent way.

---

I made a script to regenerate the tree based on parts. I think you regenerated the parts based on the corpus, but because this is a new sentence I wasn't able to do this. I had to recombine output from the POS tagger tree.

$ echo "수영할 때 눈을 보호하기 위해 쓰는 물안경은 렌즈의 굴절력과 고무 밴드의 내구성이 제품 선택의 포인트다." | ./demo.sh

Input: 수영할 때 눈을 보호하기 위하아 쓰는 물안경은 렌즈의 굴절력과 고무 밴드의 내구성이 제품 선택의 포인트다.
Parse:
포인트다.  ROOT
 +-- 물안경은  NP_SBJ
 |   +-- 쓰는  VP_MOD
 |       +-- 때  NP_AJT
 |       |   +-- 수영할  VP_MOD
 |       +-- 눈을  NP_OBJ
 |       +-- 위하아  VP
 |           +-- 보호하기  VP_OBJ
 +-- 내구성이  NP_SBJ
 |   +-- 굴절력과  NP_CNJ
 |   |   +-- 렌즈의  NP_MOD
 |   +-- 고무 밴드의  NP_MOD
 +-- 선택의  NP_MOD
     +-- 제품  NP

The intermediate step was this:

1   수영  _   NNG NNG _   2   MOD _   _
2   하 _   XSV XSV _   3   MOD _   _
3   ㄹ _   ETM ETM _   4   VP_MOD  _   _
4   때 _   NNG NNG _   12  NP_AJT  _   _
5   눈 _   NNG NNG _   6   MOD _   _
6   을 _   JKO JKO _   12  NP_OBJ  _   _
7   보호  _   NNG NNG _   8   MOD _   _
8   하 _   XSV XSV _   9   MOD _   _
9   기 _   ETN ETN _   10  VP_OBJ  _   _
10  위하  _   VV  VV  _   11  MOD _   _
11  아 _   EC  EC  _   12  VP  _   _
12  쓰 _   VV  VV  _   13  MOD _   _
13  는 _   ETM ETM _   14  VP_MOD  _   _
14  물안경   _   NNP NNP _   15  MOD _   _
15  은 _   JX  JX  _   29  NP_SBJ  _   _
16  렌즈  _   NNG NNG _   17  MOD _   _
17  의 _   JKG JKG _   18  NP_MOD  _   _
18  굴절  _   NNG NNG _   19  MOD _   _
19  력 _   NNG NNG _   20  MOD _   _
20  과 _   JC  JC  _   23  NP_CNJ  _   _
21  고무 밴드   _   NNP NNP _   22  MOD _   _
22  의 _   JKG JKG _   23  NP_MOD  _   _
23  내구  _   NNG NNG _   24  MOD _   _
24  성 _   XSN XSN _   25  MOD _   _
25  이 _   JKS JKS _   29  NP_SBJ  _   _
26  제품  _   NNG NNG _   27  NP  _   _
27  선택  _   NNG NNG _   28  MOD _   _
28  의 _   JKG JKG _   29  NP_MOD  _   _
29  포인트   _   NNG NNG _   30  MOD _   _
30  다 _   JX  JX  _   31  MOD _   _
31  .   _   SF  SF  _   0   ROOT    _   _

which gets converted to:

1   수영할   수영/NNG + 하/XSV + ㄹ/ETM  _   _   _   2   VP_MOD  _
2   때 때/NNG _   _   _   6   NP_AJT  _
3   눈을  눈/NNG + 을/JKO   _   _   _   6   NP_OBJ  _
4   보호하기    보호/NNG + 하/XSV + 기/ETN  _   _   _   5   VP_OBJ  _
5   위하아   위하/VV + 아/EC  _   _   _   6   VP  _
6   쓰는  쓰/VV + 는/ETM    _   _   _   7   VP_MOD  _
7   물안경은    물안경/NNP + 은/JX  _   _   _   14  NP_SBJ  _
8   렌즈의   렌즈/NNG + 의/JKG    _   _   _   9   NP_MOD  _
9   굴절력과    굴절/NNG + 력/NNG + 과/JC   _   _   _   11  NP_CNJ  _
10  고무 밴드의    고무 밴드/NNP + 의/JKG _   _   _   11  NP_MOD  _
11  내구성이    내구/NNG + 성/XSN + 이/JKS  _   _   _   14  NP_SBJ  _
12  제품  제품/NNG  _   _   _   13  NP  _
13  선택의   선택/NNG + 의/JKG    _   _   _   14  NP_MOD  _
14  포인트다.   포인트/NNG + 다/JX + ./SF   _   _   _   0   ROOT    _

[dsindex]

@xtknight
i think that we need to know eoj index of input sentence in addition to (morph, tag) by Komoran tagger for combining '위하','어' to '위해'. but it seems not supported yet.
(at least i couldn't find it)

there are also rules for combining root and functional word(eomi). but those are difficult to implement.
so i am not willing to recommend to you.

[xtknight]

Yes, I tried to combine them, but it's not working great. At komoran-2.0-master/KOMORAN_2.0_beta/corpus_build/dic.irregular there is a list of irregular forms. It fixed "위하+아" but it doesn't handle inflection(conjugation) of verbs. I've dealt with conjugation before so I can probably figure it out, but I'm curious why the original information about the complete word is destroyed.

What about training the model on the 어절 instead of all the individual parts? I mean training the model based on "선택의"(NP_MOD) instead of "선택/NNG + 의/JKG", for example. Is there a good reason to separate them? I am curious why they decided to separate them all in the corpus.

I made a sample here that attempts recombination. But it fails sometimes because of the eoj problems or disagreement between Komoran POS tagger and Sejong corpus.
http://ec2-52-78-70-112.ap-northeast-2.compute.amazonaws.com/syntaxnet/syntaxnet.htm
"수영할 때 눈을 보호하기 위해 쓰는 물안경은 렌즈의 굴절력과 고무 밴드의 내구성이 제품 선택의 포인트다."

My next project is to hack the Komoran tagger to return eoj index.
Apparently the latest version of Komoran is 2.4 (2014.11.24 ), but not sure if they give the source code for it, http://shineware.tistory.com/entry/KOMORAN-ver-24
Latest commit on their github appears to be 2014.12.25 so it must be update to date I guess?

[dsindex]

@xtknight
if you train model based on eoj form, model would suffer from data sparseness problem. so, it will not be recommended.
by the way, please let me know if you hack komoran for eoj index.
i use our in-house pos tagger which support every possible information including eoj index, eoj offset/length, morph offset/length(limited), and so on.
so i hope do same in komoran or other taggers.

[xtknight]

Okay, that makes sense.

There is another thing I am curious about. When eval.py runs it does not consider the phrase structure tag 'ptst' (like NP or NP_OBJ or VP) as part of the matching criteria. It seems like only seq, analyzed, and gov are important.

(sejong/eval.py)

def compare(entry_a, entry_b) :
    '''
    -1 : 비교 불가능
    0  : 다름
    1  : 같음

    entry : [seq, eoj, analyzed, ptst, gov]
    '''
    if entry_a[0] != entry_b[0] : return -1
    if entry_a[2].replace(' ','') != entry_b[2].replace(' ','') : return -1
    if entry_a[4] == entry_b[4] : return 1
    return 0

What is the reason for this? Is the ptst tag returned unimportant?

Because when I run the model on the following sentence I'm having trouble with the ptst tag:

서울에 4일 밤부터 100㎜ 넘는 폭우가 쏟아져 곳곳에서 도로함몰, 교통사고 등 비 피해가 속출했다.

...
17  도로  _   NNG NNG _   18  NP  _   _
18  함몰  _   NNG NNG _   19  MOD _   _
19  ,   _   SP  SP  _   20  NP_CNJ  _   _
...

I get this output from the model, but 도로 is marked as NP. But it seems like it should be MOD. It enters the model as NNG,NNG,SP.
According to Komoran it is marked like this:
Eojeol_parts[9]: 도로함몰, => [('도로', 'NNG'), ('함몰', 'NNG'), (',', 'SP')]

For my tree generation I expected that everything in a tagged Eojeol would be marked as MOD except for the last part (for example, MOD,MOD,NP_CNJ for 도로함몰). But I don't know why 도로 is being separated as NP. I wasn't aware the model would reclassify those parts.

[dsindex]

@xtknight

labeled attachment score(LAS) is less important than unlabeled attachment score(UAS) in practical reason. so, eval.py only check whether token's governor is correct or not. you can also modify it for measuring LAS(regarding ptst)

as you may know, Korean language is flexible for spacing within compound noun. for example, '도로함몰' or '도로 함몰' are acceptable. that is why trained model often mis-classify 'MOD' as 'NP'. but i think that is not important. if we get the eoj index from POS tagger, we can make it correct-classified.