Pretrained model for img2txt?

[ludazhao]

Please let us know which model this issue is about (specify the top-level directory)

models/img2txt

Can someone release a pre-trained model for the img2txt model trained on COCO? Would be great for someone here who doesn't have the computational resource yet to do a full training run. Thanks!

[concretevitamin]

@cshallue: could you comment on this? Thanks.

[siavashk]

+1

[cshallue]

Sorry, we're not releasing a pre-trained version of this model at this time.

[psycharo-zz]

here are links to a pre-trained model:

• initial training (1m steps without finetuning inception). perplexity ~8.7
• finetuned (1m more steps). perplexity ~8.4
• word_counts.txt

[cshallue]

@psycharo thanks for sharing! Perhaps you could also share your word_counts.txt file. Different versions of the tokenizer can yield different results, so your model is specific to the word_counts.txt file that you used.

[siavashk]

@psycharo my training is still training on our GPU instance. It seems it would take another two weeks to finish. I would appreciate it if you would also release the fine-tuned model.

[ProgramItUp]

@psycharo Thanks for sharing your checkpoint!

When I try to use it I'm getting the error: "ValueError: No checkpoint file found in: None".
I don't have any trouble doing run_inference my own checkpoint files but I can't do it on yours. I've tried lots of things: adding a trailing "/", using absolute paths, relative paths, ..... Nothing seems to work.

Suggestions welcomed.
@cshallue - Any thoughts?

Thanks all.

Last login: Sat Oct 15 07:10:56 2016 from 3.202.121.241
user123@myhost:~$ ls -l /tmp/checkpoint_tmp/
total 175356
-rw-r--r-- 1 user123 user123  19629588 Oct 15 07:04 graph.pbtxt
-rw-r--r-- 1 user123 user123 149088120 Oct 15 07:04 model.ckpt-2000000
-rw-r--r-- 1 user123 user123  10675545 Oct 15 07:04 model.ckpt-2000000.meta
-rw-rw-r-- 1 user123 user123    156438 Oct 15 07:08 word_counts.txt
user123@myhost:~$  /data/home/user123/tensorflow_models/models/im2txt/bazel-bin/im2txt/run_inference   --checkpoint_path=/tmp/checkpoint_tmp   --vocab_file=/tmp/checkpoint_tmp/word_counts.txt   --input_files=${IMAGE_FILE}
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
Traceback (most recent call last):
  File "/data/home/user123/tensorflow_models/models/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/run_inference.py", line 83, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "/data/home/user123/tensorflow_models/models/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/run_inference.py", line 49, in main
    FLAGS.checkpoint_path)
  File "/data/home/user123/tensorflow_models/models/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/inference_utils/inference_wrapper_base.py", line 118, in build_graph_from_config
    return self._create_restore_fn(checkpoint_path, saver)
  File "/data/home/user123/tensorflow_models/models/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/inference_utils/inference_wrapper_base.py", line 92, in _create_restore_fn
    raise ValueError("No checkpoint file found in: %s" % checkpoint_path)
ValueError: No checkpoint file found in: None
user123@myhost:~$

[cshallue]

@ProgramItUp Try the following: --checkpoint_path=/tmp/checkpoint_tmp/model.ckpt-2000000

When you pass a directory, it looks for a "checkpoint state" file in that directory, which is an index of all checkpoints in the directory. Your directory doesn't have a checkpoint state file, but you can just pass it the explicit filename.

[PredragBoksic]

Getting better, but...

Traceback (most recent call last):
  File "/home/gamma/bin/models-master/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/run_inference.py", line 83, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "/home/gamma/bin/models-master/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/run_inference.py", line 53, in main
    vocab = vocabulary.Vocabulary(FLAGS.vocab_file)
  File "/home/gamma/bin/models-master/im2txt/bazel-bin/im2txt/run_inference.runfiles/im2txt/im2txt/inference_utils/vocabulary.py", line 50, in __init__
    assert start_word in reverse_vocab
AssertionError

[cshallue]

Looks like the word_counts.txt file above is not formatted as expected:

b'a' 969108
b'</S>' 586368
b'<S>' 586368
b'.' 440479
b'on' 213612
b'of' 202290
b'the' 196219
b'in' 182598
b'with' 152984
...

vocabulary.py expects:

a 969108
</S> 586368
<S> 586368
. 440479
on 213612
of 202290
the 196219
in 182598
with 152984
...

A quick fix is to reformat the word_counts.txt in that way. Or, you could replace line 49 of vocabulary.py with

reverse_vocab = [eval(line.split()[0]) for line in reverse_vocab]

In the long run, I'll come up with a way to make sure word_counts.txt is outputted the same for everyone.

[PredragBoksic]

It works!

http://stablescoop.horseradionetwork.com/wp-content/uploads/2013/10/ep271.jpg

Captions for image cb340488986cc40f8ec610348b7f5a24.jpg:
  0) a woman is standing next to a horse . (p=0.000726)
  1) a woman is standing next to a horse (p=0.000638)
  2) a woman is standing next to a brown horse . (p=0.000373)