Pretrained model for img2txt? #466
Comments
|
@cshallue: could you comment on this? Thanks. |
concretevitamin
added
the
stat:awaiting model gardener
|
+1 |
|
Sorry, we're not releasing a pre-trained version of this model at this time. |
cshallue
removed
the
stat:awaiting model gardener
|
here are links to a pre-trained model:
|
|
@psycharo thanks for sharing! Perhaps you could also share your word_counts.txt file. Different versions of the tokenizer can yield different results, so your model is specific to the word_counts.txt file that you used. |
|
@psycharo my training is still training on our GPU instance. It seems it would take another two weeks to finish. I would appreciate it if you would also release the fine-tuned model. |
|
@psycharo Thanks for sharing your checkpoint! When I try to use it I'm getting the error: "ValueError: No checkpoint file found in: None". Suggestions welcomed. Thanks all. |
|
@ProgramItUp Try the following: --checkpoint_path=/tmp/checkpoint_tmp/model.ckpt-2000000 When you pass a directory, it looks for a "checkpoint state" file in that directory, which is an index of all checkpoints in the directory. Your directory doesn't have a checkpoint state file, but you can just pass it the explicit filename. |
|
Getting better, but... |
|
Looks like the word_counts.txt file above is not formatted as expected: vocabulary.py expects: A quick fix is to reformat the word_counts.txt in that way. Or, you could replace line 49 of vocabulary.py with In the long run, I'll come up with a way to make sure word_counts.txt is outputted the same for everyone. |
|
It works! http://stablescoop.horseradionetwork.com/wp-content/uploads/2013/10/ep271.jpg |
|
@PredragBoksic great! @psycharo , what version of python did you use to generate the word_counts.txt file? I expect the script to output lines of the form: not: |
|
I didn't generate the word_counts.txt file. I changed the line 49 as you suggested it, with: I have Python 2.7.12 on KUbuntu 16.04 with CUDA 8.0 and CUDNN 5.1 and GTX970. I would not know how to do it in Python, because I program in Java usually. Do you need some code to change that file? |
|
@PredragBoksic I'm asking the creator of that file. You can just keep using the workaround :) |
|
@cshallue python 3.5. I had to make a couple of dirty hacks to make it work on that version of python, this is why word_counts.txt looks different. |
|
@psycharo How many hours did this take to train? I think that people would appreciate what you shared more if you mentioned this. |
|
@PredragBoksic |
|
@cshallue Thanks for the prompt replies. Your suggestions worked. I was not able to follow the full execution path of the code: Where would be the right place to put a bit of error checking to make sure that the files In the case of the checkpoint file it would be helpful to throw an error if "checkpoint state" is not found. Thanks. |
|
There are already error checks for all those things. If no checkpoint state or no checkpoint file is found in --checkpoint_path, it will fail the check here. If --vocab_file doesn't exist it will fail the check here. If no files match --input_files then you will get the message "Running caption generation on 0 files matching..." and inference will exit: see here. |
|
I did not notice any meaningful error messages, for example when the image file was missing. I suppose that this functionality will be completed in the future. |
|
@cshallue: I am running the finetuning step of the optimization. What I noticed was that the loss function is not changing much for the initial 22000 steps. The loss is pretty much stuck at 2.40. I have attached the log file by pumping the stderr to a text file. Is the loss going to go significantly down in the remaining iterations? Or am I missing some "gotcha"? |
|
@siavashk The loss reported by the training script is expected to be really noisy: it reports on single batches of only 32 examples. Are you running the evaluation script on the validation files? We expect to see validation perplexity decreasing slowly. It decreases slowly because the model is already near optimal and because we use a smaller learning rate during finetuning. |
|
@cshallue Maybe I am overly anxious, 22000 steps is about 1% of the optimization. I am just worried that it has been three weeks since I started training this model, and it seems it is going to take another two weeks for it to converge. |
|
You won't be able to tell much from the training losses for a single batch any more. They will keep jumping around. You could always just use the model in its current form. It will probably be sensible. There is not much improvement after 1M steps of fine tuning. Or you could use the model shared in this thread above. |
|
@psycharo Thanks for sharing your checkpoint! excellent work!! |
|
I have successfully used the 1M model (model.ckpt-1000000) However I'm still struggling to use the fine-tuned 2M or 3M posted here. I've tried the solutions already discussed, but with no luck. I'm using: Tensorflow 1.3 (for gpu), CUDA 8, cudnn 5.1. (I have yet to try to downgrade to TF 1.0, could this work?). When using for simplex using the fine-tuned 2M model, as described posted by @psycharo, I get the errors discussed earlier: I can fix this issue by running the following code: However, when I try to run the evaluation using the new model2, I get the following error: Here is the full stacktrace Anyone have any idea how to make the fine-tuned model work? |
|
Thank you.. |
|
If someone is looking for the reformatted words_count file, here it is words_count.txt |
|
no,what l want to ask you is your github ” Implementation of GoogLeNet by chainer”-- googlenet/googlenet.py中的what is“nutszebra_chainer”,it have confused me long time,thank you again! here is your code websit: https://github.com/nutszebra/googlenet/blob/master/googlenet.py
发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用
发件人: Razin Shaikh<mailto:notifications@github.com>
发送时间: 2017年10月15日 16:09
收件人: tensorflow/models<mailto:models@noreply.github.com>
抄送: Subscribed<mailto:subscribed@noreply.github.com>
主题: Re: [tensorflow/models] Pretrained model for img2txt? (#466)
If someone is looking for the reformatted words_count file, here it is words_count.txt<http://s000.tinyupload.com/?file_id=00024493683399847909>
―
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#466 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AX91IJthW-f4tSibnAYgSxeylgX6-zxXks5ssb2vgaJpZM4KIXMv>.
|
|
Has anyone figured out how to export the im2txt trained model as a TensorFlow SavedModelBundle to be served by Tensorflow Serving? |
|
Has anyone meet the problem of UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte? Traceback (most recent call last): |
|
I don't know why I send the command: there have a error: |
|
@yh0903 To solve the unicode error, make sure the file is being read in binary mode in |
|
@psycharo hi, thank you for providing us with so great model. I want to ask you some question. Have you noticed how the performance changes when you finetuen the model. Is the whole models' performance increasing or first the model performance(cider or bleu) drops a little, then it gradually increase. |
|
Hi all, Tensorflow version is 1.0.1, python 2.7 This is console output: universal@universal-ubuntu:~/anaconda3/envs/MyGAN$ python test.py Caused by op u'save/RestoreV2_379', defined at: NotFoundError (see above for traceback): Key lstm/logits/biases not found in checkpoint And this is my code for testing: ` ` What has gone wrong? When I generate cattions by runnung run_inference.py file, everything is OK. But I need to create my own model based on Im2Txt so I want to know how it works. Thank you in advance |
|
Hello, I am running the script " bazel-bin\im2txt\run_inference --checkpoint_path=${CHECKPOINT_DIR} --vocab_file=${VOCAB_FILE} --input_files=${IMAGE_FILE}" using python 3.5.2 under windows 7. |
|
I am running the script " bazel-bin\im2txt\run_inference --checkpoint_path=${CHECKPOINT_DIR} --vocab_file=${VOCAB_FILE} --input_files=${IMAGE_FILE}" using python 3.5.2 (downloaded with Anaconda 3) under windows 7. |
Just my thoughts; I do hope you sort this out! Post back here with additional detail, and perhaps someone can help. :-) |
|
@victoriastuart thank you Python 3.5.5 I got the following error: Any advice? thank you |
|
@JZakraoui Seems like variable names for basic_lstm_cell were changed again. You can change the variable name as pointed out by @cshallue. Copying his code however notice the variable names It works for me with |
|
Can confirm that @vpaharia latest fix works. Steps to follow on Python 3.5.5, TF-gpu 1.8:
In my case I created models directory where I extracted @psycharo learned models, I have also put the above mentioned script in this directory to fix the models (replaced paths with ./model.ckpt-2000000). I hope that this helps others, so that they don't have to look through all the posts :) |
|
@cshallue Thank you so much for your help! Here is a 5000000 step model using TF 1.9: |
|
Hey, thank you for the checkpoint files! I was wondering if anyone managed to use one of them to fine-tune the model with new data? How would the word counts file need to look like? Does the newly created word counts file from the new dataset need to be merged with the one from MSCOCO? I am currently running the fine tuning with a merged word counts file but encountering two problems: 1.) the captions after 20,000 steps just consist of the same word repeated over and over, despite a very small loss of 0.2 2.) I let the model fine-tune over night and somehow only the last 5 checkpoints got saved. Does anyone know how to prevent the overwriting of checkpoints and keep all of them? Thank you in advance! |
With this I could load the checkpoint file too, notice the variable names / values in the checkpoint file model.ckpt-2000000: tensor_name: lstm/BasicLSTMCell/Linear/Bias |
I've noticed that small amount of steps generate pretty bad results. I get one word responses and I'm around 366906 steps. I'm going to continue running and see how the results improve. I additionally had to follow the comment here: #7204 (comment) |
Hey! can you please tell us if the results improved? and how many steps did it take? I would be really grateful if you could share your log file as well! Thank you in advance! |
|
HI, python im2txt/run_inference.py --checkpoint_path="im2txt/model/train/newmodel.ckpt-2000000" --vocab_file="im2txt/data but get the error below can anyone help? |
|
got the same issue |
|
Got
Resolved? |
|
Hi @szhang12345, @fontes99 and @Vishwanath-Ayyappan where are you running the code? If you are running it locally as a package you need to add the the directory to the PYTHONPATH like this: export PYTHONPATH="${PYTHONPATH}:/home/users/thenameofyouruser/anotherlocation/Im2txt/im2txt". I suppose this is very much not relevant at the moment, but maybe someone else can make use of it in the future. |
and others
Please let us know which model this issue is about (specify the top-level directory)
models/img2txt
Can someone release a pre-trained model for the img2txt model trained on COCO? Would be great for someone here who doesn't have the computational resource yet to do a full training run. Thanks!
The text was updated successfully, but these errors were encountered: