Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replication of model prediction - deviation of ~2% random seeds missing #128

Open
partizanos opened this issue May 16, 2022 · 0 comments
Open

Comments

@partizanos
Copy link
Contributor

partizanos commented May 16, 2022

Hello,

I try to replicate the training and evaluation using LSTM network for the phenotype benchmark.
I would like to ask if there are specific random seeds set in your system for the model training. (for the folders I saw the code provided them but for training no)
ie
from tensorflow import set_random_seed set_random_seed(2)
I get very close results (+/- 0.02 deviation) and I suspect this might be the issue. But let me know if there was some change
image
screenshot refers to this version of the paper:https://arxiv.org/pdf/1703.07771.pdf

Some more potential reasons for non replicability

  • I believe my data folders have the correct size ie: train/test size: 35622/6282
  • Regarding the environment I have the following dependencies using conda .
    I faced issues with newer versions of python (I use python=3.7.12=hf930737_100_cpython)
    complete set of dependencies: deps.txt

Thank you again for your work and active maintenance of this repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant