Error: Incomplete database file. Database building did not complete successfully

[mictadlo]

Hi,
I have just run into Error: Incomplete database file. Database building did not complete successfully. I used the following file as protein evidence:

NbenthamianaGenbankAA.clean.fasta.gz

  ./funannotate-docker predict -i NbLab330.genome.softmasked.fasta -o annotate --species LAB330 --protein_evidence NbenthamianaGenbankAA.clean.fasta -o annotate --ploidy 2 --cpus 12
-------------------------------------------------------
[Oct 26 03:26 AM]: OS: Debian GNU/Linux 10, 32 cores, ~ 132 GB RAM. Python: 3.8.12
[Oct 26 03:26 AM]: Running funannotate v1.8.10
[Oct 26 03:26 AM]: GeneMark not found and $GENEMARK_PATH environmental variable missing. Will skip GeneMark ab-initio prediction.
[Oct 26 03:26 AM]: Skipping CodingQuarry as no --rna_bam passed
[Oct 26 03:26 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco          
  glimmerhmm   busco          
  snap         busco          
[Oct 26 03:36 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
[Oct 26 03:38 AM]: Genome loaded: 20 scaffolds; 2,836,899,092 bp; 77.56% repeats masked
[Oct 26 03:38 AM]: Mapping 0 proteins to genome using diamond and exonerate
[Oct 26 03:38 AM]: CMD ERROR: diamond blastx --threads 8 -q /mnt/OneTB/annotation/funannotate/annotate/predict_misc/genome.softmasked.fa --db diamond -o diamond.matches.tab -e 1e-10 -k 0 --more-sensitive -f 6 sseqid slen sstart send qseqid qlen qstart qend pident length evalue score qcovhsp qframe
b'diamond v2.0.13.151 (C) Max Planck Society for the Advancement of Science\nDocumentation, support and updates available at http://www.diamondsearch.org\nPlease cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)\n\n#CPU threads: 8\nScoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)\nTemporary directory: \n#Target sequences to report alignments for: unlimited\nOpening the database...  [0s]\nError: Incomplete database file. Database building did not complete successfully.\n'
Traceback (most recent call last):
  File "/venv/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/venv/lib/python3.8/site-packages/funannotate/funannotate.py", line 710, in main
    mod.main(arguments)
  File "/venv/lib/python3.8/site-packages/funannotate/predict.py", line 1051, in main
    lib.exonerate2hints(Exonerate, hintsP)
  File "/venv/lib/python3.8/site-packages/funannotate/library.py", line 4026, in exonerate2hints
    with open(file, 'r') as input:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/OneTB/annotation/funannotate/annotate/predict_misc/protein_alignments.gff3'

Thank you in advance,

Best wishes,

Michal

[nextgenusfs]

Gunzip the proteins file and see if same error. Oh never mind i see in command it was not gzipped.

Not sure but perhaps it ran out of memory building the database?

[nextgenusfs]

Do you have the logfile as it will have slightly more information? Also if you leave the protein evidence blank and let it use uniprot does that also fail? That would tell us if it is the protein database file you passed or something else like out of memory.

[mictadlo]

funannotate-p2g.log
funannotate-predict.log

Furthermore, I am running it also with no evidence. How much memory is required?

[nextgenusfs]

I don't think it's a memory issue but more likely something up with the fasta file you fed it, like invalid characters, etc. As the default uniprot database is most likely larger than your file. I will try at some point today with your file.

[mictadlo]

I rerun it with the original Genbank FASTA file without any cleaning of the header and the error has changed.
NbenthamianaGenbankAA.fasta.gz

Now I got

0.016s]\nSearching alignments...  [0.122s]\nProcessing query block 1, reference block 1/1, shape 16/16, index chunk 4/4.\nBuilding reference seed array...  [0.003s]\nBuilding query seed array...  [6.257s]\nComputing hash join...  [0.981s]\nMasking low complexity seeds...  [0.022s]\nSearching alignments...  [0.124s]\nDeallocating buffers...  [0s]\nClearing query masking...  [0.36s]\nComputing alignments... '
Traceback (most recent call last):
  File "/venv/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/venv/lib/python3.8/site-packages/funannotate/funannotate.py", line 710, in main
    mod.main(arguments)
  File "/venv/lib/python3.8/site-packages/funannotate/predict.py", line 1051, in main
    lib.exonerate2hints(Exonerate, hintsP)
  File "/venv/lib/python3.8/site-packages/funannotate/library.py", line 4026, in exonerate2hints
    with open(file, 'r') as input:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/OneTB/annotation/funannotate/annotate/predict_misc/protein_alignments.gff3'

funannotate-p2g.log
funannotate-predict.log

What did I miss?

Thank you in advance

[hyphaltip]

its a 2GB genome so maybe some memory issues, I also made sure to funannotate: - I downloaded the plant busco from
https://busco-data.ezlab.org/v4/data/lineages/viridiplantae_odb10.2020-09-10.tar.gz

and Nben genome from
ftp://ftp.solgenomics.net/genomes/Nicotiana_benthamiana/assemblies/Niben.genome.v0.4.4.contigs.fasta.gz

I'll try to run on my side when I get a chance but 2Gb genomes do take resources so maybe your machine with ~132gb of RAM is reading limit when Diamond runs? Is your process run through a job manager (eg slurm) and getting a specific memory allocation that is less than 132Gb?

funannotate predict -i Niben.genome.v1.0.1.scaffolds.nrcontigs.fasta \
--protein_evidence NbenthamianaGenbankAA.clean.fasta --busco_seed_species arabidopsis \
--busco_db ./viridiplantae_odb10 --organism other --species 'Nicotiana benthamiana' \
-o Nben --ploidy 2 --cpus 8

[mictadlo]

Hi, we have a newer benth genome which is 2.8GB. If you think it would make any difference then I could send you a link to it. I ran it on Debian GNU/Linux 10, 32 cores, ~ 132 GB RAM. By any chance, do you have a singularity container?

[nextgenusfs]

It is possible the error in protein2genome mapping might be due to the recent change I made where I specified the default --tmpdir /tmp -- this running in docker might run out of space on the container. Can you try running command again but add --tmpdir . to the command?

EDIT: although this is probably not the case since the p2g.log says it errored on diamond. I'm not sure why. But you can go into the container and try to run those commands and see if you get more useful info than what is being captured in the logs.

[nextgenusfs]

I think @hyphaltip has some singularity containers customized to their system -- there is a recipe to convert the docker to singularity in the repo.

[mictadlo]

Hi, --tmpdir . has not helped. How can I login into the docker container and execute diamond?

[mictadlo]

Hi,
After login inside the funannotate docker container and executing the diamond commands I got Error: Error reading input stream at line 2: Invalid character (F) in sequence

I changed the last in funannotate-docker to exec docker run -ti --rm "${IT[@]}" --user "${USER}" -e TZ="${TZ}" --workdir "${WORKDIR}" --mount "${MOUNT}" nextgenusfs/funannotate:latest bash. Next, I ran the following commands:

$ ./funannotate-docker predict -i NbLab330.genome.softmasked.fasta -o annotate --species LAB330 --protein_evidence NbenthamianaGenbankAA.clean.fasta -o annotate --ploidy 2 --cpus 12 --tmpdir .
I have no name!@03f7b3ce6426:/mnt/OneTB/annotation/funannotate$ ls 
LAB330				      annotate
NbLab330.genome.softmasked.fasta      funannotate-docker
NbenthamianaGenbankAA.clean.fasta     funannotate-setup.log
NbenthamianaGenbankAA.clean.fasta.gz  p2g_3262a5da-9210-4108-8c02-589680e7fe7b
NbenthamianaGenbankAA.fasta	      viridiplantae_odb10
NbenthamianaGenbankAA.fasta.gz	      viridiplantae_odb10.2020-09-10.tar.gz

I have no name!@03f7b3ce6426:/mnt/OneTB/annotation/funannotate$ ls
LAB330				      annotate
NbLab330.genome.softmasked.fasta      funannotate-docker
NbenthamianaGenbankAA.clean.fasta     funannotate-setup.log
NbenthamianaGenbankAA.clean.fasta.gz  p2g_3262a5da-9210-4108-8c02-589680e7fe7b
NbenthamianaGenbankAA.fasta	      viridiplantae_odb10
NbenthamianaGenbankAA.fasta.gz	      viridiplantae_odb10.2020-09-10.tar.gz


I have no name!@03f7b3ce6426:/mnt/OneTB/annotation/funannotate$ diamond makedb --in NbenthamianaGenbankAA.fasta --db diamond -p 8
diamond v2.0.13.151 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org
Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)

#CPU threads: 8
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Database input file: NbenthamianaGenbankAA.fasta
Opening the database file...  [0s]
Loading sequences...  [0.008s]
Masking sequences...  [0.011s]
Writing sequences...  [0s]
Hashing sequences...  [0s]
Loading sequences...  [0s]
Writing trailer...  [0s]
Closing the input file...  [0s]
Closing the database file...  [0s]

Database sequences  1009
  Database letters  459890
     Database hash  4be250d90c5507a46010ef33b2213930
        Total time  0.022000s


I have no name!@03f7b3ce6426:/mnt/OneTB/annotation/funannotate$ diamond blastx --threads 8 -q NbenthamianaGenbankAA.fasta --db diamond -o diamond.matches.tab -e 1e-10 -k 0 --more-sensitive -f 6 sseqid slen sstart send qseqid qlen qstart qend pident length evalue score qcovhsp qframe
diamond v2.0.13.151 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org
Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)

#CPU threads: 8
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: 
#Target sequences to report alignments for: unlimited
Opening the database...  [0.001s]
Database: diamond.dmnd (type: Diamond database, sequences: 1009, letters: 459890)
Block size = 2000000000
Opening the input file...  [0s]
Opening the output file...  [0s]
Loading query sequences...  [0s]
Error: Error reading input stream at line 2: Invalid character (F) in sequence
I have no name!@03f7b3ce6426:/mnt/OneTB/annotation/funannotate$ diamond blastx --threads 8 -q NbenthamianaGenbankAA.fasta --db diamond -o diamond.matches.tab -e 1e-10 -k 0 --more-sensitive -f 6 sseqid slen sstart send qseqid qlen qstart qend pident length evalue score qcovhsp qframeAA

How is it possible to fix it?

Thank you in advance,

Michal

[nextgenusfs]

diamond query is your genome against the protein database, here you ran protein against protein.

[mictadlo]

Hi,
I am using diamond version 2.0.13 and according to

funannotate/funannotate/aux_scripts/funannotate-p2g.py

Lines 92 to 107 in fa4b1e2

	if lib.getDiamondVersion() >= '2.0.5':
	# run in frameshift mode
	cmd = ['diamond', 'blastx', '--threads', str(cpus), '-q', input,
	'--db', 'diamond', '-o', 'diamond.matches.tab', '-e', '1e-10',
	'-k', '0', '--more-sensitive', '--unal', '0', '-c', '1', '-F', '15',
	'-f', '6', 'sseqid', 'slen',
	'sstart', 'send', 'qseqid', 'qlen', 'qstart', 'qend', 'pident',
	'length', 'evalue', 'score', 'qcovhsp', 'qframe']
	else:
	if int(cpus) > 8:
	cpus = 8
	cmd = ['diamond', 'blastx', '--threads', str(cpus), '-q', input,
	'--db', 'diamond', '-o', 'diamond.matches.tab', '-e', '1e-10',
	'-k', '0', '--more-sensitive', '-f', '6', 'sseqid', 'slen',
	'sstart', 'send', 'qseqid', 'qlen', 'qstart', 'qend', 'pident',
	'length', 'evalue', 'score', 'qcovhsp', 'qframe']

it should have used the frameshift mode but looking at the logs it looks like it did not use it.

 funannotate predict -i NbLab330.genome.softmasked.fasta -o annotate --species LAB330 --protein_evidence NbenthamianaGenbankAA.clean.fasta -o annotate --ploidy 2 --cpus 4 --busco_db viridiplantae_odb10
-------------------------------------------------------
[Nov 17 10:44 PM]: OS: Debian GNU/Linux 11, 32 cores, ~ 132 GB RAM. Python: 3.8.12
[Nov 17 10:44 PM]: Running funannotate v1.8.9
[Nov 17 10:44 PM]: GeneMark not found and $GENEMARK_PATH environmental variable missing. Will skip GeneMark ab-initio prediction.
[Nov 17 10:44 PM]: Skipping CodingQuarry as no --rna_bam passed
[Nov 17 10:44 PM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco          
  glimmerhmm   busco          
  snap         busco          
[Nov 17 10:54 PM]: Loading genome assembly and parsing soft-masked repetitive sequences
[Nov 17 10:57 PM]: Genome loaded: 20 scaffolds; 2,836,899,092 bp; 77.56% repeats masked
[Nov 17 10:57 PM]: Mapping 0 proteins to genome using diamond and exonerate
[Nov 17 10:57 PM]: CMD ERROR: diamond blastx --threads 4 -q /mnt/OneTB/annotation/funannotate/annotate/predict_misc/genome.softmasked.fa --db diamond -o diamond.matches.tab -e 1e-10 -k 0 --more-sensitive -f 6 sseqid slen sstart send qseqid qlen qstart qend pident length evalue score qcovhsp qframe
b'diamond v2.0.13.151 (C) Max Planck Society for the Advancement of Science\nDocumentation, support and updates available at http://www.diamondsearch.org\nPlease cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)\n\n#CPU threads: 4\nScoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)\nTemporary directory: \n#Target sequences to report alignments for: unlimited\nOpening the database...  [0.004s]\nError: Incomplete database file. Database building did not complete successfully.\n'
Traceback (most recent call last):
  File "/mnt/OneTB/miniconda3/envs/funannotate/bin/funannotate", line 10, in <module>
    sys.exit(main())
  File "/mnt/OneTB/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/mnt/OneTB/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 1049, in main
    lib.exonerate2hints(Exonerate, hintsP)
  File "/mnt/OneTB/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/library.py", line 4026, in exonerate2hints
    with open(file, 'r') as input:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/OneTB/annotation/funannotate/annotate/predict_misc/protein_alignments.gff3'

How do make diamond use frameshift mode?

Thank you in advance

[mictadlo]

P.S.

 >>> '2.0.13' >= '2.0.5'
False

Therefore, it does not use frameshift mode. Maybe, it would be a good idea to run it always as frameshift mode.

[nextgenusfs]

I believe this is all fixed in master as I rewrote that script https://github.com/nextgenusfs/funannotate/blob/master/funannotate/aux_scripts/funannotate-p2g.py.

Upgrade with pip to install master from git to see if that fixes it.