Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow .gz files as input #2

Open
shahab-sarmashghi opened this issue Mar 14, 2018 · 3 comments
Open

Allow .gz files as input #2

shahab-sarmashghi opened this issue Mar 14, 2018 · 3 comments

Comments

@shahab-sarmashghi
Copy link
Owner

No description provided.

shahab-sarmashghi pushed a commit that referenced this issue Nov 17, 2021
@BenKuhnhaeuser
Copy link

Hi Shahab, I wonder whether this has yet been implemented? I'm working on a huge dataset and not having to unzip all fastq files would be super useful. Many thanks, Ben

@shahab-sarmashghi
Copy link
Owner Author

shahab-sarmashghi commented Aug 30, 2022

Hi Ben, ultimately inputs need to be decompressed since skmer runs jellyfish internally, and jellyfish doesn't support compressed inputs. I can add .gz input support to skmer, but be ware that effectively it would decompress the input, write it to a temp disk space, and then remove it after the processing, something that can be done using a wrapper (ex. bash) script too. I'll try to implement and test this when I find some time to work on it. I'll post here once done.

@BenKuhnhaeuser
Copy link

Hi Shahab, Thank you for providing these insights. It might be quite a computational burden if decompressing (and after running Skmer re-compressing) needs to be done in a single job, and maybe your suggestion of a wrapper script makes more sense. I've done that for now on my on task. But if you can find a straightforward way of dealing with compressed files using parallelisation (e.g. using pigz) then it might still be worthwhile as it would make Skmer a bit more user-friendly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants