Multiple connection slowdown

[davidfstein]

I've noticed when multiple concurrent connections are made to the same annotation archive, there is a significant slow down. My use case involves potentially significant concurrent processing. Is this the expected behavior? Do you know of any workaround?

[brentp]

Hi David, I would guess that this is due to disk speed. You could try putting the archive in multiple different drives and spreading the concurrent connections between them.

[davidfstein]

Got it that makes sense. Would you expect the disk issue to mainly be a bottleneck during the writing of the output or are there intermediate steps that could be problematic?

[brentp]

ah, yes, writing the VCF output would also be an issue. I was thinking of reading the archive from multiple sources stressing the disk due to random access.

[davidfstein]

Got it, thanks. Ill try to do some benchmarking and update this issue later.