csvsort memory limit

[simogeo]

Hi,
I've tried to sort a 1 Gb (only) file with csvsort and -c option ... My computer ends up freezing and using all memory (12 Gb)

Is that "normal" ? is that a known issue ? I've also seen issues #581 and #338.
Is there any tip to do so. Thanks

[jpmckinney]

Is it faster with --no-inference?

[simogeo]

I'll try it soon ... but it looks like a memory leak more than a performance issue ...

[simogeo]

same behaviour with no-inference option.

Here is the whole command line :

csvsort -d ";" --no-inference  -c DCLT ./FD_MOBPRO_2011.txt > ./FD_MOBPRO_2011-order-by-lieu-de-travail-test.txt

[onyxfish]

@simogeo You could try this with the 1.0.0 release, but realistically sorting a 1GB file is unlikely to ever be performant with csvkit. That's pretty far outside the intended use-case.

[simogeo]

got it. Thanks