Skip to content

Go error for a very large tar file #14

@rvencu

Description

@rvencu

I tried to split a 2.3TB tar file into many 1GB tar files. With tarp I got this error after 55 shards:

panic: runtime error: index out of range [1] with length 0

goroutine 19 [running]:
github.com/tmbdev/tarp/dpipes.FnameSplit({0xc0002201e0?, 0xc0005dc030?})
        /root/tarp/dpipes/rawtario.go:36 +0xd2
github.com/tmbdev/tarp/dpipes.Aggregate(0x0?, 0xc0003ec060)
        /root/tarp/dpipes/rawtario.go:55 +0x137
created by github.com/tmbdev/tarp/dpipes.TarSource.func1
        /root/tarp/dpipes/tario.go:55 +0x9e

At first I thought the tar is damaged but trying the old tarproc in python (installed with pip install tarproc) I was able to split the big tar. The speed dropped from 32 shards per minute to 18 shards per minute but I am happy my problem is solved.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions