Skip to content

Slow ipfs add #1216

Closed
Closed
@pr0d1r2

Description

Recently i started adding whole rubygems to IPFS (see: https://twitter.com/Pr0d1r2/status/596668070489264129/photo/1 & https://twitter.com/Pr0d1r2/status/596739849652203522/photo/1)

ipfs version 0.3.3

Whole process suppose to take 36h which is a lot of time for 155GB of data.

This is strange because machine is dedicated server (older version of https://www.hetzner.de/hosting/produkte_rootserver/ex40):
8 core: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
32GB of RAM (non ECC)
RAID1: /dev/md3 on /home type ext4 (rw,relatime,user_xattr,acl,barrier=1,data=ordered)

hdparm -tT /dev/md3

/dev/md3:
 Timing cached reads:   25722 MB in  2.00 seconds = 12876.63 MB/sec
 Timing buffered disk reads: 240 MB in  3.02 seconds =  79.59 MB/sec

CPU usage by ipfs daemon and ipfs add -r . processes is 5-15% (out of 800%):
screen shot 2015-05-09 at 05 47 54

Using recent version of go (go version go1.4.2 linux/amd64), downloaded from https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions