Increase default valueThreshold from 32B to 1KB

[jarifibrahim]

Increase value threshold from 32 bytes to 1 KB to allow easier space
reclaimation. There have been multiple reports about value log garbage
collection being unsatisfactory. This commit increases the default value
of threshold from 32 bytes to 1 KB which would allow keys and values
upto 1 KB to be co-allocated in the LSM tree. This would allow GC to
reclaim space easily for values less than 1 KB.

---

This change is 

[jarifibrahim]

There was a race condition between this line and the flushDiscardStats function. Moving it before starting the doWrites go routine seems to have fixed the race.

[ashish-goswami]

Minor comment. Make sure CI is green before merging.

Reviewable status: 0 of 7 files reviewed, 1 unresolved discussion (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)

---

db.go, line 383 at r1 (raw file):

Previously, jarifibrahim (Ibrahim Jarif) wrote…

There was a race condition between this line and the flushDiscardStats function. Moving it before starting the doWrites go routine seems to have fixed the race.

Not able to understand this. We are already initialising it while creating db instance.

	db = &DB{
		imm:           make([]*skl.Skiplist, 0, opt.NumMemtables),
		flushChan:     make(chan flushTask, opt.NumMemtables),
		writeCh:       make(chan *request, kvWriteChCapacity),
		opt:           opt,
		manifest:      manifestFile,
		dirLockGuard:  dirLockGuard,
		valueDirGuard: valueDirLockGuard,
		orc:           newOracle(opt),
		pub:           newPublisher(),
	}

If its necessary, please add a comment explaining it.

[jarifibrahim]

Reviewable status: 0 of 7 files reviewed, 1 unresolved discussion (waiting on @ashish-goswami and @manishrjain)

---

db.go, line 383 at r1 (raw file):

Previously, ashish-goswami (Ashish Goswami) wrote…

Not able to understand this. We are already initialising it while creating db instance.

	db = &DB{
		imm:           make([]*skl.Skiplist, 0, opt.NumMemtables),
		flushChan:     make(chan flushTask, opt.NumMemtables),
		writeCh:       make(chan *request, kvWriteChCapacity),
		opt:           opt,
		manifest:      manifestFile,
		dirLockGuard:  dirLockGuard,
		valueDirGuard: valueDirLockGuard,
		orc:           newOracle(opt),
		pub:           newPublisher(),
	}

If its necessary, please add a comment explaining it.

Actually, there was a second initialization that wasn't needed. I've raised a new PR to remove the second initialization.

[jarifibrahim]

The build failure will be fixed by #1382