public API refactoring

[mreiferson]

• consolidate the various inconsistent ways to configure certain Reader properties into an nsq.Config struct, with no public members, that exposes Set(key string, value interface{}) (this method will leverage the pre-existing Configure(...) code to coerce values into the right types)
• improve plumbing of and expose all usable Message commands (FIN, REQ, TOUCH)
• eliminate "async" interface (and associated FinishedMessage code)
• refactor the "sync" interface to require use of the Message.{FIN,REQ,TOUCH} methods rather than interpret the return value from the handler
• remove SendCommand and buffering such that users of Conn flush explicitly (and the Conn has a bufio.Writer)
• potentially rename Reader/Writer (which have pre-existing golang conventions) to Consumer/Producer
• simplify Writer error handling (so you dont have to check err and frameType)
• provide a simple logging level wrapper
• support "requeue without backoff"
• s/msg.Write/msg.WriteTo to satisfy WriterTo and remove msg.EncodeBytes() (you can use WriteTo)
• remove use of binary.{Read,Write} (slow)
• use SetHandler over AddHandler and add HandlerFunc convenience type
• cleanup some other exported methods like SetMaxInFlight, ConnectionMaxInFlight, etc.

The last two will result in a public API that mirrors the same changes we made to pynsq which some confusion and ambiguity.

[mreiferson]

RFR @jehiah @jphines @ploxiln - here's a quick example of some of the features of the new API

https://gist.github.com/mreiferson/11403980

[jehiah]

some of the logger stuff seems a little complicated.

One logging setup i've liked for it's exposed API is https://github.com/araddon/httpstream/blob/master/log.go#L29

which i feel like would translate here to

func (r *Reader) SetLogger(l *log.Logger, level string)

logLevel     = flag.String("logging", "debug", "Which log level: [debug,info,warn,error,fatal]")
reader.SetLogger(log.New(os.Stderr, "", log.Ldate|log.Ltime|log.Lshortfile), *logLevel)

I don't think we need to expose a way to handle or intercept individual messages, and this would be ideal for some of my uses where i embed multiple readers in a program and want to tag log lines with prefixes.

[mreiferson]

Yea, struggled coming up with a clean API for it. I'll see how your suggestion turns out.

[mreiferson]

RFR @jehiah - updated the logging API

[mreiferson]

@jehiah ping

FWIW @elubow has been using this branch for some consumers without issue in high volume situations, so it's been soaked.

[jehiah]

I've read through this several times, and i think i'm good. What do you want to do here to get this out?

[mreiferson]

I assume (hope) that most users of this package, at this point, are aware of the importance of pinning dependencies (whether vendored or managed). So I think the best choice is to plow ahead:

1. document/tag master as last stable backwards compatible revision of existing API
2. tag this as 1.0-alpha (maybe?)
3. merge this
4. new PR on the NSQ repo side which bumps the go-nsq dependency and updates all apps to use the new API

Agreed?

[mreiferson]

BTW, how do you feel about renaming Reader/Writer to Consumer/Producer to avoid the confusion with Go's naming idioms?

[jehiah]

Sounds good. Avoiding the confusion of those overloaded terms would be good. It's a naming change that should prob cary through to other client libraries

[mreiferson]

ok, bumped master to stable v0.3.7, RFM @jehiah

[jehiah]

I just realized this dropped SetMaxInFlight which is intentionally exported so that you have an api to pause and resume a Consumer.