improved buf validation + hex support

[bucko13]

See this bcoin issue for context: bcoin-org/bcoin#463

Not sure how this should be structured but bval.buf can accept a hex. Maybe there should also be a bval.hex method as well or bval.buf should accept an optional encoding option to know how to read it and give more descriptive errors (i.e. if hex encoding should return error that says hex was expected).