Fix byte size estimation with kafka producer

[blakeembrey]

Unfortunately 1.4.x broke with this because it's accidentally sending the raw key and value instead of serialised byte strings.

[jeffwidman]

Makes sense to me at first glance. Be good to have @tvoinarovskyi double-check as he wrote this originally in fbea5f0 just in case he did this on purpose for some reason...

[tvoinarovskyi]

Wow, sorry about that.

[tvoinarovskyi]

Will take another look in the evening, but seems like a legit error )

[jeffwidman]

Also, we might also want to change the function signature of _estimate_size_in_bytes(), it looks misleading that the function is asking for key, value, so that would make more sense why this mistake happened when actually using the function... maybe rename the signature to key_bytes, value_bytes for clarity.

[jeffwidman]

Thanks for the fast update @blakeembrey.

_estimate_size_in_bytes() passes the values through to DefaultRecordBatchBuilder.estimate_size_in_bytes() or LegacyRecordBatchBuilder.estimate_size_in_bytes() both of which also have function signatures of key, value... so I'm afraid it's turtles all the way down.

Probably all 5 of these usages should get updated for clarity: https://github.com/dpkp/kafka-python/search?utf8=%E2%9C%93&q=estimate_size_in_bytes&type=

Sorry to nitpick here, it's just these functions are somewhat self-documenting, and since you're already knee-deep into it, might as well finish making it consistent all the way through.

[blakeembrey]

@jeffwidman I've updated more of the places where key, value is passed around and expected to be bytes. I believe that headers are also expected to be a list of tuples of bytes.

[jeffwidman]

@blakeembrey thanks! Looks like tests are failing?

Oops, tests are failing...

[blakeembrey]

I’ll have to look into it next week.

[tvoinarovskyi]

Please, leave the kafka/record part intact. It's the same naming as in Java.

[jeffwidman]

If that's true, then probably should just revert this back to the original fix of c8945c0 and drop the other commits. I'm fine with that, and sorry for the runaround @blakeembrey.

[blakeembrey]

@jeffwidman No problem, just deleted the extra commits.

[tvoinarovskyi]

gj