Fixed fastavro not being used if schemas aren't manually specified…

[mparry]

… Don't force fastavro to re-parse from JSON every time; cache the result. Added a flag to disable message key decoding.

As the commit message summarises, this improves a few things. I'll elaborate:

1. Addresses fastavro is no longer used by default #616, where fastavro would no longer be used by default.
2. I also addressed Use a parsed schema in the cached decoder function #518 while I was working on the above, so that fastavro will no longer be forced to reparse the schema dict for every message.
3. Finally, I made the key decoding in AvroConsumer optional (but on by default). The main reason for this is that it is not, as far as I can tell, a requirement to encode both for a given topic. One very good reason for only Avro-encoding message values and not keys is that KSQL actually requires this! See Add support for generic message keys (e.g. message keys in Avro or JSON format, STRUCT) ksql#824, for example.

Potentially point 3 at least would be superseded by #502, if that's still planned. However, we needed a way of disabling this more immediately. Let me know if you would therefore like me to minimise this PR at all.

[ghost]

It looks like @mparry hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

[mparry]

We have attempted to sign the CLA as an organisation previously, but the bot's response suggests that it hasn't worked. Is there someone that I can contact about this, to try and figure out why? Thanks.

[swenzel]

I ran into that issue with fastavro not being used since after the last update our tests failed. The bad thing about python3-avro is that it is not only slower but also behaves differently on logical types which it keeps as bytes instead of turning them into floats (in case of decimal logical type).
This makes me think whether it's maybe better to explicitly choose whether to use a FastAvroConsumer or a "normal" AvroConsumer so you actually notice when it fails. But that's another story.
I look forward to seeing your pull request being merged!

[lowercase24]

Is this stalled?

[mparry]

I think that all of this is addressed in other, smaller PRs - respectively to my original bullet points: #773, #627, #787. (One is yet to be merged, but is still a better bet than mine now.)