How to resolve InvalidCommitOffsetSizeError?

[xzhuang000]

Hi everyone,
When I was trying to manually commit the consumer offset, I got this InvalidCommitOffsetSizeError error, is anyone familiar with this error and can give me some hints, e.g. what should I check? In addition, when the consumer does the auto commit, it also has the similar InvalidCommitOffsetSizeError issue.

     kafka_consumer.commit()
   File "/app/venv/local/lib/python2.7/site-packages/kafka/consumer/group.py", line 520, in commit
     self._coordinator.commit_offsets_sync(offsets)
   File "/app/venv/local/lib/python2.7/site-packages/kafka/coordinator/consumer.py", line 514, in commit_offsets_sync
     raise future.exception # pylint: disable-msg=raising-bad-type
 kafka.errors.InvalidCommitOffsetSizeError: [Error 28] InvalidCommitOffsetSizeError

Kafka-python version: 1.1.1
Kafka broker version: wurstmeister/kafka:2.12-2.3.0 (docker image provided by wurstmeister)

Thank you very much!

[dpkp]

From kafka/errors.py: 'This error indicates that an offset commit was rejected because of oversize metadata.' . metadata is any arbitrary string that a client associates with its commits. So unless you're explicitly setting metadata somehow, it should be empty. If you are setting it, the error is telling you that your metadata is too large.