Skip to content

Commit 4925ecd

Browse files
kubummanub
authored andcommitted
fix OutOfMemory issue by making the log cleaner dedupe buffer much smaller (#60)
Similar issue in KAFKA Reference 1: https://issues.apache.org/jira/browse/KAFKA-1881 Reference 2: https://reviews.apache.org/r/31447/diff/1/
1 parent ca287be commit 4925ecd

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

embedded-kafka/src/main/scala/net/manub/embeddedkafka/EmbeddedKafka.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,10 @@ sealed trait EmbeddedKafkaSupport {
311311
properties.setProperty("port", config.kafkaPort.toString)
312312
properties.setProperty("log.dir", kafkaLogDir.toAbsolute.path)
313313
properties.setProperty("log.flush.interval.messages", 1.toString)
314+
315+
// The total memory used for log deduplication across all cleaner threads, keep it small to not exhaust suite memory
316+
properties.setProperty("log.cleaner.dedupe.buffer.size", "1048577")
317+
314318
config.customBrokerProperties.foreach {
315319
case (key, value) => properties.setProperty(key, value)
316320
}

0 commit comments

Comments
 (0)