Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to change kafka log path #729

Open
luoluo2001108 opened this issue Oct 8, 2022 · 3 comments
Open

how to change kafka log path #729

luoluo2001108 opened this issue Oct 8, 2022 · 3 comments

Comments

@luoluo2001108
Copy link

luoluo2001108 commented Oct 8, 2022

I change the logs.dir in server.properties ,but does not take effect.the value will be change after each reboot with docker-compose down/up,,why ?

log.dirs=/kafka/kafka-logs-56a3b229e3a1---change the log.dirs does not take effect

how to fix the log path ,even reboot kafka by docker-compose down/up ?

@pseacrest
Copy link

Same question herre

@uriinf
Copy link

uriinf commented Dec 26, 2022

You can use KAFKA_LOG_DIRS environment variable to override detault

@opencmit2
Copy link

The default storage path for kafka configuration files is tmp/kafka logs. If you modify the kafka configuration file, the data storage path for kafka is as follows:
log.dirs=/mnt/kafka/data

In addition to storing data, there are also Kafka's running log files, so we'd better also migrate them to a large disk. The kafka log file is stored in the logs folder of the kafka installation directory by default. In v2.7.1 and earlier versions, modifications cannot be directly defined in the configuration file, but only the running file/kafka/bin/kafka run class.sh can be modified

Since log.dirs is in the plural form, it actually supports configuring multiple paths, because Kafka can "load evenly" store data in multiple directories. Of course, it must be mounted to multiple disks instead of multiple directories on the same disk. Attaching multiple disks and allowing multiple heads to perform write operations simultaneously will greatly improve the throughput of Kafka.
Kafka load balancing is measured relative to the number of partitions in Kafka, rather than based on disk space.

For example, you can specify two different disks (mnt1, mnt2) as the data storage directory, and modify the configuration file as follows:

log.dirs=/mnt1/kafka/data,/mnt2/kafka/data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants