Changing Kafka Server logs directory

Change where Kafka Server logs are stored

Estimated reading time: 2 minutes

Kafka Broker logs are invaluable for troubleshooting issues in the Kafka cluster. They provide insights into broker operations, message flows, and errors. This post explains changing the directory location where the broker process writes its logs.

Note: This post explains the steps to change the location of the Kafka Broker Process logs. Do not confuse this configuration with the topic data directory configuration, also called “logs” in Apache Kafka.

Updating the Kafka Server logs config property

Follow these steps to update the Kafka Server Logs directory in the file:

Step 1: Stop the broker

Stop the Kafka broker before making configuration changes to ensure no new data is written during modification.

Step 2: Update the File

Locate and open the file in your Kafka configuration directory. This file contains all the configuration settings for the Kafka broker.

Step 3: Change the Kafka Broker Logs directory

Find the kafka.log4j.dir property in the file. This property defines where Kafka Broker process log files are written. Example:

Step 3: Restart the Kafka Broker and verify

After making the changes to the configuration file, restart the broker and verify that the new Kafka broker logs are written in the new directory.

To verify the changes, you can tail the log file:

Note: Implement this change across all the brokers that require the process logs directory to be changed.

Before you go

It is advisable not to store the logs on the root mount point when changing the default Kafka Broker Log directory location. If the root mount point fills up, it can lead to issues on the server. Also, be careful when updating the configuration. A similar property, log.dirs defines where the Topic data is stored. Double-check before committing any changes.

Further reading

Leave a Reply

Your email address will not be published. Required fields are marked *