Installing Kafka With Zookeeper

Overview

Kafka offers two installation methods:

• With Zookeeper (Classic method): Traditional setup where Zookeeper manages configurations, leadership elections, and coordination
• KRaft Mode: Introduced in Kafka 2.8, manages metadata without Zookeeper

This guide covers the traditional Zookeeper-based installation for Windows, Mac, Linux, and Docker.

Windows Installation

Prerequisites

• Download Kafka from Apache Kafka website
• Download and install Java (Oracle JDK)
• Use PowerShell or Git Bash for commands

Installation Steps

1. Create Kafka Directory

   - Create folder C:\kafka

   - Extract Kafka files into this folder

2. Set Environment Variables

   - KAFKA_HOME: Point to Kafka folder

   - PATH: Add Kafka to system PATH

   - JAVA_HOME: Point to Java installation directory

3. Start Zookeeper

   - Run Zookeeper start command

   - Wait for confirmation message

4. Start Kafka

   - Run Kafka server start command

   - Verify Kafka is running

Mac Installation

Prerequisites

• Homebrew installed
• Java (install via Homebrew if needed)

Installation Steps

1. Install Java

   bash

   1brew install java

2. Set JAVA_HOME

   - Add to profile configuration

   - Reload profile

   - Verify: java -version

3. Install Kafka

   bash

   1brew install kafka

   - Verify: Check Kafka version

4. Start Zookeeper

   - Run Zookeeper start command

   - Wait for info message confirming startup

5. Start Kafka

   - Run Kafka server start command

   - Verify startup confirmation message

Linux Installation

Installation Steps

1. Install Java

   bash

   1sudo apt update
   2sudo apt install openjdk-11-jdk
   3java -version

2. Set JAVA_HOME

   - Add to .bashrc file

   - Reload configuration

   - Confirm path is set

3. Download and Extract Kafka

   - Download from Apache Kafka website

   - Extract files

   - Move to /opt/kafka directory

   - Navigate into Kafka directory

4. Start Zookeeper

   - Run Zookeeper start command

   - Watch for info message

5. Start Kafka

   - Run Kafka server start command

   - Wait for confirmation message

Docker Installation

Setup Steps

1. Create Kafka Directory

   bash

   1mkdir ~/kafka
   2cd ~/kafka

2. Create docker-compose.yml

   yaml

   1version: '3.8'
   2services:
   3   zookeeper:
   4      image: confluentinc/cp-zookeeper:7.3.0
   5      environment:
   6      ZOOKEEPER_CLIENT_PORT: 2181
   7   kafka:
   8      image: confluentinc/cp-kafka:7.3.0
   9      ports:
   10      - "9092:9092"
   11      environment:
   12      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
   13      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092

   Configuration Details

   Zookeeper Service:

   • Uses Confluent Zookeeper image (version 7.3.0)
   • ZOOKEEPER_CLIENT_PORT: 2181: Default port for Zookeeper communication

   Kafka Service:

   • Uses Confluent Kafka image (version 7.3.0)
   • Exposes port 9092 for client connections
   • KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181: Connects to Zookeeper
   • KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092: Advertises broker address to clients

   Running Docker Compose

   bash

   1docker-compose up -d

   This starts both Zookeeper and Kafka services in detached mode.