Understand core components: Producers publish events to Topics, Consumers subscribe and process events, and Kafka Broker manages message storage and delivery; 2. Locally build Kafka: use Docker to quickly start ZooKeeper and Kafka services, expose port 9092; 3. Java integration Kafka: introduce kafka-clients dependencies, or use Spring Kafka to improve development efficiency; 4. Write Producer: configure KafkaProducer to send JSON format order events to orders topics; 5. Write Consumer: Subscribe to orders topics through KafkaConsumer, process messages asynchronously and implement business logic; 6. Follow best practices: standardize event structure, ensure consumer idempotence, reasonably naming topics, monitor consumption lag, and elegantly handle errors; 7. Optional Spring Boot extension: simplify development with @KafkaListener and KafkaTemplate to improve maintainability. This architecture achieves high scalability and responsiveness through event decoupling services, and is suitable for modern distributed systems.

Event-Driven Architecture (EDA) is a powerful design pattern that enables systems to communicate through events—changes in state that are published, routed, and consumed asynchronously. When combined with Java and Apache Kafka, it becomes a scalable, resilient, and high-performance solution for modern distributed applications.

Here's how you can implement Event-Driven Architecture using Java and Apache Kafka effectively.

1. Understanding the Core Components

Before jumping into code, understand the key pieces:

• Producers : Java applications that publish events (messages) to Kafka topics.
• Topics : Named streams of records; categories or feeds to which events are written.
• Consumers : Java applications that subscribe to topics and process events.
• Kafka Broker : The server that manages topics and stores messages.
• ZooKeeper / Kafka Raft (KRaft) : Coordinates brokers (ZooKeeper in older versions; KRaft in newer ones).

Events typically represent business actions like OrderCreated , PaymentProcessed , or UserRegistered .

2. Setting Up Apache Kafka Locally

You'll need Kafka running. Use Docker for quick setup:

 # docker-compose.yml
version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    Ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

Run with:

 docker-compose up -d

Now your Kafka broker is accessible at localhost:9092 .

3. Adding Kafka Dependencies in Java (Maven)

Use the official Kafka clients:

 <dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>3.7.0</version>
</dependency>

For better productivity, consider Spring Boot with Spring Kafka :

 <dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

But here we'll focus on raw Kafka clients to show core concepts.

4. Creating a Kafka Producer in Java

A producer sends events to a topic:

 import org.apache.kafka.clients.producer.*;
import java.util.Properties;

public class OrderProducer {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        Producer<String, String> producer = new KafkaProducer<>(props);

        String topic = "orders";
        String key = "order-123";
        String value = "{\"orderId\": \"123\", \"status\": \"CREATED\", \"amount\": 99.9}";

        ProducerRecord<String, String> record = new ProducerRecord<>(topic, key, value);

        producer.send(record, (metadata, exception) -> {
            if (exception != null) {
                System.err.println("Send failed: " exception.getMessage());
            } else {
                System.out.printf("Sent to %s partition %d offset %d%n",
                        metadata.topic(), metadata.partition(), metadata.offset());
            }
        });

        producer.flush();
        producer.close();
    }
}

This sends an OrderCreated event as JSON to the orders topic.

? Tip: Use Avro, Protobuf, or JSON Schema for structured, versioned events in production.

5. Creating a Kafka Consumer in Java

The consumer reads events from the same topic:

 import org.apache.kafka.clients.consumer.*;
import java.time.Duration;
import java.util.Collections;
import java.util.Properties;

public class OrderConsumer {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("group.id", "order-processing-group");
        props.put("auto.offset.reset", "earliest");
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

        Consumer<String, String> consumer = new KafkaConsumer<>(props);
        consumer.subscribe(Collections.singletonList("orders"));

        try {
            while (true) {
                ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
                for (ConsumerRecord<String, String> record : records) {
                    System.out.printf("Received: key=%s, value=%s, topic=%s, partition=%d, offset=%d%n",
                            record.key(), record.value(), record.topic(), record.partition(), record.offset());

                    // Process business logic here
                    handleOrderEvent(record.value());
                }
            }
        } finally {
            consumer.close();
        }
    }

    private static void handleOrderEvent(String value) {
        // Parse JSON and react accordingly
        System.out.println("Processing order event: " value);
        // eg, update DB, trigger payment service, send email
    }
}

Note:

• group.id : Enables consumer groups for scalability and failover.
• auto.offset.reset=earliest : Starts reading from beginning if no offset exists.

6. Best Practices for EDA with Kafka & Java

? Use meaningful topic names
eg, user-signups , payments-failed , not topic1 .

? Structure event data consistently
Include metadata like event type, timestamp, version:

 {
  "eventId": "abc-123",
  "eventType": "OrderCreated",
  "version": "1.0",
  "timestamp": "2025-04-05T10:00:00Z",
  "data": { "orderId": "123", "amount": 99.9 }
}

? Make consumers idealpotent
Since Kafka guaranteees at-least-once delivery, duplicates may occur.

? Scale horizontally
Add more consumers in the same group—they'll split partitions automatically.

? Monitor lag and throughput
Use tools like Kafka Manager, Confluent Control Center, or Prometheus Grafana.

? Handle errors gracefully
Don't crash on bad messages. Log, retry (with backoff), or send to a dead-letter topic.

7. Extending with Spring Boot (Optional but Recommended)

With Spring Kafka, things get simpler:

 @Component
public class OrderEventListener {

    @KafkaListener(topics = "orders", groupId = "order-processing-group")
    public void listen(String message) {
        System.out.println("Received via Spring: " message);
        // Business logic
    }
}

And auto-configured producer:

 @Autowired
private KafkaTemplate<String, String> kafkaTemplate;

public void sendOrderEvent(String key, String payload) {
    kafkaTemplate.send("orders", key, payload);
}

Much cleaner and integrates well with DI, logging, metrics, etc.

Final Thoughts

Implementing Event-Driven Architecture with Java and Kafka lets you build loosely coupled, scalable, and responsive systems. Start small—produce one event, consume it—and gradually expand to pipelines involving multiple services.

Key takeaways:

• Kafka acts as the central nervous system.
• Producers emit facts; consumers react.
• Design events around business semantics.
• Use async processing to decouple components.

It's not just about tech—it's about changing how your services think and talk to each other.

Basically, once you get the flow down, it scales beautifully.

The above is the detailed content of Implementing Event-Driven Architecture with Java and Apache Kafka. For more information, please follow other related articles on the PHP Chinese website!