Blog

March 21, 2025

Kafka 4: Changes and Upgrade Considerations

Joe Carder
Joe Carder

Apache Kafka

Apache Kafka 4, the much-anticipated next major release of the popular event streaming platform, is here. In this blog, find out what's changing in Kafka 4.0 and how to plan your next Kafka upgrade.

Editor's Note: This blog was originally published in December 2024 and updated in March 2025 after the GA release of Kafka 4.0.

Table of Contents

  1. Apache Kafka Project Update 
  2. Kafka 4 Release Date 
  3. Kafka 4 Changes 
  4. Kafka 4 Migration and Upgrade Considerations
  5. What to Expect in Future Kafka 4.x Releases
Back to top

Apache Kafka Project Update 

With four minor releases (3.6 through 3.9), several patches, and a major release on the horizon, 2024 was arguably the most eventful in the history of the Apache Kafka project. The biggest development, of course, is the release of Kafka 4 in early 2025, which we will discuss more in depth later in this blog. First, let's review the last few 3.x releases that contained significant updates related to some of the key changes coming in 4.0.

Most of the 3.x updates were made with the upcoming 4.0 Zookeeper deprecation in mind. ZooKeeper has been replaced by Kafka Raft (KRaft) mode and an official Zookeeper to KRaft migration process was introduced in 3.6 and designated as production ready in 3.7. Prior to 3.6, the only way to move to a KRaft-based Kafka cluster was a complete “lift and shift” process, which entailed installing a new KRaft-based cluster and then manually moving topics, producers, and consumers.

JBOD (Just a Bunch of Disks) support for migrating KRaft clusters also was added in 3.7, and some existing features got enhancements as well, such as improved client metrics and observability as defined in KIP-714 and early access to the next-gen consumer rebalancing protocol defined in KIP-848. Java 11 was also marked for deprecation in 3.7 and is longer be supported in 4.0.

With 3.8 and 3.9, Log4j appender was deprecated (and also targeted for removal in 4.0) and KIP-848 was promoted to preview status. There were also several improvements made to KRaft migration, and the quorum protocol implemented in KRaft. Support for dynamic KRaft quorums (as detailed in KIP-853) makes adding or removing controller nodes without downtime a much simpler process. With these improvements, Kafka 3.9 basically became the de facto “bridge release” to 4.0.   

Back to top

Kafka 4 Release Date 

Kafka 4.0 was released was on March 18, 2025.

Back to top

Kafka 4 Changes 

As expected based on the latter 3.x releases described above, the biggest changes in Kafka 4 are removals, all noteworthy, though some more monumental than others. 

Kafka Raft Mode (KRaft) Replaced ZooKeeper 

The most notable change in Kafka 4 is that you can no longer run Kafka with ZooKeeper, with KRaft becoming the sole implementation for cluster management. While KRaft mode was marked as production ready for new clusters in 3.3, a few key pieces were needed before ZooKeeper deprecation and removal could be implemented. With the introduction and refinement of the migration process and JBOD support, the Kafka development community feels that total removal of ZooKeeper is finally ready with 4.0.

MirrorMaker 1 Removed

While not as huge of an architectural shift as the ZooKeeper removal, MirrorMaker 1 support also went away in 4.0. Given that most organizations dropped  MirrorMaker 1 for MirrorMaker 2 quite some time ago, we expect this change to be less impactful to the Kafka ecosystem, but it is still notable nonetheless.

Kafka Components Logging Moved to Log4j2

With Log4j marked for deprecation in 3.8, 4.0 also marks the complete transition from Log4j to Log4j2. After the Log4Shell vulnerability was disclosed in late 2021, an industry-wide effort to move to Log4j2 was put into motion. For this reason, most organizations already have moved off of Log4j, so while still a noteworthy change, it should not be all that impactful (and if you are still using Log4j, your systems are already most likely pwned at this point!). 

Want More Kafka Insights? 

t

Get the Decision Maker's Guide to Apache Kafka for tips on partition strategy, using Kafka with Spark, security best practices, and more. 

Read Guide

Back to top

Kafka 4 Migration and Upgrade Considerations

There are definitely some considerations that should be taken into account when planning your KRaft migration. First, if this is your first foray into KRaft, don’t plan on retiring your entire ZooKeeper infrastructure anytime soon. Best practices dictate that organizations should be running dedicated controller nodes for production clusters, so your production infrastructure will most likely not change. For dev and integration/testing environments, running in mix-mode is fine, so you might see some infrastructure reclamation occurring in those environments.

Another major consideration is the upgrade path you will need to take. Since ZooKeeper is gone in 4.0, there will be no migration functionality associated with 4.0. So, for organizations still running Zookeeper on a Kafka version prior to 3.7, an interim upgrade to 3.9 would be required. Technically, with migration improvements introduced with 3.9, I’d recommend doing this interim step even for installations later than 3.7. The upgrade path would look something like:

3.x => 3.9 => ZK to KR migration => 4.0

Also of note is that Kafka 3.5 and later use a version of ZooKeeper that is not wire-compatible with version 2.4 and older. As such, for older Kafka clusters, a couple of additional interim steps will be required as well. You would need to upgrade to Kafka 3.4, and then upgrade the version of ZooKeeper to 3.8. That migration path might look something like this:

2.3 => 3.4 => ZK 3.8 => 3.9 => ZK to KR migration => 4.0

This should be an edge case since older versions prior to 2.4 should mostly be retired at this point.

Back to top

What to Expect in Future Kafka 4.x Releases

If past precedence is any indication of future plans, I believe we will see continued improvements for containerization support and metrics collection, as well as refinements in the KRaft migration process. In regards to consumer performance, the full release of KIP-848 will also bring significant changes. Moving the complexity of the rebalancing protocol away from clients into the Group Coordinator, with a more modern event-loop process, creates a more incremental approach to rebalancing, where group-wide synchronization events will no longer be required for all coordination events. 

Regardless, the future of Kafka looks pretty bright, with these enhancements likely to make the already popular event-streaming platform even better and more efficient. 

Are You Getting the Most Out of Your Kafka Deployments? 

OpenLogic provides technical support for Apache Kafka as well as managed services including 24-7 administration and performance monitoring. Talk to an Enterprise Architect today to explore your options. 

Kafka Support and Services

Additional Resources

Back to top