Kafka on Azure: Event Hubs, HDInsight, Confluent, or self-managed?
There are four ways to run Kafka on Azure: point existing Kafka clients at Azure Event Hubs, which speaks the Kafka protocol; run open-source Apache Kafka on Azure HDInsight; buy Confluent Cloud or another vendor through the Azure Marketplace; or self-manage brokers on AKS or virtual machines. Event Hubs is the default for most teams, and the other three exist to buy back Kafka features it lacks.
Azure has no product called "Azure Kafka", which is why the search leads to comparison pages. The right question is how much of Kafka you need: the wire protocol and the client libraries, or the whole ecosystem of Connect, Streams, and cluster-level tuning. This guide takes the four options in order of how little you have to operate, and ends with when to skip Kafka entirely.
Azure Event Hubs: Kafka clients without a Kafka cluster
Event Hubs exposes a Kafka-compatible endpoint on its Standard, Premium, and Dedicated tiers (not Basic). Any client that speaks Kafka protocol 1.0 or later connects by changing three settings: the bootstrap server, the security protocol, and the credentials. The concepts map one to one: a namespace is a cluster, an event hub is a topic, and partitions and consumer groups keep their names.
bootstrap.servers=NAMESPACE.servicebus.windows.net:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="$ConnectionString" \
password="Endpoint=sb://NAMESPACE.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=...";
Production deployments should swap the connection string for OAuth with a Microsoft Entra identity, but the shape of the change is the same: no code, only configuration.
The limits are where the tiers matter. Standard keeps events for up to 7 days and allows 32 partitions per event hub; Premium and Dedicated raise retention to 90 days and add far more partitions. Capacity on Standard is bought in throughput units of 1 MB/s in and 2 MB/s out. Kafka Streams and Kafka Connect are client-side libraries, so they run on compute you provide rather than inside Event Hubs, and support for some features depends on tier. Check Microsoft's Kafka compatibility list before assuming transactions or a specific compression codec will work.
Pick Event Hubs when your producers and consumers are ordinary Kafka clients and you want ingestion that scales with a slider. For the head-to-head with a real cluster, see Kafka vs Azure Event Hubs.
Azure HDInsight: open-source Kafka on managed virtual machines
HDInsight provisions a genuine Apache Kafka cluster on VMs inside your subscription. You get the full project: Kafka Connect, Kafka Streams, MirrorMaker, ACLs, and every broker setting, with HDInsight handling provisioning, OS patching, and Azure Monitor integration. Brokers are spread across fault and update domains with managed disks attached, so a single host failure does not take a partition offline.
What you still own is everything a Kafka operator owns: partition counts, retention and disk sizing, consumer lag, and rebalances. Upgrades between Kafka versions typically mean standing up a new cluster and migrating. You also pay per node hour whether the cluster is busy or idle, so HDInsight makes sense for steady, high-volume workloads that need Kafka features Event Hubs does not offer, run by a team that does not want to operate Kubernetes.
Confluent Cloud and other vendors through the Marketplace
Confluent Cloud is available as an Azure Native ISV service: you create it from the Azure portal, it bills through your Azure subscription, and the data stays in the Azure region you choose. In return you get the Confluent additions on top of Kafka, such as Schema Registry, a managed connector catalog, stream processing with ksqlDB or Flink, and tiered storage that makes long retention affordable. Aiven, Redpanda Cloud, and WarpStream sell similar managed Kafka or Kafka-compatible services through the Marketplace.
The tradeoff is price per gigabyte, which is higher than Event Hubs, against the time your team would spend assembling those pieces around HDInsight or a self-managed cluster. Our Kafka vs Confluent comparison covers what you are paying for.
Self-managed Kafka on AKS or VMs
Running your own brokers gives total control and the lowest raw compute cost at scale, and the highest people cost. On AKS the usual route is the Strimzi operator, which manages brokers, topics, and users as Kubernetes resources. Kafka 3.3 and later can run in KRaft mode, so a new cluster no longer needs ZooKeeper. Choose this when you already have a platform team running Kubernetes, need a network topology or a Kafka version no managed service offers, or run at a volume where managed pricing stops making sense.
Which option to choose
| Option | You operate | Kafka compatibility | Best for |
|---|---|---|---|
| Event Hubs Kafka endpoint | Nothing | Protocol only; some features tier-dependent | Standard clients, elastic ingestion, teams new to Kafka |
| HDInsight | Brokers, topics, upgrades | Full open-source Kafka | Steady high-volume workloads needing Connect or Streams |
| Confluent Cloud and vendors | Nothing | Full Kafka plus vendor extras | Teams that want the whole ecosystem managed |
| Self-managed on AKS or VMs | Everything | Full Kafka, any version | Platform teams with special network or version needs |
Start with Event Hubs unless you can name the Kafka feature you need that it lacks. Partition layout matters on every option, so read our Kafka partition strategy guide before you size a namespace or a cluster.
When you do not need Kafka on Azure at all
Many "Kafka on Azure" searches come from teams that want a queue, not a stream. If consumers should each receive a message once and the message can be deleted afterwards, Azure Service Bus is simpler and cheaper; Kafka vs Azure Service Bus and our Azure message queue guide cover that choice. Kafka pays off when you need replay, many independent consumer groups, or ordered partitions at high throughput, which is what using Kafka as a message queue explains in detail.
The other case is events that leave your system. If a topic's ultimate destination is your customers' HTTP endpoints, Kafka is the internal pipe and delivery becomes a webhook problem: per-endpoint retries, HMAC signatures, secret rotation, and a searchable delivery log. Svix handles that layer regardless of which Kafka option you choose, so the topic stays a topic and customers get reliable webhooks.
Frequently asked questions
Does Azure have a managed Kafka service?
Not under that name. Azure Event Hubs exposes a Kafka-compatible endpoint that existing Kafka clients can use with a configuration change, and Azure HDInsight runs open-source Apache Kafka on managed virtual machines. Confluent Cloud is also sold as an Azure Native ISV service through the Marketplace.
Is Azure Event Hubs the same as Kafka?
No. Event Hubs is a separate service that implements the Kafka wire protocol on its Standard, Premium, and Dedicated tiers. Producers and consumers work unchanged, but retention, partition limits, and feature support depend on the tier, and Kafka Streams or Connect run on compute you provide.
Can I move from Event Hubs to a real Kafka cluster later?
Yes, because clients only need a new bootstrap server and credentials. Plan the partition count and consumer group names on Event Hubs the way you would on Kafka, and migration is mostly a matter of replaying or dual-writing during the cutover.
Ready to send webhooks?
Svix handles signing, retries, rate limiting, and delivery observability for the webhooks you send to your users, so your team can stay focused on your product.
Start sending webhooks with Svix or read the build vs. buy analysis