List all kafka topics

Question

I m using kafka 0 10 without zookeeper  I want to get kafka topics list  This command is not working since we re not using zookeeper   bin kafka-topics sh --list --zookeeper localhost 2181  How can I get the same output without zookeeper

User · Answer

The Kafka clients no longer require zookeeper but the Kafka servers do need it to operate.

You can get a list of topics with the new AdminClient API but the shell command that ship with Kafka have not yet been rewritten to use this new API.

The other way to use Kafka without Zookeeper is to use a SaaS Kafka-as-a-Service provider such as Confluent Cloud so you don’t see or operate the Kafka brokers (and the required backend Zookeeper ensemble).

For example on Confluent Cloud you would just use the following zookeeper free CLI command:

ccloud topic list

User · Answer

Kafka requires zookeeper and indeed the list of topics is stored there  hence the kafka-topics tool needs to connect to zookeeper too  kafka-clients apis in the newer versions no longer talk to zookeeper directly  perhaps that s why you re under the impression a setup without zookeeper is possible  It is not  as kafka relies on it internally  For reference see  http   kafka apache org documentation html quickstart Step 2      Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don t already have one

User · Answer

Kafka 2 2 and up  Newer versions of Kafka no longer requires ZooKeeper connection string to list topics  but can directly go via the Kafka brokers  kafka-topics sh is provided in the bin  folder when downloading Kafka  To list topics  do the following   bin kafka-topics sh --list --bootstrap-server  lt BROKER-LIST gt

User · Answer

Please use kafka-topics sh --list --bootstrap-server localhost 9092 to list down all topics

User · Answer

to see that topic if we run the list topic command      bin kafka-topics sh --list --zookeeper localhost 2181  To check if the data is landing in Kafka      bin kafka-console-consumer sh --zookeeper localhost 2181 --topic twitterstream --from-beginning

User · Answer

Kafka is a distributed system and needs Zookeeper  you have to start zookeeper too   Follow  Quick Start  here   https   kafka apache org 0100 documentation html quickstart

User · Answer

Zookeeper is required for running Kafka  zookeeper is must  still if you want to see topic list without zookeeper then you need kafka monitoring tool such as Kafka Monitor Tool  kafka-manager etc

User · Answer

To read messages you should use   kafka-console-consumer sh --bootstrap-server kafka1 9092 kafka2 9092 kafka3 9092 --topic messages --from-beginning   --bootstrap-server is required attribute  You can use only single kafka1 9020 node

User · Answer

Using Confluent s REST Proxy API  curl -X GET -H  quot Accept  application vnd kafka v2 json quot  localhost 8082 topics   where localhost 8082 is Kafka Proxy address

User · Answer

Commands    To start the kafka                    nohup   kafka bin kafka-server-start sh   kafka config server properties     kafka kafka log 2  amp 1  amp           To list out all the topic on on kafka                    bin kafka-topics sh  --list --zookeeper localhost 2181          To check the data is landing on kafka topic and to print it out                    bin kafka-console-consumer sh --bootstrap-server localhost 9092 --topic your topic name --from-beginning

User · Answer

You have a stale version of the package with commands that no longer accept zookeeper but rather bootstrap-server as the connection  Confluent will then connect with Zookeeper internally    https   www confluent io download   5 3 or greater

User · Answer

For dockerized kafka zookeeper  docker ps   find you zookeeper container id  docker exec -it  lt id gt  bash  cd bin    zkCli sh  ls  brokers topics

User · Answer

You need to start the zookeeper server first  So first go to kafka bin windows and run  zookeeper-server-start bat       config zookeeper properties   then in the same folder with a new cmd windows start the kafka servers by running  kafka-server-start bat       config server properties      Note  if you starting it for the first time then there are certain   changes to be made in these files   then inside kafka bin windows run  kafka-topics bat --zookeeper localhost 2181 --list   to list down all the topics existing

User · Answer

Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don t already have one    If you do not want to install and have a separate zookeeper server  you can use the convenience script packaged with kafka to get a quick-and-dirty single-node ZooKeeper instance    Starting the single-node Zookeeper instance    bin zookeeper-server-start sh config zookeeper properties   Starting the Kafka Server   bin kafka-server-start sh config server properties   Listing the Topics available in Kafka    bin kafka-topics sh --list --zookeeper localhost 2181

User · Answer

You can try using the below two command and list all Kafka topic  bin kafka-topics sh --describe --zookeeper 192 168 0 142 2181 192 168 9 115 2181 192 168 4 57 2181 bin kafka-topics sh --zookeeper 192 168 0 142 2181 192 168 9 115 2181 192 168 4 57 218 --list

[apache-kafka] List all kafka topics

Examples related to apache-kafka

Examples related to apache-zookeeper