半兽人 发表于: 2015-03-10   最后更新时间: 2024-02-19 10:18:54  
{{totalSubscript}} 订阅, 55,443 游览

You have the option of either adding topics manually or having them be created automatically when data is first published to a non-existent topic. If topics are auto-created then you may want to tune the default topic configurations used for auto-created topics.

Topics are added and modified using the topic tool:

bin/kafka-topics.sh --zookeeper zk_host:port/chroot --create --topic my_topic_name 
       --partitions 20 --replication-factor 3 --config x=y

The replication factor controls how many servers will replicate each message that is written. If you have a replication factor of 3 then up to 2 servers can fail before you will lose access to your data. We recommend you use a replication factor of 2 or 3 so that you can transparently bounce machines without interrupting data consumption.

The partition count controls how many logs the topic will be sharded into. There are several impacts of the partition count. First each partition must fit entirely on a single server. So if you have 20 partitions the full data set (and read and write load) will be handled by no more than 20 servers (no counting replicas). Finally the partition count impacts the maximum parallelism of your consumers. This is discussed in greater detail in the concepts section.

The configurations added on the command line override the default settings the server has for things like the length of time data should be retained. The complete set of per-topic configurations is documented here.

更新于 2024-02-19

不懂咖啡 1年前


半兽人 -> 不懂咖啡 1年前


River 5年前


半兽人 -> River 5年前


多想一点 5年前
bin/kafka-topics.sh --zookeeper zk_host:port/chroot --create --topic my_topic_name 
       --partitions 20 --replication-factor 3 --config x=y


半兽人 -> 多想一点 5年前

x=y 只是语法说明额。

多想一点 -> 半兽人 5年前

感谢你的回复,为什么我在1.1.1版本的kafka上加上--config x=y,他就不能创建topic啊