kafka投入运行的ZooKeeper

原创
半兽人 发表于: 2015-03-10   最后更新时间: 2021-08-12 10:57:37  
{{totalSubscript}} 订阅, 21,866 游览

Operationally, we do the following for a healthy ZooKeeper installation:
在操作上,我们做以下健康的zookeeper安装方式:

  • Redundancy in the physical/hardware/network layout: try not to put them all in the same rack, decent (but don't go nuts) hardware, try to keep redundant power and network paths, etc.
    在物理,硬件,网络布局的冗余: 尽量不要把它们放在同一机架内,良好的(但不要发疯) 硬件,尽量保持冗余的电源和网络路径等等.

  • I/O segregation: if you do a lot of write type traffic you'll almost definitely want the transaction logs on a different disk group than application logs and snapshots (the write to the ZooKeeper service has a synchronous write to disk, which can be slow).
    I/0隔离:如果你有很多写操作,你最好把事务日志放在与应用日志、快照不同的一个磁盘组上。(写到zookeeper服务器会同步写磁盘,会慢点。)

  • Application segregation: Unless you really understand the application patterns of other apps that you want to install on the same box, it can be a good idea to run ZooKeeper in isolation (though this can be a balancing act with the capabilities of the hardware).
    应用隔离:zookeeper不要和别的应用安装在一起,除非你真的了解你想要安装在同一机器中其他应用的模式,最好单独运行zookeeper(尽管zookeeper是可以平衡使用硬件的性能的)

  • Use care with virtualization: It can work, depending on your cluster layout and read/write patterns and SLAs, but the tiny overheads introduced by the virtualization layer can add up and throw off ZooKeeper, as it can be very time sensitive
    小心使用虚拟化:可以使用,这取决于你的集群布局和read/write模式和SLA,但是微小的虚拟化层引入的开销加起来会断开zookeeper,因为它非常的敏感。

  • ZooKeeper configuration and monitoring: It's java, make sure you give it 'enough' heap space (We usually run them with 3-5G, but that's mostly due to the data set size we have here). Unfortunately we don't have a good formula for it. As far as monitoring, both JMZ and the 4 letter commands are very useful, they do overlap in some cases (and in those cases we prefer the 4 letter commands, they seem more predictable, or at the very least, they work better with the LI monitoring infrastructure)
    zookeeper配置和监控:它是java,首先确保你给它足够的堆空间(我们通常设置3-5G,这个配置主要是根据我们的情况下配置的),不幸的时我们没有它算法公式。至于监控,JMZ和4个字母的命令非常有用,它们某些情况下重叠(在这种情况下,我们更喜欢4个字母的命令,他们似乎更可预测的,或者至少,它们与LI监控基础设施更好的工作)

  • Don't overbuild the cluster: large clusters, especially in a write heavy usage pattern, means a lot of intracluster communication (quorums on the writes and subsequent cluster member updates), but don't underbuild it (and risk swamping the cluster).
    不要过度建设集群:大型集群,尤其是在写入沉重的使用模式,意味着很多内通讯(规定人数的写入和后续的集群成员的更新),但不underbuild它(和风险覆盖集群)。

  • Try to run on a 3-5 node cluster: ZooKeeper writes use quorums and inherently that means having an odd number of machines in a cluster. Remember that a 5 node cluster will cause writes to slow down compared to a 3 node cluster, but will allow more fault tolerance.
    尝试在3 - 5个节点集群上运行:Zookeeper写入使用规定人数,本质上这意味着在集群中有个奇数的机器。要知道,一个5节点的集群比3节点集群要慢,但将允许更多的容错能力。

Overall, we try to keep the ZooKeeper system as small as will handle the load (plus standard growth capacity planning) and as simple as possible. We try not to do anything fancy with the configuration or application layout as compared to the official release as well as keep it as self contained as possible. For these reasons, we tend to skip the OS packaged versions, since it has a tendency to try to put things in the OS standard hierarchy, which can be 'messy', for want of a better way to word it.
总体来看,我们尽量保持zookeeper尽可能小的处理负载 (标准增长容量规划) 并尽可能的简单。我们尽量不做什么花里胡哨的配置或应用程序的布局,相比,我们尽可能的保持使用官方版本的发布。基于这些原因,我们倾向于跳过操作系统打包的版本,因为它会把焦点集中在操作系统标准层次结构中。

更新于 2021-08-12

liebe41 5年前

今天页面上那个可以选章节的小标签不在了,不知道是不是静态文件加载出问题了

半兽人 -> liebe41 5年前

感谢,已修复。空指针异常。

查看kafka更多相关的文章或提一个关于kafka的问题,也可以与我们一起分享文章