A 20/20 on what was being installed by Kafka Cookbook
NOTE: the hindsight series are based on old HowTos and are being ported to the more recent versions. Some of this info may be presently out of date until this porting effort is finished.
Apache Kafka is data mover framework for moving large amounts of data throughout your data centers. It also has support for extracting data from sources using its producer or connector framework, transforming this data as its moved, and tieing it to consumers using its consumer or connector API.
The use cases for Kakfa have been broaden since its inception, however at its start and what its best at is being used as a generalized replication log capable of moving data from one source, maintaining a full copy for any clients, and making the data readily available to those clients in a very efficient manner.
Misc – Partition Calculation
partitions: 360
with 360 partitions, the following # servers will be able to handle equal # of partitions
2,3,4,5,6,8,9,10,12,15,18,20,24 (13)
60: 2*2*3*5 - divisors 2,3,4,5,6,-,-,-,10, -,12, -, -,15, -, -, -, -,20, -, -, -, -, - (9) 120: 2*2*2*3*5 - divisors 2,3,4,5,6,-,8,-,10, -,12, -, -,15, -, -, -, -,20, -, -, -,24, - (11) 180: 2*2*3*3*5 - divisors 2,3,4,5,6,-,8,9,10, -,12, -, -,15, -, -,18, -,20, -, -, -, -, - (12) 240: 2*2*2*2*3*5 - divisors 2,3,4,5,6,-,8,-,10, -,12, -, -,15,16, -, -, -,20, -, -, -,24, - (12) 300: 2*2*3*5*5 - divisors 2,3,4,5,6,-,-,-,10, -,12, -, -,15, -, -, -, -,20, -, -, -, -,25 (10) 360: 2*2*2*3*3*5 - divisors 2,3,4,5,6,-,8,9,10, -,12, -, -,15, -, -,18, -,20, -, -, -,24, - (13) 420: 2*2*3*5*7 - divisors 2,3,4,5,6,7,-,-,10, -,12, -,14,15, -, -, -, -,20,21, -, -, -, - (12) 600: 2*2*2*3*5*5 - divisors 2,3,4,5,6,-,8,-,10, -,12, -, -,15, -, -, -, -,20, -, -, -, -,25 (11) 720: 2*2*2*2*3*3*5 - divisors 2,3,4,5,6,-,8,9,10, -,12, -, -,15,16, -,18, -,20, -, -, -,24, - (14) 840: 2*2*2*3*5*7 - divisors 2,3,4,5,6,7,-,-,10, -,12, -,14,15, -, -, -, -,20,21, -, -,24, - (13) 900: 2*2*3*3*5*5 - divisors 2,3,4,5,6,-,-,9,10, -,12, -, -,15, -, -,18, -,20, -, -, -, -,25 (12) 1200: 2*2*2*2*3*5*5 - divisors 2,3,4,5,6,-,8,-,10, -,12, -, -,15,16, -, -, -,20, -, -, -,24,25 (13) 1680: 2*2*2*2*3*5*7 - divisors 2,3,4,5,6,7,-,-,10, -,12, -,14,15,16, -, -, -,20,21, -, -,24, - (14) 2520: 2*2*2*3*3*5*7 - divisors 2,3,4,5,6,7,-,9,10, -,12, -,14,15, -, -,18, -,20,21, -, -,24, - (15)
22 3 5 - 60 222 3 5 - 120 2222 3 5 - 240
22 33 5 - 180 222 33 5 - 360 2222 33 5 - 720
22 3 55 - 300 222 3 55 - 600 2222 3 55 - 1200
22 3 5 7 - 420 222 3 5 7 - 840 2222 3 5 7 - 1680
22 33 5 7 - 1260 222 33 5 7 - 2520 2222 33 5 7 - 5040