Using Observer Zookeeper in Kafka DR scenarioExplaining Apache ZooKeeperRunning into LeaderNotAvailableException when using Kafka 0.8.1 with Zookeeper 3.4.6Is Zookeeper a must for Kafka?Kafka very slow/unresponsive on openstack - how to troubleshootZookeeper & Kafka error KeeperErrorCode=NodeExistsKafka replicas not in sync once a host is replacedKilling node with __consumer_offsets leads to no message consumption at consumersHow to specify advertised listeners for Kafka multi broker setup on kubernetes and expose the cluster expernally?Kafka roundtrip 2+ secondskafka and zookepeer high availability configuration
Why is the core ChaCha primitive not good for use in a CRCF? Why create BLAKE?
If צחק refers to the 3 worst misdeeds why Itzchak was called so?
Plot Dini's surface
How to spot dust on lens quickly while doing a shoot outdoors?
What does this docker log entry mean?
Does "solicit" mean the solicitor must receive what is being solicited in context of 52 U.S. Code Section 30121?
Ethics: Is it ethical for a professor to conduct research using a student's ideas without giving them credit?
Running code in a different tmux pane
Are we sinners because we sin or do we sin because we are sinners?
Cage Length (Rear Derallieur) and Total Capacity
Why are session states synchronized with high availability?
Was it possible for a message from Paris to reach London within 48 hours in 1782?
How can I add an ammeter and/or voltmeter to my home breaker panel?
"Startup" working hours - is it normal to be asked to work 11 hours/ day?
Is there any way to get an instant or sorcery on the field as a permanent? What would happen if this occurred?
Would an intelligent alien civilisation categorise EM radiation the same as us?
Does removing finitely many points from an open set yield an open set?
Can the UK and the EU delay Brexit again?
Where does the budget surplus of a conference go?
What latex template to use when you do not know the journal you are going to submit
Elevator design implementation in C++
Why does Greedo say "Maclunkey" in the Mos Eisley Cantina?
What other tricks were there to get more data onto floppy disks?
Bash script that shows changing real time values from commands
Using Observer Zookeeper in Kafka DR scenario
Explaining Apache ZooKeeperRunning into LeaderNotAvailableException when using Kafka 0.8.1 with Zookeeper 3.4.6Is Zookeeper a must for Kafka?Kafka very slow/unresponsive on openstack - how to troubleshootZookeeper & Kafka error KeeperErrorCode=NodeExistsKafka replicas not in sync once a host is replacedKilling node with __consumer_offsets leads to no message consumption at consumersHow to specify advertised listeners for Kafka multi broker setup on kubernetes and expose the cluster expernally?Kafka roundtrip 2+ secondskafka and zookepeer high availability configuration
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;
If we have a stretch kafka cluster of two DCs with 6 brokers and 5 zookeepers, split in below fashion:
DC1 : 3 zookeepers & 3 brokers ; DC2 : 2 zookeepers & 3 brokers
For a DR scenario in case, DC1 fails. How can we use a spare zookeeper, say an observer zookeeper(preferably in cloud) to come for the rescue and avoid a disaster?
Can an extra zookeeper serve a DR situation?
Suggestions welcome!
Thanks!
apache-kafka apache-zookeeper
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
add a comment
|
If we have a stretch kafka cluster of two DCs with 6 brokers and 5 zookeepers, split in below fashion:
DC1 : 3 zookeepers & 3 brokers ; DC2 : 2 zookeepers & 3 brokers
For a DR scenario in case, DC1 fails. How can we use a spare zookeeper, say an observer zookeeper(preferably in cloud) to come for the rescue and avoid a disaster?
Can an extra zookeeper serve a DR situation?
Suggestions welcome!
Thanks!
apache-kafka apache-zookeeper
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
add a comment
|
If we have a stretch kafka cluster of two DCs with 6 brokers and 5 zookeepers, split in below fashion:
DC1 : 3 zookeepers & 3 brokers ; DC2 : 2 zookeepers & 3 brokers
For a DR scenario in case, DC1 fails. How can we use a spare zookeeper, say an observer zookeeper(preferably in cloud) to come for the rescue and avoid a disaster?
Can an extra zookeeper serve a DR situation?
Suggestions welcome!
Thanks!
apache-kafka apache-zookeeper
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
If we have a stretch kafka cluster of two DCs with 6 brokers and 5 zookeepers, split in below fashion:
DC1 : 3 zookeepers & 3 brokers ; DC2 : 2 zookeepers & 3 brokers
For a DR scenario in case, DC1 fails. How can we use a spare zookeeper, say an observer zookeeper(preferably in cloud) to come for the rescue and avoid a disaster?
Can an extra zookeeper serve a DR situation?
Suggestions welcome!
Thanks!
apache-kafka apache-zookeeper
apache-kafka apache-zookeeper
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
edited Jul 1 at 11:26
Tony
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
asked Mar 28 at 22:09
TonyTony
2592 silver badges14 bronze badges
2592 silver badges14 bronze badges
add a comment
|
add a comment
|
1 Answer
1
active
oldest
votes
I'm not entirely sure how an extra Zookeeper will save you here.
If you had a replication factor of 3, then a DC is down, you're potentially losing entire topics that are on those brokers. Even if one replica is available, then any producer with acks=all or a topic config of min ISR = 2, will start causing issues
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
add a comment
|
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55407621%2fusing-observer-zookeeper-in-kafka-dr-scenario%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I'm not entirely sure how an extra Zookeeper will save you here.
If you had a replication factor of 3, then a DC is down, you're potentially losing entire topics that are on those brokers. Even if one replica is available, then any producer with acks=all or a topic config of min ISR = 2, will start causing issues
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
add a comment
|
I'm not entirely sure how an extra Zookeeper will save you here.
If you had a replication factor of 3, then a DC is down, you're potentially losing entire topics that are on those brokers. Even if one replica is available, then any producer with acks=all or a topic config of min ISR = 2, will start causing issues
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
add a comment
|
I'm not entirely sure how an extra Zookeeper will save you here.
If you had a replication factor of 3, then a DC is down, you're potentially losing entire topics that are on those brokers. Even if one replica is available, then any producer with acks=all or a topic config of min ISR = 2, will start causing issues
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
I'm not entirely sure how an extra Zookeeper will save you here.
If you had a replication factor of 3, then a DC is down, you're potentially losing entire topics that are on those brokers. Even if one replica is available, then any producer with acks=all or a topic config of min ISR = 2, will start causing issues
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
answered Apr 13 at 0:17
cricket_007cricket_007
95.8k11 gold badges59 silver badges141 bronze badges
95.8k11 gold badges59 silver badges141 bronze badges
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
add a comment
|
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
How about keeping 3 brokers in each DC?
– Tony
Apr 15 at 7:13
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
Sure, that would be better. I assume you've read the whitepapers on this? confluent.io/blog/…
– cricket_007
Apr 15 at 19:01
add a comment
|
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55407621%2fusing-observer-zookeeper-in-kafka-dr-scenario%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
