Mixing
Want to do On-Premises Setup of Kafka cluster(confluent community platform) on Azure
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I want to do On-Premises Setup of Kafka cluster(confluent community platform) on Azure with 3 nodes and load balancer.
I want to use the confluent services like RestProxy, Schema Registry & Connectors(s3, elastic search, etc...)
Below reference, I have checked already
https://aws.amazon.com/blogs/big-data/best-practices-for-running-apache-kafka-on-aws/
https://medium.com/code-tech/kafka-in-aws-with-ssl-offloading-using-load-balancer-c337da1435c3
what would be the best architectural setup I should follow?
Approach I took
I followed the below architecture and did the setup on Azure.
azure apache-kafka confluent asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
|
show 2 more comments
I want to do On-Premises Setup of Kafka cluster(confluent community platform) on Azure with 3 nodes and load balancer.
I want to use the confluent services like RestProxy, Schema Registry & Connectors(s3, elastic search, etc...)
Below reference, I have checked already
https://aws.amazon.com/blogs/big-data/best-practices-for-running-apache-kafka-on-aws/
https://medium.com/code-tech/kafka-in-aws-with-ssl-offloading-using-load-balancer-c337da1435c3
what would be the best architectural setup I should follow?
Approach I took
I followed the below architecture and did the setup on Azure.
azure apache-kafka confluent asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
I'm voting to close this question as off-topic because it's asking for architecure and deployment recommendations.
– cricket_007
Jan 3 at 16:48
Also, S3 Connect won't work in Azure because S3 doesn't exist there. And "On-Premises" doesn't really mix with Azure, a cloud platform
– cricket_007
Jan 3 at 16:49
@cricket_007, Thanks for your response. I don't think, we can't post any architecturally related recommendation on this portal. Please recommend if you have any good approach but don't vote down. And "On-Premises", I mean to manage the kafka cluster on your own(maybe It's the wrong word in this case).
– Priyabrata
Jan 4 at 11:33
@cricket_007, Regarding S3 connect, It works wherever you do the setup of confluent kafka platform. In the S3 connect setup you have to pass your target S3 information(s3 bucket url and key)
– Priyabrata
Jan 4 at 11:35
See idownvotedbecau.se/noattempt ... Basically, you should try installing it following the resources you've already gathered, then come back with any problems you're running into. IMO, you don't require a load balancer, as it becomes a single point of failure, and you need to ensure it has higher uptime and redundancies than the services behind it (which is not provided by plain install of Confluent Platform)
– cricket_007
Jan 5 at 19:22
|
show 2 more comments
I want to do On-Premises Setup of Kafka cluster(confluent community platform) on Azure with 3 nodes and load balancer.
I want to use the confluent services like RestProxy, Schema Registry & Connectors(s3, elastic search, etc...)
Below reference, I have checked already
https://aws.amazon.com/blogs/big-data/best-practices-for-running-apache-kafka-on-aws/
https://medium.com/code-tech/kafka-in-aws-with-ssl-offloading-using-load-balancer-c337da1435c3
what would be the best architectural setup I should follow?
Approach I took
I followed the below architecture and did the setup on Azure.
azure apache-kafka confluent asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
I want to do On-Premises Setup of Kafka cluster(confluent community platform) on Azure with 3 nodes and load balancer.
I want to use the confluent services like RestProxy, Schema Registry & Connectors(s3, elastic search, etc...)
Below reference, I have checked already
https://aws.amazon.com/blogs/big-data/best-practices-for-running-apache-kafka-on-aws/
https://medium.com/code-tech/kafka-in-aws-with-ssl-offloading-using-load-balancer-c337da1435c3
what would be the best architectural setup I should follow?
Approach I took
I followed the below architecture and did the setup on Azure.
azure apache-kafka confluent azure apache-kafka confluent asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
edited Jan 9 at 5:46
Priyabrata
asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
asked Jan 3 at 11:10
PriyabrataPriyabrata
213112
213112
I'm voting to close this question as off-topic because it's asking for architecure and deployment recommendations.
– cricket_007
Jan 3 at 16:48
Also, S3 Connect won't work in Azure because S3 doesn't exist there. And "On-Premises" doesn't really mix with Azure, a cloud platform
– cricket_007
Jan 3 at 16:49
@cricket_007, Thanks for your response. I don't think, we can't post any architecturally related recommendation on this portal. Please recommend if you have any good approach but don't vote down. And "On-Premises", I mean to manage the kafka cluster on your own(maybe It's the wrong word in this case).
– Priyabrata
Jan 4 at 11:33
@cricket_007, Regarding S3 connect, It works wherever you do the setup of confluent kafka platform. In the S3 connect setup you have to pass your target S3 information(s3 bucket url and key)
– Priyabrata
Jan 4 at 11:35
See idownvotedbecau.se/noattempt ... Basically, you should try installing it following the resources you've already gathered, then come back with any problems you're running into. IMO, you don't require a load balancer, as it becomes a single point of failure, and you need to ensure it has higher uptime and redundancies than the services behind it (which is not provided by plain install of Confluent Platform)
– cricket_007
Jan 5 at 19:22
|
show 2 more comments
I'm voting to close this question as off-topic because it's asking for architecure and deployment recommendations.
– cricket_007
Jan 3 at 16:48
Also, S3 Connect won't work in Azure because S3 doesn't exist there. And "On-Premises" doesn't really mix with Azure, a cloud platform
– cricket_007
Jan 3 at 16:49
@cricket_007, Thanks for your response. I don't think, we can't post any architecturally related recommendation on this portal. Please recommend if you have any good approach but don't vote down. And "On-Premises", I mean to manage the kafka cluster on your own(maybe It's the wrong word in this case).
– Priyabrata
Jan 4 at 11:33
@cricket_007, Regarding S3 connect, It works wherever you do the setup of confluent kafka platform. In the S3 connect setup you have to pass your target S3 information(s3 bucket url and key)
– Priyabrata
Jan 4 at 11:35
See idownvotedbecau.se/noattempt ... Basically, you should try installing it following the resources you've already gathered, then come back with any problems you're running into. IMO, you don't require a load balancer, as it becomes a single point of failure, and you need to ensure it has higher uptime and redundancies than the services behind it (which is not provided by plain install of Confluent Platform)
– cricket_007
Jan 5 at 19:22
I'm voting to close this question as off-topic because it's asking for architecure and deployment recommendations.
– cricket_007
Jan 3 at 16:48
I'm voting to close this question as off-topic because it's asking for architecure and deployment recommendations.
– cricket_007
Jan 3 at 16:48
Also, S3 Connect won't work in Azure because S3 doesn't exist there. And "On-Premises" doesn't really mix with Azure, a cloud platform
– cricket_007
Jan 3 at 16:49
Also, S3 Connect won't work in Azure because S3 doesn't exist there. And "On-Premises" doesn't really mix with Azure, a cloud platform
– cricket_007
Jan 3 at 16:49
@cricket_007, Thanks for your response. I don't think, we can't post any architecturally related recommendation on this portal. Please recommend if you have any good approach but don't vote down. And "On-Premises", I mean to manage the kafka cluster on your own(maybe It's the wrong word in this case).
– Priyabrata
Jan 4 at 11:33
@cricket_007, Thanks for your response. I don't think, we can't post any architecturally related recommendation on this portal. Please recommend if you have any good approach but don't vote down. And "On-Premises", I mean to manage the kafka cluster on your own(maybe It's the wrong word in this case).
– Priyabrata
Jan 4 at 11:33
@cricket_007, Regarding S3 connect, It works wherever you do the setup of confluent kafka platform. In the S3 connect setup you have to pass your target S3 information(s3 bucket url and key)
– Priyabrata
Jan 4 at 11:35
@cricket_007, Regarding S3 connect, It works wherever you do the setup of confluent kafka platform. In the S3 connect setup you have to pass your target S3 information(s3 bucket url and key)
– Priyabrata
Jan 4 at 11:35
See idownvotedbecau.se/noattempt ... Basically, you should try installing it following the resources you've already gathered, then come back with any problems you're running into. IMO, you don't require a load balancer, as it becomes a single point of failure, and you need to ensure it has higher uptime and redundancies than the services behind it (which is not provided by plain install of Confluent Platform)
– cricket_007
Jan 5 at 19:22
See idownvotedbecau.se/noattempt ... Basically, you should try installing it following the resources you've already gathered, then come back with any problems you're running into. IMO, you don't require a load balancer, as it becomes a single point of failure, and you need to ensure it has higher uptime and redundancies than the services behind it (which is not provided by plain install of Confluent Platform)
– cricket_007
Jan 5 at 19:22
|
show 2 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54021155%2fwant-to-do-on-premises-setup-of-kafka-clusterconfluent-community-platform-on-a%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54021155%2fwant-to-do-on-premises-setup-of-kafka-clusterconfluent-community-platform-on-a%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I'm voting to close this question as off-topic because it's asking for architecure and deployment recommendations.
– cricket_007
Jan 3 at 16:48
Also, S3 Connect won't work in Azure because S3 doesn't exist there. And "On-Premises" doesn't really mix with Azure, a cloud platform
– cricket_007
Jan 3 at 16:49
@cricket_007, Thanks for your response. I don't think, we can't post any architecturally related recommendation on this portal. Please recommend if you have any good approach but don't vote down. And "On-Premises", I mean to manage the kafka cluster on your own(maybe It's the wrong word in this case).
– Priyabrata
Jan 4 at 11:33
@cricket_007, Regarding S3 connect, It works wherever you do the setup of confluent kafka platform. In the S3 connect setup you have to pass your target S3 information(s3 bucket url and key)
– Priyabrata
Jan 4 at 11:35
See idownvotedbecau.se/noattempt ... Basically, you should try installing it following the resources you've already gathered, then come back with any problems you're running into. IMO, you don't require a load balancer, as it becomes a single point of failure, and you need to ensure it has higher uptime and redundancies than the services behind it (which is not provided by plain install of Confluent Platform)
– cricket_007
Jan 5 at 19:22