Mixing
How to receive data from multithread in Java using Custom Event Listener?
I have a about 500 threads(assume that call as X for all these threads), and these threads receive data from different restful services continuously. Also, i have 20 threads(assume that call as Y for all these threads) for receive data returned from other threads(from X).
Should I use BlockingQueue or Custom Event Listener? Which one is better?
Also, there should be no data loss.
java multithreading thread-safety streaming
asked Jan 1 at 16:10
AdamR.AdamR.
14118
add a comment |
I have a about 500 threads(assume that call as X for all these threads), and these threads receive data from different restful services continuously. Also, i have 20 threads(assume that call as Y for all these threads) for receive data returned from other threads(from X).
Should I use BlockingQueue or Custom Event Listener? Which one is better?
Also, there should be no data loss.
java multithreading thread-safety streaming
asked Jan 1 at 16:10
AdamR.AdamR.
14118
add a comment |
I have a about 500 threads(assume that call as X for all these threads), and these threads receive data from different restful services continuously. Also, i have 20 threads(assume that call as Y for all these threads) for receive data returned from other threads(from X).
Should I use BlockingQueue or Custom Event Listener? Which one is better?
Also, there should be no data loss.
java multithreading thread-safety streaming
asked Jan 1 at 16:10
AdamR.AdamR.
14118
I have a about 500 threads(assume that call as X for all these threads), and these threads receive data from different restful services continuously. Also, i have 20 threads(assume that call as Y for all these threads) for receive data returned from other threads(from X).
Should I use BlockingQueue or Custom Event Listener? Which one is better?
Also, there should be no data loss.
java multithreading thread-safety streaming
java multithreading thread-safety streaming
asked Jan 1 at 16:10
AdamR.AdamR.
14118
asked Jan 1 at 16:10
AdamR.AdamR.
14118
asked Jan 1 at 16:10
AdamR.AdamR.
14118
asked Jan 1 at 16:10
AdamR.AdamR.
14118
asked Jan 1 at 16:10
AdamR.AdamR.
14118
14118
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
Limitation of using BlockingQueue, it has a queue with a threshold point. Post this point, no more threads can be processed. In a scenario, Lost Data couldn't be recovered when exceptions (failures) are thrown by bunch of threads (X or Y) (which couldn't be processed again).
However, Custom Event Listener is preferable approach where acknowledgment can be done for all thread, and in case of thread exception (failure), we can retrieve lost data by using some reprocess mechanism.
Can use ReactiveX(Observable) library for this problem.
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
add a comment |
Why don't you try a messaging queue( like Kafka, rabbitmq)
which work like a data holder between the threads x and Y.
You put data in queue from x's and fetch/process the data via Y's which is asynchronous.
You should opt for this as even in Event Listener if we consider number of x 500 and y 20. then even for each call of x's there be 20 call (assume).which can become a bottle neck on heavy load.
answered Jan 2 at 6:45
SachinSachin
1115
add a comment |
BlockingQueue is better (usually).
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53996948%2fhow-to-receive-data-from-multithread-in-java-using-custom-event-listener%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Limitation of using BlockingQueue, it has a queue with a threshold point. Post this point, no more threads can be processed. In a scenario, Lost Data couldn't be recovered when exceptions (failures) are thrown by bunch of threads (X or Y) (which couldn't be processed again).
However, Custom Event Listener is preferable approach where acknowledgment can be done for all thread, and in case of thread exception (failure), we can retrieve lost data by using some reprocess mechanism.
Can use ReactiveX(Observable) library for this problem.
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
add a comment |
Limitation of using BlockingQueue, it has a queue with a threshold point. Post this point, no more threads can be processed. In a scenario, Lost Data couldn't be recovered when exceptions (failures) are thrown by bunch of threads (X or Y) (which couldn't be processed again).
However, Custom Event Listener is preferable approach where acknowledgment can be done for all thread, and in case of thread exception (failure), we can retrieve lost data by using some reprocess mechanism.
Can use ReactiveX(Observable) library for this problem.
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
add a comment |
Limitation of using BlockingQueue, it has a queue with a threshold point. Post this point, no more threads can be processed. In a scenario, Lost Data couldn't be recovered when exceptions (failures) are thrown by bunch of threads (X or Y) (which couldn't be processed again).
However, Custom Event Listener is preferable approach where acknowledgment can be done for all thread, and in case of thread exception (failure), we can retrieve lost data by using some reprocess mechanism.
Can use ReactiveX(Observable) library for this problem.
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
Limitation of using BlockingQueue, it has a queue with a threshold point. Post this point, no more threads can be processed. In a scenario, Lost Data couldn't be recovered when exceptions (failures) are thrown by bunch of threads (X or Y) (which couldn't be processed again).
However, Custom Event Listener is preferable approach where acknowledgment can be done for all thread, and in case of thread exception (failure), we can retrieve lost data by using some reprocess mechanism.
Can use ReactiveX(Observable) library for this problem.
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
answered Jan 1 at 17:37
Prashant SinghPrashant Singh
496
496
add a comment |
add a comment |
Why don't you try a messaging queue( like Kafka, rabbitmq)
which work like a data holder between the threads x and Y.
You put data in queue from x's and fetch/process the data via Y's which is asynchronous.
You should opt for this as even in Event Listener if we consider number of x 500 and y 20. then even for each call of x's there be 20 call (assume).which can become a bottle neck on heavy load.
answered Jan 2 at 6:45
SachinSachin
1115
add a comment |
Why don't you try a messaging queue( like Kafka, rabbitmq)
which work like a data holder between the threads x and Y.
You put data in queue from x's and fetch/process the data via Y's which is asynchronous.
You should opt for this as even in Event Listener if we consider number of x 500 and y 20. then even for each call of x's there be 20 call (assume).which can become a bottle neck on heavy load.
answered Jan 2 at 6:45
SachinSachin
1115
add a comment |
Why don't you try a messaging queue( like Kafka, rabbitmq)
which work like a data holder between the threads x and Y.
You put data in queue from x's and fetch/process the data via Y's which is asynchronous.
You should opt for this as even in Event Listener if we consider number of x 500 and y 20. then even for each call of x's there be 20 call (assume).which can become a bottle neck on heavy load.
answered Jan 2 at 6:45
SachinSachin
1115
Why don't you try a messaging queue( like Kafka, rabbitmq)
which work like a data holder between the threads x and Y.
You put data in queue from x's and fetch/process the data via Y's which is asynchronous.
You should opt for this as even in Event Listener if we consider number of x 500 and y 20. then even for each call of x's there be 20 call (assume).which can become a bottle neck on heavy load.
answered Jan 2 at 6:45
SachinSachin
1115
answered Jan 2 at 6:45
SachinSachin
1115
answered Jan 2 at 6:45
SachinSachin
1115
answered Jan 2 at 6:45
SachinSachin
1115
1115
add a comment |
add a comment |
BlockingQueue is better (usually).
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
add a comment |
BlockingQueue is better (usually).
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
add a comment |
BlockingQueue is better (usually).
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
BlockingQueue is better (usually).
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
answered Jan 2 at 10:49
Alexei KaigorodovAlexei Kaigorodov
10.1k11229
10.1k11229
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53996948%2fhow-to-receive-data-from-multithread-in-java-using-custom-event-listener%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
