Mixing
Getting the results from a docker swarm redis service with 3 nodes
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
This is a conceptual question. Lets say that i have a Python application which consists of three services: the actual Python application, celery worked process and redis .
I am planning to use redis as the results backend as well. Now when i deploy my application to a 3 node Swarm cluster it would ideally have all the 3 nodes running all of the my 3 services.
Now lets say i have the below code for a celery task: tasks.py
import time
from celery import Celery
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
@app.task(name='tasks.add')
def add(x, y):
total = x + y
print('{} + {} = {}'.format(x, y, total))
time.sleep(10)
return total
And the below code for getting the result from my backend redis store: invoker.py
import time
from tasks import add
from celery.result import AsyncResult
result = add.delay(1, 2)
while True:
_result2 = AsyncResult(result.task_id)
status = _result2.status
print(status)
if 'SUCCESS' in status:
print('result after 5 sec wait {_result2}'.format(_result2=_result2.get()))
break
time.sleep(5)
So i have two questions related to swarm mode of running this :
In the tasks.py file in the line :
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
the "localhost" string should be changed to whichever service name i am giving to the redis service in my swarm cluster?
- If the worker starts with my task and then in between that time the redis node which has invoked the task fails then how does swarm allow me to get the result from the backend since the original redis node which started the task is no longer there ?
Many many thanks for any answers and apologies if this is a bit vague since i am trying to understand how everything fits together in production swarm mode.
python redis celery docker-swarm
asked Jan 3 at 8:51
user1867151user1867151
93011023
add a comment |
This is a conceptual question. Lets say that i have a Python application which consists of three services: the actual Python application, celery worked process and redis .
I am planning to use redis as the results backend as well. Now when i deploy my application to a 3 node Swarm cluster it would ideally have all the 3 nodes running all of the my 3 services.
Now lets say i have the below code for a celery task: tasks.py
import time
from celery import Celery
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
@app.task(name='tasks.add')
def add(x, y):
total = x + y
print('{} + {} = {}'.format(x, y, total))
time.sleep(10)
return total
And the below code for getting the result from my backend redis store: invoker.py
import time
from tasks import add
from celery.result import AsyncResult
result = add.delay(1, 2)
while True:
_result2 = AsyncResult(result.task_id)
status = _result2.status
print(status)
if 'SUCCESS' in status:
print('result after 5 sec wait {_result2}'.format(_result2=_result2.get()))
break
time.sleep(5)
So i have two questions related to swarm mode of running this :
In the tasks.py file in the line :
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
the "localhost" string should be changed to whichever service name i am giving to the redis service in my swarm cluster?
- If the worker starts with my task and then in between that time the redis node which has invoked the task fails then how does swarm allow me to get the result from the backend since the original redis node which started the task is no longer there ?
Many many thanks for any answers and apologies if this is a bit vague since i am trying to understand how everything fits together in production swarm mode.
python redis celery docker-swarm
asked Jan 3 at 8:51
user1867151user1867151
93011023
add a comment |
This is a conceptual question. Lets say that i have a Python application which consists of three services: the actual Python application, celery worked process and redis .
I am planning to use redis as the results backend as well. Now when i deploy my application to a 3 node Swarm cluster it would ideally have all the 3 nodes running all of the my 3 services.
Now lets say i have the below code for a celery task: tasks.py
import time
from celery import Celery
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
@app.task(name='tasks.add')
def add(x, y):
total = x + y
print('{} + {} = {}'.format(x, y, total))
time.sleep(10)
return total
And the below code for getting the result from my backend redis store: invoker.py
import time
from tasks import add
from celery.result import AsyncResult
result = add.delay(1, 2)
while True:
_result2 = AsyncResult(result.task_id)
status = _result2.status
print(status)
if 'SUCCESS' in status:
print('result after 5 sec wait {_result2}'.format(_result2=_result2.get()))
break
time.sleep(5)
So i have two questions related to swarm mode of running this :
In the tasks.py file in the line :
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
the "localhost" string should be changed to whichever service name i am giving to the redis service in my swarm cluster?
- If the worker starts with my task and then in between that time the redis node which has invoked the task fails then how does swarm allow me to get the result from the backend since the original redis node which started the task is no longer there ?
Many many thanks for any answers and apologies if this is a bit vague since i am trying to understand how everything fits together in production swarm mode.
python redis celery docker-swarm
asked Jan 3 at 8:51
user1867151user1867151
93011023
This is a conceptual question. Lets say that i have a Python application which consists of three services: the actual Python application, celery worked process and redis .
I am planning to use redis as the results backend as well. Now when i deploy my application to a 3 node Swarm cluster it would ideally have all the 3 nodes running all of the my 3 services.
Now lets say i have the below code for a celery task: tasks.py
import time
from celery import Celery
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
@app.task(name='tasks.add')
def add(x, y):
total = x + y
print('{} + {} = {}'.format(x, y, total))
time.sleep(10)
return total
And the below code for getting the result from my backend redis store: invoker.py
import time
from tasks import add
from celery.result import AsyncResult
result = add.delay(1, 2)
while True:
_result2 = AsyncResult(result.task_id)
status = _result2.status
print(status)
if 'SUCCESS' in status:
print('result after 5 sec wait {_result2}'.format(_result2=_result2.get()))
break
time.sleep(5)
So i have two questions related to swarm mode of running this :
In the tasks.py file in the line :
app = Celery('tasks', backend='redis://localhost:6379/0', broker='redis://localhost:6379/0')
the "localhost" string should be changed to whichever service name i am giving to the redis service in my swarm cluster?
- If the worker starts with my task and then in between that time the redis node which has invoked the task fails then how does swarm allow me to get the result from the backend since the original redis node which started the task is no longer there ?
Many many thanks for any answers and apologies if this is a bit vague since i am trying to understand how everything fits together in production swarm mode.
python redis celery docker-swarm
python redis celery docker-swarm
asked Jan 3 at 8:51
user1867151user1867151
93011023
asked Jan 3 at 8:51
user1867151user1867151
93011023
asked Jan 3 at 8:51
user1867151user1867151
93011023
asked Jan 3 at 8:51
user1867151user1867151
93011023
asked Jan 3 at 8:51
user1867151user1867151
93011023
93011023
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54018972%2fgetting-the-results-from-a-docker-swarm-redis-service-with-3-nodes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54018972%2fgetting-the-results-from-a-docker-swarm-redis-service-with-3-nodes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
