Sending message from Celery task to ChannelsSending messages to django-channels via celeryBest practice for Websocket authentification - Django ChannelsDjango Channels send message from python websocket-client moduleHow should I configure Redis for Django Channels 2.0?How do I send channels 2.x group message from django-celery 3 task?Django Channels 2 stops sending message after a while but keeps receiving from client sideSending data to django channels groups via django viewssending message to client using Django Channels from Celery tasks.pydjango channels with redis: messages created before websocket connection established are lost
How did Lefschetz do mathematics without hands?
Company threatening to call my current job after I declined their offer
What do you call a notepad used to keep a record?
If two black hole event horizons overlap (touch) can they ever separate again?
How can a valley surrounded by mountains be fertile and rainy?
Bin Packing with Relational Penalization
Put my student loan in parents’ second mortgage - help?
How receiver knows the exact frequency in the channel to "listen to"?
Who voices the character "Finger" in The Fifth Element?
How is this practical and very old scene shot?
Does a Hand Crossbow with the Repeating Shot Infusion still require a Free Hand to use?
Can a stressful Wish's Strength reduction be cured early by a Greater Restoration spell?
How can I deal with extreme temperatures in a hotel room?
Just graduated with a master’s degree, but I internalised nothing
Journal standards vs. personal standards
I just started should I accept a farewell lunch for a coworker I don't know?
Can a nowhere continuous function have a connected graph?
My colleague is constantly blaming me for his errors
How to describe POV characters?
The warming up game
Most elegant way to write a one-shot 'if'
Most important new papers in computational complexity
Is it okay to submit a paper from a master's thesis without informing the advisor?
Using “ser” without "un/una"?
Sending message from Celery task to Channels
Sending messages to django-channels via celeryBest practice for Websocket authentification - Django ChannelsDjango Channels send message from python websocket-client moduleHow should I configure Redis for Django Channels 2.0?How do I send channels 2.x group message from django-celery 3 task?Django Channels 2 stops sending message after a while but keeps receiving from client sideSending data to django channels groups via django viewssending message to client using Django Channels from Celery tasks.pydjango channels with redis: messages created before websocket connection established are lost
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1
I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.
I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.
tasks.py
def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)
consumers.py
class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'
# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()
async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)
# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass
# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)
settings.py
CHANNEL_LAYERS =
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,
Since there is no exception/error, I am completely at a loss as to which part of this process is failing.
- Celery triggers the task? Yes
- The task runs and sends a message to the channel layer? Yes
- The consumer receives the message from the group and executes the
send()
? Yes - Client receives the a websocket message? NO
Is this a problem between Channels and Redis? Is it a problem between Channels and the client?
django celery django-channels celerybeat
|
show 1 more comment
Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1
I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.
I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.
tasks.py
def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)
consumers.py
class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'
# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()
async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)
# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass
# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)
settings.py
CHANNEL_LAYERS =
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,
Since there is no exception/error, I am completely at a loss as to which part of this process is failing.
- Celery triggers the task? Yes
- The task runs and sends a message to the channel layer? Yes
- The consumer receives the message from the group and executes the
send()
? Yes - Client receives the a websocket message? NO
Is this a problem between Channels and Redis? Is it a problem between Channels and the client?
django celery django-channels celerybeat
Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message
– Ken4scholars
Mar 25 at 14:50
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong
– Ken4scholars
Mar 25 at 15:00
Please show your completeconsumers.py
or the complete Consumer class. And have you added channel to group in yourws_connect
? Were you able to send and receive messages if not via tasks? What isself.group_name
? Is your channel added to this group?
– spiritsree
Mar 25 at 16:47
@spiritsree I have updated this with the fullconsumers.py
file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.
– Charles Koch
Mar 25 at 17:17
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency?eventlet
orgevent
orprefork
?
– spiritsree
Mar 25 at 20:43
|
show 1 more comment
Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1
I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.
I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.
tasks.py
def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)
consumers.py
class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'
# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()
async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)
# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass
# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)
settings.py
CHANNEL_LAYERS =
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,
Since there is no exception/error, I am completely at a loss as to which part of this process is failing.
- Celery triggers the task? Yes
- The task runs and sends a message to the channel layer? Yes
- The consumer receives the message from the group and executes the
send()
? Yes - Client receives the a websocket message? NO
Is this a problem between Channels and Redis? Is it a problem between Channels and the client?
django celery django-channels celerybeat
Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1
I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.
I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.
tasks.py
def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)
consumers.py
class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'
# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()
async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)
# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass
# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)
settings.py
CHANNEL_LAYERS =
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,
Since there is no exception/error, I am completely at a loss as to which part of this process is failing.
- Celery triggers the task? Yes
- The task runs and sends a message to the channel layer? Yes
- The consumer receives the message from the group and executes the
send()
? Yes - Client receives the a websocket message? NO
Is this a problem between Channels and Redis? Is it a problem between Channels and the client?
django celery django-channels celerybeat
django celery django-channels celerybeat
edited Mar 25 at 17:20
Charles Koch
asked Mar 25 at 14:09
Charles KochCharles Koch
265 bronze badges
265 bronze badges
Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message
– Ken4scholars
Mar 25 at 14:50
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong
– Ken4scholars
Mar 25 at 15:00
Please show your completeconsumers.py
or the complete Consumer class. And have you added channel to group in yourws_connect
? Were you able to send and receive messages if not via tasks? What isself.group_name
? Is your channel added to this group?
– spiritsree
Mar 25 at 16:47
@spiritsree I have updated this with the fullconsumers.py
file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.
– Charles Koch
Mar 25 at 17:17
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency?eventlet
orgevent
orprefork
?
– spiritsree
Mar 25 at 20:43
|
show 1 more comment
Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message
– Ken4scholars
Mar 25 at 14:50
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong
– Ken4scholars
Mar 25 at 15:00
Please show your completeconsumers.py
or the complete Consumer class. And have you added channel to group in yourws_connect
? Were you able to send and receive messages if not via tasks? What isself.group_name
? Is your channel added to this group?
– spiritsree
Mar 25 at 16:47
@spiritsree I have updated this with the fullconsumers.py
file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.
– Charles Koch
Mar 25 at 17:17
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency?eventlet
orgevent
orprefork
?
– spiritsree
Mar 25 at 20:43
Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message
– Ken4scholars
Mar 25 at 14:50
Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message
– Ken4scholars
Mar 25 at 14:50
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong
– Ken4scholars
Mar 25 at 15:00
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong
– Ken4scholars
Mar 25 at 15:00
Please show your complete
consumers.py
or the complete Consumer class. And have you added channel to group in your ws_connect
? Were you able to send and receive messages if not via tasks? What is self.group_name
? Is your channel added to this group?– spiritsree
Mar 25 at 16:47
Please show your complete
consumers.py
or the complete Consumer class. And have you added channel to group in your ws_connect
? Were you able to send and receive messages if not via tasks? What is self.group_name
? Is your channel added to this group?– spiritsree
Mar 25 at 16:47
@spiritsree I have updated this with the full
consumers.py
file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.– Charles Koch
Mar 25 at 17:17
@spiritsree I have updated this with the full
consumers.py
file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.– Charles Koch
Mar 25 at 17:17
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency?
eventlet
or gevent
or prefork
?– spiritsree
Mar 25 at 20:43
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency?
eventlet
or gevent
or prefork
?– spiritsree
Mar 25 at 20:43
|
show 1 more comment
1 Answer
1
active
oldest
votes
It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55339738%2fsending-message-from-celery-task-to-channels%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
add a comment |
It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
add a comment |
It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.
It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.
answered Mar 26 at 14:49
Charles KochCharles Koch
265 bronze badges
265 bronze badges
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
add a comment |
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco
– Marco Polo
May 20 at 12:05
add a comment |
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55339738%2fsending-message-from-celery-task-to-channels%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message
– Ken4scholars
Mar 25 at 14:50
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong
– Ken4scholars
Mar 25 at 15:00
Please show your complete
consumers.py
or the complete Consumer class. And have you added channel to group in yourws_connect
? Were you able to send and receive messages if not via tasks? What isself.group_name
? Is your channel added to this group?– spiritsree
Mar 25 at 16:47
@spiritsree I have updated this with the full
consumers.py
file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.– Charles Koch
Mar 25 at 17:17
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency?
eventlet
orgevent
orprefork
?– spiritsree
Mar 25 at 20:43