Sending message from Celery task to ChannelsSending messages to django-channels via celeryBest practice for Websocket authentification - Django ChannelsDjango Channels send message from python websocket-client moduleHow should I configure Redis for Django Channels 2.0?How do I send channels 2.x group message from django-celery 3 task?Django Channels 2 stops sending message after a while but keeps receiving from client sideSending data to django channels groups via django viewssending message to client using Django Channels from Celery tasks.pydjango channels with redis: messages created before websocket connection established are lost

How did Lefschetz do mathematics without hands?

Company threatening to call my current job after I declined their offer

What do you call a notepad used to keep a record?

If two black hole event horizons overlap (touch) can they ever separate again?

How can a valley surrounded by mountains be fertile and rainy?

Bin Packing with Relational Penalization

Put my student loan in parents’ second mortgage - help?

How receiver knows the exact frequency in the channel to "listen to"?

Who voices the character "Finger" in The Fifth Element?

How is this practical and very old scene shot?

Does a Hand Crossbow with the Repeating Shot Infusion still require a Free Hand to use?

Can a stressful Wish's Strength reduction be cured early by a Greater Restoration spell?

How can I deal with extreme temperatures in a hotel room?

Just graduated with a master’s degree, but I internalised nothing

Journal standards vs. personal standards

I just started should I accept a farewell lunch for a coworker I don't know?

Can a nowhere continuous function have a connected graph?

My colleague is constantly blaming me for his errors

How to describe POV characters?

The warming up game

Most elegant way to write a one-shot 'if'

Most important new papers in computational complexity

Is it okay to submit a paper from a master's thesis without informing the advisor?

Using “ser” without "un/una"?



Sending message from Celery task to Channels


Sending messages to django-channels via celeryBest practice for Websocket authentification - Django ChannelsDjango Channels send message from python websocket-client moduleHow should I configure Redis for Django Channels 2.0?How do I send channels 2.x group message from django-celery 3 task?Django Channels 2 stops sending message after a while but keeps receiving from client sideSending data to django channels groups via django viewssending message to client using Django Channels from Celery tasks.pydjango channels with redis: messages created before websocket connection established are lost






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








1















Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1



I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.



I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.



tasks.py



def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)


consumers.py



class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'

# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()

async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)

# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass

# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)


settings.py



CHANNEL_LAYERS = 
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,



Since there is no exception/error, I am completely at a loss as to which part of this process is failing.



  1. Celery triggers the task? Yes

  2. The task runs and sends a message to the channel layer? Yes

  3. The consumer receives the message from the group and executes the send()? Yes

  4. Client receives the a websocket message? NO

Is this a problem between Channels and Redis? Is it a problem between Channels and the client?










share|improve this question
























  • Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message

    – Ken4scholars
    Mar 25 at 14:50











  • Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong

    – Ken4scholars
    Mar 25 at 15:00











  • Please show your complete consumers.py or the complete Consumer class. And have you added channel to group in your ws_connect? Were you able to send and receive messages if not via tasks? What is self.group_name? Is your channel added to this group?

    – spiritsree
    Mar 25 at 16:47











  • @spiritsree I have updated this with the full consumers.py file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.

    – Charles Koch
    Mar 25 at 17:17











  • It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? eventlet or gevent or prefork?

    – spiritsree
    Mar 25 at 20:43

















1















Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1



I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.



I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.



tasks.py



def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)


consumers.py



class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'

# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()

async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)

# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass

# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)


settings.py



CHANNEL_LAYERS = 
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,



Since there is no exception/error, I am completely at a loss as to which part of this process is failing.



  1. Celery triggers the task? Yes

  2. The task runs and sends a message to the channel layer? Yes

  3. The consumer receives the message from the group and executes the send()? Yes

  4. Client receives the a websocket message? NO

Is this a problem between Channels and Redis? Is it a problem between Channels and the client?










share|improve this question
























  • Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message

    – Ken4scholars
    Mar 25 at 14:50











  • Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong

    – Ken4scholars
    Mar 25 at 15:00











  • Please show your complete consumers.py or the complete Consumer class. And have you added channel to group in your ws_connect? Were you able to send and receive messages if not via tasks? What is self.group_name? Is your channel added to this group?

    – spiritsree
    Mar 25 at 16:47











  • @spiritsree I have updated this with the full consumers.py file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.

    – Charles Koch
    Mar 25 at 17:17











  • It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? eventlet or gevent or prefork?

    – spiritsree
    Mar 25 at 20:43













1












1








1


2






Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1



I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.



I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.



tasks.py



def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)


consumers.py



class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'

# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()

async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)

# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass

# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)


settings.py



CHANNEL_LAYERS = 
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,



Since there is no exception/error, I am completely at a loss as to which part of this process is failing.



  1. Celery triggers the task? Yes

  2. The task runs and sends a message to the channel layer? Yes

  3. The consumer receives the message from the group and executes the send()? Yes

  4. Client receives the a websocket message? NO

Is this a problem between Channels and Redis? Is it a problem between Channels and the client?










share|improve this question
















Django 2.1.1,
Django Channels 2.1.3,
Celery 4.2.1



I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.



I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.



tasks.py



def import_job(self):
# (do task calculations, store in data dict)
message = 'type': 'send_my_data',
'data': json.dumps(thecalcs)
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)('core-data', message)


consumers.py



class AsyncDataConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.channel_group_name = 'core-data'

# Join the group
await self.channel_layer.group_add(
self.channel_group_name,
self.channel_name
)
await self.accept()

async def disconnect(self, close_code):
# Leave the group
await self.channel_layer.group_discard(
self.channel_group_name,
self.channel_name
)

# Receive message from WebSocket
async def receive(self, text_data=None, bytes_data=None):
pass

# Receive message from the group
async def send_my_data(self, event):
text = event['data']
# Send message to WebSocket
await self.send(text_data=text)


settings.py



CHANNEL_LAYERS = 
'default':
'BACKEND': 'channels_redis.core.RedisChannelLayer',
'CONFIG':
"hosts": [('127.0.0.1', 6379)],
,
,



Since there is no exception/error, I am completely at a loss as to which part of this process is failing.



  1. Celery triggers the task? Yes

  2. The task runs and sends a message to the channel layer? Yes

  3. The consumer receives the message from the group and executes the send()? Yes

  4. Client receives the a websocket message? NO

Is this a problem between Channels and Redis? Is it a problem between Channels and the client?







django celery django-channels celerybeat






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 25 at 17:20







Charles Koch

















asked Mar 25 at 14:09









Charles KochCharles Koch

265 bronze badges




265 bronze badges












  • Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message

    – Ken4scholars
    Mar 25 at 14:50











  • Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong

    – Ken4scholars
    Mar 25 at 15:00











  • Please show your complete consumers.py or the complete Consumer class. And have you added channel to group in your ws_connect? Were you able to send and receive messages if not via tasks? What is self.group_name? Is your channel added to this group?

    – spiritsree
    Mar 25 at 16:47











  • @spiritsree I have updated this with the full consumers.py file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.

    – Charles Koch
    Mar 25 at 17:17











  • It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? eventlet or gevent or prefork?

    – spiritsree
    Mar 25 at 20:43

















  • Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message

    – Ken4scholars
    Mar 25 at 14:50











  • Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong

    – Ken4scholars
    Mar 25 at 15:00











  • Please show your complete consumers.py or the complete Consumer class. And have you added channel to group in your ws_connect? Were you able to send and receive messages if not via tasks? What is self.group_name? Is your channel added to this group?

    – spiritsree
    Mar 25 at 16:47











  • @spiritsree I have updated this with the full consumers.py file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.

    – Charles Koch
    Mar 25 at 17:17











  • It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? eventlet or gevent or prefork?

    – spiritsree
    Mar 25 at 20:43
















Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message

– Ken4scholars
Mar 25 at 14:50





Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message

– Ken4scholars
Mar 25 at 14:50













Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong

– Ken4scholars
Mar 25 at 15:00





Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong

– Ken4scholars
Mar 25 at 15:00













Please show your complete consumers.py or the complete Consumer class. And have you added channel to group in your ws_connect? Were you able to send and receive messages if not via tasks? What is self.group_name? Is your channel added to this group?

– spiritsree
Mar 25 at 16:47





Please show your complete consumers.py or the complete Consumer class. And have you added channel to group in your ws_connect? Were you able to send and receive messages if not via tasks? What is self.group_name? Is your channel added to this group?

– spiritsree
Mar 25 at 16:47













@spiritsree I have updated this with the full consumers.py file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.

– Charles Koch
Mar 25 at 17:17





@spiritsree I have updated this with the full consumers.py file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem.

– Charles Koch
Mar 25 at 17:17













It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? eventlet or gevent or prefork?

– spiritsree
Mar 25 at 20:43





It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? eventlet or gevent or prefork?

– spiritsree
Mar 25 at 20:43












1 Answer
1






active

oldest

votes


















2














It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.






share|improve this answer























  • Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

    – Marco Polo
    May 20 at 12:05










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55339738%2fsending-message-from-celery-task-to-channels%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









2














It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.






share|improve this answer























  • Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

    – Marco Polo
    May 20 at 12:05















2














It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.






share|improve this answer























  • Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

    – Marco Polo
    May 20 at 12:05













2












2








2







It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.






share|improve this answer













It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.







share|improve this answer












share|improve this answer



share|improve this answer










answered Mar 26 at 14:49









Charles KochCharles Koch

265 bronze badges




265 bronze badges












  • Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

    – Marco Polo
    May 20 at 12:05

















  • Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

    – Marco Polo
    May 20 at 12:05
















Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

– Marco Polo
May 20 at 12:05





Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco

– Marco Polo
May 20 at 12:05








Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.







Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.



















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55339738%2fsending-message-from-celery-task-to-channels%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript