Lambda function does not retry the sqs message processing in case the Java code throws a runtimeExceptionRead SQS queue from AWS LambdaSQS dead letter queue not triggered on AWS Lambda invocation errorsAWS Lambda Java Function is successful but times outHow to discard records?AWS SQS doesn't reliably trigger LambdaHow to monitor AWS Lambda events through Cloudwatch eventsMessages disappearing from SQS queue to LambdaAWS Lambda (Java impl) not showing error message in the logAWS Lambda function does not scale out enough to process SQS messagesDoes AWS Lambda duplicate every message?
Is the order of words purely based on convention?
Difference between types of yeast
Is it acceptable to say that a reviewer's concern is not going to be addressed because then the paper would be too long?
What happens to a net with the Returning Weapon artificer infusion after it hits?
Difference between "rip up" and "rip down"
Why does C++ have 'Undefined Behaviour' and other languages like C# or Java don't?
Hangman Game (YAHG)
What did Jesse Pinkman mix into Walt's coffee?
How can this Stack Exchange site have an animated favicon?
Need Improvement on Script Which Continuosly Tests Website
How do you program Babbage's Difference Engine?
How to justify getting additional team member when the current team is doing well?
Why does my browser attempt to download pages from http://clhs.lisp.se instead of viewing them normally?
Could Apollo astronauts see city lights from the moon?
How to deal with a Homophobic PC
Does the Horizon Walker ranger's Planar Warrior feature bypass resistance to non-magical attacks?
Excel Solver linear programming - Is it possible to use average of values as a constraint without #DIV/0! errors or sacrificing linearity?
I am 15 years old and do not go to a Yeshiva but would like to learn Talmud. A few rabbis near me said they could teach me. How should I start
We are on WHV, my boyfriend was in a small collision, we are leaving in 2 weeks what happens if we don’t pay the damages?
Why is 6. Nge2 better, and 7. d5 a necessary push in this game?
May I know how to stop these death waves?
Can my former employer sue me if I don't give them the photos I took (taking pictures was not part of my job description)?
Does wetting a beer glass change the foam characteristics?
Why weren't the Death Star plans transmitted electronically?
Lambda function does not retry the sqs message processing in case the Java code throws a runtimeException
Read SQS queue from AWS LambdaSQS dead letter queue not triggered on AWS Lambda invocation errorsAWS Lambda Java Function is successful but times outHow to discard records?AWS SQS doesn't reliably trigger LambdaHow to monitor AWS Lambda events through Cloudwatch eventsMessages disappearing from SQS queue to LambdaAWS Lambda (Java impl) not showing error message in the logAWS Lambda function does not scale out enough to process SQS messagesDoes AWS Lambda duplicate every message?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have a lambda function written in java that listens to sqs events and tries to do some kind of processing on those sqs messages.
As per lambda documentation, if lambda code throws a runtimeException then, lambda would retry the same message twice before it sends it back to the queue. However, I don't see that behavior.
I only see it processing the message just once.
Here is a snippet from the relevant lambda code in this case.
@Override
public Boolean handleRequest(SQSEvent sqsEvent, Context context)
try
........some queue message processing.....
catch(Exception ex)
throw new RuntimeException("exception occurred");
Is this not good enough for lambda to retry the message 2 more times? I did check the cloudwatch to see what lambda logs and it just has logs from the very first processing only and not the retries.
Can someone tell me what did I miss here because of which it does not work as expected.
aws-lambda amazon-sqs aws-java-sdk
add a comment
|
I have a lambda function written in java that listens to sqs events and tries to do some kind of processing on those sqs messages.
As per lambda documentation, if lambda code throws a runtimeException then, lambda would retry the same message twice before it sends it back to the queue. However, I don't see that behavior.
I only see it processing the message just once.
Here is a snippet from the relevant lambda code in this case.
@Override
public Boolean handleRequest(SQSEvent sqsEvent, Context context)
try
........some queue message processing.....
catch(Exception ex)
throw new RuntimeException("exception occurred");
Is this not good enough for lambda to retry the message 2 more times? I did check the cloudwatch to see what lambda logs and it just has logs from the very first processing only and not the retries.
Can someone tell me what did I miss here because of which it does not work as expected.
aws-lambda amazon-sqs aws-java-sdk
add a comment
|
I have a lambda function written in java that listens to sqs events and tries to do some kind of processing on those sqs messages.
As per lambda documentation, if lambda code throws a runtimeException then, lambda would retry the same message twice before it sends it back to the queue. However, I don't see that behavior.
I only see it processing the message just once.
Here is a snippet from the relevant lambda code in this case.
@Override
public Boolean handleRequest(SQSEvent sqsEvent, Context context)
try
........some queue message processing.....
catch(Exception ex)
throw new RuntimeException("exception occurred");
Is this not good enough for lambda to retry the message 2 more times? I did check the cloudwatch to see what lambda logs and it just has logs from the very first processing only and not the retries.
Can someone tell me what did I miss here because of which it does not work as expected.
aws-lambda amazon-sqs aws-java-sdk
I have a lambda function written in java that listens to sqs events and tries to do some kind of processing on those sqs messages.
As per lambda documentation, if lambda code throws a runtimeException then, lambda would retry the same message twice before it sends it back to the queue. However, I don't see that behavior.
I only see it processing the message just once.
Here is a snippet from the relevant lambda code in this case.
@Override
public Boolean handleRequest(SQSEvent sqsEvent, Context context)
try
........some queue message processing.....
catch(Exception ex)
throw new RuntimeException("exception occurred");
Is this not good enough for lambda to retry the message 2 more times? I did check the cloudwatch to see what lambda logs and it just has logs from the very first processing only and not the retries.
Can someone tell me what did I miss here because of which it does not work as expected.
aws-lambda amazon-sqs aws-java-sdk
aws-lambda amazon-sqs aws-java-sdk
asked Mar 28 at 18:11
HaryHary
3015 silver badges24 bronze badges
3015 silver badges24 bronze badges
add a comment
|
add a comment
|
2 Answers
2
active
oldest
votes
You are missing the throws clause in handleRequest. If you don't have that then, lambda would just swallow the exception
public Boolean handleRequest(SQSEvent sqsEvent, Context context) throws RuntimeException
Other than that, what Thales Munussi has told you about synchronous polling is absolutely right. When you hook sqs with lambda, lambda polls sqs which keeps an open connection between the two hence making it a synchronous connection
As per aws documentation, lambda doesn’t retry in such synchronous cases. Setting up a dlq and retires in sqs itself is your best recourse
Keep in mind that lambda would send the message back to the queue after a runtime exception is thrown in your java code
Based on the redrive setting in your sqs, the sqs will generate the same event based on redrive number.
Once lambda fails to process successfully for redrive number of times, message is sent to the DLQ from the main queue
add a comment
|
The documentation says it retries two more times if the invocation is asynchronous. SQS is a poll-based system. Lambda will poll the Queue and all of its invocations will be synchronous.
For poll-based AWS services (Amazon Kinesis, Amazon DynamoDB, Amazon
Simple Queue Service), AWS Lambda polls the stream or message queue
and invokes your Lambda function synchronously.
What you can do is configure a DLQ on your source SQS queue in case your message fails so you can either analyse it further or process the message again based on the logic you have configured.
EDIT
The OP is not able to see the messages in the DLQ for somehow. I have attached images to show it works.
Lambda sqs-test is triggered by a new message in SQS queue sqs-test
These are the Queues (sqs-test-dlq is configured as a DLQ for sqs-test).

This is the code for the Lambda function:

This is the configuration for sqs-test

And this is the redrive policy

After the messages failed in the Lambda function, they were successfully sent to the configured DLQ:

You must be missing some basic configuration because it works seamlessly as the images above show.
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
|
show 7 more comments
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55404296%2flambda-function-does-not-retry-the-sqs-message-processing-in-case-the-java-code%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You are missing the throws clause in handleRequest. If you don't have that then, lambda would just swallow the exception
public Boolean handleRequest(SQSEvent sqsEvent, Context context) throws RuntimeException
Other than that, what Thales Munussi has told you about synchronous polling is absolutely right. When you hook sqs with lambda, lambda polls sqs which keeps an open connection between the two hence making it a synchronous connection
As per aws documentation, lambda doesn’t retry in such synchronous cases. Setting up a dlq and retires in sqs itself is your best recourse
Keep in mind that lambda would send the message back to the queue after a runtime exception is thrown in your java code
Based on the redrive setting in your sqs, the sqs will generate the same event based on redrive number.
Once lambda fails to process successfully for redrive number of times, message is sent to the DLQ from the main queue
add a comment
|
You are missing the throws clause in handleRequest. If you don't have that then, lambda would just swallow the exception
public Boolean handleRequest(SQSEvent sqsEvent, Context context) throws RuntimeException
Other than that, what Thales Munussi has told you about synchronous polling is absolutely right. When you hook sqs with lambda, lambda polls sqs which keeps an open connection between the two hence making it a synchronous connection
As per aws documentation, lambda doesn’t retry in such synchronous cases. Setting up a dlq and retires in sqs itself is your best recourse
Keep in mind that lambda would send the message back to the queue after a runtime exception is thrown in your java code
Based on the redrive setting in your sqs, the sqs will generate the same event based on redrive number.
Once lambda fails to process successfully for redrive number of times, message is sent to the DLQ from the main queue
add a comment
|
You are missing the throws clause in handleRequest. If you don't have that then, lambda would just swallow the exception
public Boolean handleRequest(SQSEvent sqsEvent, Context context) throws RuntimeException
Other than that, what Thales Munussi has told you about synchronous polling is absolutely right. When you hook sqs with lambda, lambda polls sqs which keeps an open connection between the two hence making it a synchronous connection
As per aws documentation, lambda doesn’t retry in such synchronous cases. Setting up a dlq and retires in sqs itself is your best recourse
Keep in mind that lambda would send the message back to the queue after a runtime exception is thrown in your java code
Based on the redrive setting in your sqs, the sqs will generate the same event based on redrive number.
Once lambda fails to process successfully for redrive number of times, message is sent to the DLQ from the main queue
You are missing the throws clause in handleRequest. If you don't have that then, lambda would just swallow the exception
public Boolean handleRequest(SQSEvent sqsEvent, Context context) throws RuntimeException
Other than that, what Thales Munussi has told you about synchronous polling is absolutely right. When you hook sqs with lambda, lambda polls sqs which keeps an open connection between the two hence making it a synchronous connection
As per aws documentation, lambda doesn’t retry in such synchronous cases. Setting up a dlq and retires in sqs itself is your best recourse
Keep in mind that lambda would send the message back to the queue after a runtime exception is thrown in your java code
Based on the redrive setting in your sqs, the sqs will generate the same event based on redrive number.
Once lambda fails to process successfully for redrive number of times, message is sent to the DLQ from the main queue
edited Mar 29 at 3:41
answered Mar 28 at 21:34
Ray SRay S
669 bronze badges
669 bronze badges
add a comment
|
add a comment
|
The documentation says it retries two more times if the invocation is asynchronous. SQS is a poll-based system. Lambda will poll the Queue and all of its invocations will be synchronous.
For poll-based AWS services (Amazon Kinesis, Amazon DynamoDB, Amazon
Simple Queue Service), AWS Lambda polls the stream or message queue
and invokes your Lambda function synchronously.
What you can do is configure a DLQ on your source SQS queue in case your message fails so you can either analyse it further or process the message again based on the logic you have configured.
EDIT
The OP is not able to see the messages in the DLQ for somehow. I have attached images to show it works.
Lambda sqs-test is triggered by a new message in SQS queue sqs-test
These are the Queues (sqs-test-dlq is configured as a DLQ for sqs-test).

This is the code for the Lambda function:

This is the configuration for sqs-test

And this is the redrive policy

After the messages failed in the Lambda function, they were successfully sent to the configured DLQ:

You must be missing some basic configuration because it works seamlessly as the images above show.
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
|
show 7 more comments
The documentation says it retries two more times if the invocation is asynchronous. SQS is a poll-based system. Lambda will poll the Queue and all of its invocations will be synchronous.
For poll-based AWS services (Amazon Kinesis, Amazon DynamoDB, Amazon
Simple Queue Service), AWS Lambda polls the stream or message queue
and invokes your Lambda function synchronously.
What you can do is configure a DLQ on your source SQS queue in case your message fails so you can either analyse it further or process the message again based on the logic you have configured.
EDIT
The OP is not able to see the messages in the DLQ for somehow. I have attached images to show it works.
Lambda sqs-test is triggered by a new message in SQS queue sqs-test
These are the Queues (sqs-test-dlq is configured as a DLQ for sqs-test).

This is the code for the Lambda function:

This is the configuration for sqs-test

And this is the redrive policy

After the messages failed in the Lambda function, they were successfully sent to the configured DLQ:

You must be missing some basic configuration because it works seamlessly as the images above show.
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
|
show 7 more comments
The documentation says it retries two more times if the invocation is asynchronous. SQS is a poll-based system. Lambda will poll the Queue and all of its invocations will be synchronous.
For poll-based AWS services (Amazon Kinesis, Amazon DynamoDB, Amazon
Simple Queue Service), AWS Lambda polls the stream or message queue
and invokes your Lambda function synchronously.
What you can do is configure a DLQ on your source SQS queue in case your message fails so you can either analyse it further or process the message again based on the logic you have configured.
EDIT
The OP is not able to see the messages in the DLQ for somehow. I have attached images to show it works.
Lambda sqs-test is triggered by a new message in SQS queue sqs-test
These are the Queues (sqs-test-dlq is configured as a DLQ for sqs-test).

This is the code for the Lambda function:

This is the configuration for sqs-test

And this is the redrive policy

After the messages failed in the Lambda function, they were successfully sent to the configured DLQ:

You must be missing some basic configuration because it works seamlessly as the images above show.
The documentation says it retries two more times if the invocation is asynchronous. SQS is a poll-based system. Lambda will poll the Queue and all of its invocations will be synchronous.
For poll-based AWS services (Amazon Kinesis, Amazon DynamoDB, Amazon
Simple Queue Service), AWS Lambda polls the stream or message queue
and invokes your Lambda function synchronously.
What you can do is configure a DLQ on your source SQS queue in case your message fails so you can either analyse it further or process the message again based on the logic you have configured.
EDIT
The OP is not able to see the messages in the DLQ for somehow. I have attached images to show it works.
Lambda sqs-test is triggered by a new message in SQS queue sqs-test
These are the Queues (sqs-test-dlq is configured as a DLQ for sqs-test).

This is the code for the Lambda function:

This is the configuration for sqs-test

And this is the redrive policy

After the messages failed in the Lambda function, they were successfully sent to the configured DLQ:

You must be missing some basic configuration because it works seamlessly as the images above show.
edited Mar 28 at 19:08
answered Mar 28 at 18:17
Thales MinussiThales Minussi
2,9501 gold badge7 silver badges28 bronze badges
2,9501 gold badge7 silver badges28 bronze badges
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
|
show 7 more comments
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
the way it is configured, lambda has sqs configured and it listens to sqs events. In that case, I assume it is asynch invocation, no ?
– Hary
Mar 28 at 18:21
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
Nope, it’s synchronous. Behind the scenes it’s a polling mechanism, still
– Thales Minussi
Mar 28 at 18:25
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
“This consists of Amazon Simple Queue Service. If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function.”
– Thales Minussi
Mar 28 at 18:30
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
Polling mechanism i believe would just tell the lamda that a message is ready and lamda is still fetching the message asynchronously. Is that not correct ?
– Hary
Mar 28 at 18:32
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
In any case, when I throw runtimeException, it doesn't even go in the DLQ. I see that sqs just lost the message in case lambda encountered a runtime exception which never went in DLQ as well
– Hary
Mar 28 at 18:34
|
show 7 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55404296%2flambda-function-does-not-retry-the-sqs-message-processing-in-case-the-java-code%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown