Why does Amazon SES accept my get requests, but deny my similarly structured post requests?HTTP test server accepting GET/POST requestsHow to generate Signature in AWS from JavaWS eb init missing equal-sign errorAmazon SES Emails in BulkAmazon SES 403 errorGetting “SignatureDoesNotMatch” error with delete Bucket Replication operation in Amazon s3Unexpected response status: 401 while accessing X-Amz-Target: Logs_20181106.GetLogEvents in AWSAWS Signature is IncorrectPut Object requests with Object Lock parameters require AWS Signature Version 4How to send gatling request with AWS Signature Version 4?
What is the difference between 1/3, 1/2, and full casters?
High income, sudden windfall
What is "I bet" in German?
What is AM-CM inequality?
Why/when is AC-DC-AC conversion superior to direct AC-Ac conversion?
Iterate over non-const variables in C++
Unethical behavior : should I report it?
Trying to build a function to compute divided difference for arbitrary list of points
What's the difference between 2a and 10a charging options?
What to do when you reach a conclusion and find out later on that someone else already did?
How can I receive packages while in France?
At what rate does the volume (velocity) of a note decay?
Is there anything wrong with Thrawn?
Send a single HTML email from Thunderbird, overriding the default "plain text" setting
Is a fighting a fallen friend with the help of a redeemed villain story too much for one book
What does "see" in "the Holy See" mean?
Why are all my history books dividing Chinese history after the Han dynasty?
Explain why watch 'jobs' does not work but watch 'ps' work?
kids pooling money for Lego League and taxes
Word for showing a small part of something briefly to hint to its existence or beauty without fully uncovering it
What does コテッと mean?
Why are so many countries still in the Commonwealth?
How can I stop myself from micromanaging other PCs' actions?
How do I run a game when my PCs have different approaches to combat?
Why does Amazon SES accept my get requests, but deny my similarly structured post requests?
HTTP test server accepting GET/POST requestsHow to generate Signature in AWS from JavaWS eb init missing equal-sign errorAmazon SES Emails in BulkAmazon SES 403 errorGetting “SignatureDoesNotMatch” error with delete Bucket Replication operation in Amazon s3Unexpected response status: 401 while accessing X-Amz-Target: Logs_20181106.GetLogEvents in AWSAWS Signature is IncorrectPut Object requests with Object Lock parameters require AWS Signature Version 4How to send gatling request with AWS Signature Version 4?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
0
This is the code for the get request I currently use to send an email through Amazon's Simple email service:
import datetime
import hashlib
import hmac
import urllib.parse
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'GET'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_headers = 'host:' + host + 'n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
canonical_querystring_part1 = 'Action=SendEmail'
'&Destination.ToAddresses.member.1='
'&Message.Body.Html.Charset=UTF-8'
'&Message.Body.Html.Data='
'&Message.Body.Text.Charset=UTF-8'
'&Message.Body.Text.Data='
'&Message.Subject.Charset=UTF-8'
'&Message.Subject.Data='
'&Source='.format(urllib.parse.quote(my_email, safe=''),
urllib.parse.quote('<b>Html Hello.</b>', safe=''),
urllib.parse.quote('Non Html hello.', safe=''),
urllib.parse.quote('Asyncio Subject line.', safe=''),
urllib.parse.quote(my_email, safe=''))
canonical_querystring_part2 = '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring_part2 += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring_part2 += '&X-Amz-Date=' + amz_date
canonical_querystring_part2 += '&X-Amz-Expires=30'
canonical_querystring_part2 += '&X-Amz-SignedHeaders=' + signed_headers
canonical_querystring = canonical_querystring_part1 + canonical_querystring_part2
payload_hash = hashlib.sha256(''.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature
request_url = endpoint + "?" + canonical_querystring
r = requests.get(request_url)
It is a bit long, but it runs fine. This is my attempt at doing the exact same thing, but with a post request:
import datetime
import hashlib
import hmac
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'POST'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
content_type = 'application/x-www-form-urlencoded; charset=utf-8'
# Request parameters for CreateTable--passed in a JSON block.
request_parameters = ''
request_parameters += "'body': 'Action': 'SendEmail', 'Destination.ToAddresses.member.1': '%s', 'Message.Body.Html.Charset': 'UTF-8', "
"'Message.Body.Html.Data': 'HTMLMESSAGE', 'Message.Body.Text.Charset': 'UTF-8', 'Message.Body.Text.Data': 'nonHTMLmessage', 'Message.Subject.Charset': 'UTF-8', "
"'Message.Subject.Data': 'subject','Source': '%s', " % (my_email, my_email)
request_parameters += "'Content-Type': '%s', " % content_type
request_parameters += "'context': 'client_region': 'us-east-1'"
request_parameters += ''
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_querystring = ''
canonical_headers = 'content-type:' + content_type + 'n' + 'host:' + host + 'n' + 'x-amz-date:' + amz_date + 'n'
signed_headers = 'content-type;host;x-amz-date'
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, date_stamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
headers = 'Content-Type': content_type, 'X-Amz-Date': amz_date, 'Authorization': authorization_header
r = requests.post(endpoint, params=request_parameters, headers=headers)
Im following Amazon's documentation as best I can for the v4 signing process here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
Im also using complete get and post request examples they have for SES found here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/query-interface-examples.html
For some reason it is not working. I tried changing the kwarg in my post request from params to json to data.
r = requests.post(endpoint, params=request_parameters, headers=headers)
<IncompleteSignatureException>
<Message>When Content-Type:application/x-www-form-urlencoded, URL cannot include query-string parameters (after '?'): '/?%7B'body':%20%7B'Action':%20'SendEmail',%20'Destination.ToAddresses.member.1':%20'ArbiBushka717@gmail.com',%20'Message.Body.Html.Charset':%20'UTF-8',%20'Message.Body.Html.Data':%20'HTMLMESSAGE',%20'Message.Body.Text.Charset':%20'UTF-8',%20'Message.Body.Text.Data':%20'nonHTMLmessage',%20'Message.Subject.Charset':%20'UTF-8',%20'Message.Subject.Data':%20'subject','Source':%20'ArbiBushka717@gmail.com'%7D,%20'Content-Type':%20'application/x-www-form-urlencoded;%20charset=utf-8',%20'context':%20%7B'client_region':%20'us-east-1'%7D%7D'</Message>
</IncompleteSignatureException>
r = requests.post(endpoint, data=request_parameters, headers=headers)
<AccessDeniedException/>
r = requests.post(endpoint, json=request_parameters, headers=headers)
<InvalidSignatureException>
<Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
</InvalidSignatureException>
Each form of post request gives a different error. Does anyone with experience in structuring post requests (or requests to Amazon's API) know what I am doing wrong?
Before anyone writes, "Use their Python SDK." I can't. Their SDK is blocking and I have to do this asynchronously. I plan to move from requests to aiohttp. I am trying to get this in requests for now because that is simpler to ask this question in.
Please help if you can, thank you for reading.
python amazon-web-services post python-requests amazon-ses
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
add a comment |
0
This is the code for the get request I currently use to send an email through Amazon's Simple email service:
import datetime
import hashlib
import hmac
import urllib.parse
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'GET'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_headers = 'host:' + host + 'n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
canonical_querystring_part1 = 'Action=SendEmail'
'&Destination.ToAddresses.member.1='
'&Message.Body.Html.Charset=UTF-8'
'&Message.Body.Html.Data='
'&Message.Body.Text.Charset=UTF-8'
'&Message.Body.Text.Data='
'&Message.Subject.Charset=UTF-8'
'&Message.Subject.Data='
'&Source='.format(urllib.parse.quote(my_email, safe=''),
urllib.parse.quote('<b>Html Hello.</b>', safe=''),
urllib.parse.quote('Non Html hello.', safe=''),
urllib.parse.quote('Asyncio Subject line.', safe=''),
urllib.parse.quote(my_email, safe=''))
canonical_querystring_part2 = '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring_part2 += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring_part2 += '&X-Amz-Date=' + amz_date
canonical_querystring_part2 += '&X-Amz-Expires=30'
canonical_querystring_part2 += '&X-Amz-SignedHeaders=' + signed_headers
canonical_querystring = canonical_querystring_part1 + canonical_querystring_part2
payload_hash = hashlib.sha256(''.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature
request_url = endpoint + "?" + canonical_querystring
r = requests.get(request_url)
It is a bit long, but it runs fine. This is my attempt at doing the exact same thing, but with a post request:
import datetime
import hashlib
import hmac
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'POST'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
content_type = 'application/x-www-form-urlencoded; charset=utf-8'
# Request parameters for CreateTable--passed in a JSON block.
request_parameters = ''
request_parameters += "'body': 'Action': 'SendEmail', 'Destination.ToAddresses.member.1': '%s', 'Message.Body.Html.Charset': 'UTF-8', "
"'Message.Body.Html.Data': 'HTMLMESSAGE', 'Message.Body.Text.Charset': 'UTF-8', 'Message.Body.Text.Data': 'nonHTMLmessage', 'Message.Subject.Charset': 'UTF-8', "
"'Message.Subject.Data': 'subject','Source': '%s', " % (my_email, my_email)
request_parameters += "'Content-Type': '%s', " % content_type
request_parameters += "'context': 'client_region': 'us-east-1'"
request_parameters += ''
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_querystring = ''
canonical_headers = 'content-type:' + content_type + 'n' + 'host:' + host + 'n' + 'x-amz-date:' + amz_date + 'n'
signed_headers = 'content-type;host;x-amz-date'
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, date_stamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
headers = 'Content-Type': content_type, 'X-Amz-Date': amz_date, 'Authorization': authorization_header
r = requests.post(endpoint, params=request_parameters, headers=headers)
Im following Amazon's documentation as best I can for the v4 signing process here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
Im also using complete get and post request examples they have for SES found here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/query-interface-examples.html
For some reason it is not working. I tried changing the kwarg in my post request from params to json to data.
r = requests.post(endpoint, params=request_parameters, headers=headers)
<IncompleteSignatureException>
<Message>When Content-Type:application/x-www-form-urlencoded, URL cannot include query-string parameters (after '?'): '/?%7B'body':%20%7B'Action':%20'SendEmail',%20'Destination.ToAddresses.member.1':%20'ArbiBushka717@gmail.com',%20'Message.Body.Html.Charset':%20'UTF-8',%20'Message.Body.Html.Data':%20'HTMLMESSAGE',%20'Message.Body.Text.Charset':%20'UTF-8',%20'Message.Body.Text.Data':%20'nonHTMLmessage',%20'Message.Subject.Charset':%20'UTF-8',%20'Message.Subject.Data':%20'subject','Source':%20'ArbiBushka717@gmail.com'%7D,%20'Content-Type':%20'application/x-www-form-urlencoded;%20charset=utf-8',%20'context':%20%7B'client_region':%20'us-east-1'%7D%7D'</Message>
</IncompleteSignatureException>
r = requests.post(endpoint, data=request_parameters, headers=headers)
<AccessDeniedException/>
r = requests.post(endpoint, json=request_parameters, headers=headers)
<InvalidSignatureException>
<Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
</InvalidSignatureException>
Each form of post request gives a different error. Does anyone with experience in structuring post requests (or requests to Amazon's API) know what I am doing wrong?
Before anyone writes, "Use their Python SDK." I can't. Their SDK is blocking and I have to do this asynchronously. I plan to move from requests to aiohttp. I am trying to get this in requests for now because that is simpler to ask this question in.
Please help if you can, thank you for reading.
python amazon-web-services post python-requests amazon-ses
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
It will be very helpful if you can post the actual request bodies
– bwest
Mar 26 at 18:30
add a comment |
0
0
0
This is the code for the get request I currently use to send an email through Amazon's Simple email service:
import datetime
import hashlib
import hmac
import urllib.parse
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'GET'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_headers = 'host:' + host + 'n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
canonical_querystring_part1 = 'Action=SendEmail'
'&Destination.ToAddresses.member.1='
'&Message.Body.Html.Charset=UTF-8'
'&Message.Body.Html.Data='
'&Message.Body.Text.Charset=UTF-8'
'&Message.Body.Text.Data='
'&Message.Subject.Charset=UTF-8'
'&Message.Subject.Data='
'&Source='.format(urllib.parse.quote(my_email, safe=''),
urllib.parse.quote('<b>Html Hello.</b>', safe=''),
urllib.parse.quote('Non Html hello.', safe=''),
urllib.parse.quote('Asyncio Subject line.', safe=''),
urllib.parse.quote(my_email, safe=''))
canonical_querystring_part2 = '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring_part2 += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring_part2 += '&X-Amz-Date=' + amz_date
canonical_querystring_part2 += '&X-Amz-Expires=30'
canonical_querystring_part2 += '&X-Amz-SignedHeaders=' + signed_headers
canonical_querystring = canonical_querystring_part1 + canonical_querystring_part2
payload_hash = hashlib.sha256(''.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature
request_url = endpoint + "?" + canonical_querystring
r = requests.get(request_url)
It is a bit long, but it runs fine. This is my attempt at doing the exact same thing, but with a post request:
import datetime
import hashlib
import hmac
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'POST'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
content_type = 'application/x-www-form-urlencoded; charset=utf-8'
# Request parameters for CreateTable--passed in a JSON block.
request_parameters = ''
request_parameters += "'body': 'Action': 'SendEmail', 'Destination.ToAddresses.member.1': '%s', 'Message.Body.Html.Charset': 'UTF-8', "
"'Message.Body.Html.Data': 'HTMLMESSAGE', 'Message.Body.Text.Charset': 'UTF-8', 'Message.Body.Text.Data': 'nonHTMLmessage', 'Message.Subject.Charset': 'UTF-8', "
"'Message.Subject.Data': 'subject','Source': '%s', " % (my_email, my_email)
request_parameters += "'Content-Type': '%s', " % content_type
request_parameters += "'context': 'client_region': 'us-east-1'"
request_parameters += ''
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_querystring = ''
canonical_headers = 'content-type:' + content_type + 'n' + 'host:' + host + 'n' + 'x-amz-date:' + amz_date + 'n'
signed_headers = 'content-type;host;x-amz-date'
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, date_stamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
headers = 'Content-Type': content_type, 'X-Amz-Date': amz_date, 'Authorization': authorization_header
r = requests.post(endpoint, params=request_parameters, headers=headers)
Im following Amazon's documentation as best I can for the v4 signing process here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
Im also using complete get and post request examples they have for SES found here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/query-interface-examples.html
For some reason it is not working. I tried changing the kwarg in my post request from params to json to data.
r = requests.post(endpoint, params=request_parameters, headers=headers)
<IncompleteSignatureException>
<Message>When Content-Type:application/x-www-form-urlencoded, URL cannot include query-string parameters (after '?'): '/?%7B'body':%20%7B'Action':%20'SendEmail',%20'Destination.ToAddresses.member.1':%20'ArbiBushka717@gmail.com',%20'Message.Body.Html.Charset':%20'UTF-8',%20'Message.Body.Html.Data':%20'HTMLMESSAGE',%20'Message.Body.Text.Charset':%20'UTF-8',%20'Message.Body.Text.Data':%20'nonHTMLmessage',%20'Message.Subject.Charset':%20'UTF-8',%20'Message.Subject.Data':%20'subject','Source':%20'ArbiBushka717@gmail.com'%7D,%20'Content-Type':%20'application/x-www-form-urlencoded;%20charset=utf-8',%20'context':%20%7B'client_region':%20'us-east-1'%7D%7D'</Message>
</IncompleteSignatureException>
r = requests.post(endpoint, data=request_parameters, headers=headers)
<AccessDeniedException/>
r = requests.post(endpoint, json=request_parameters, headers=headers)
<InvalidSignatureException>
<Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
</InvalidSignatureException>
Each form of post request gives a different error. Does anyone with experience in structuring post requests (or requests to Amazon's API) know what I am doing wrong?
Before anyone writes, "Use their Python SDK." I can't. Their SDK is blocking and I have to do this asynchronously. I plan to move from requests to aiohttp. I am trying to get this in requests for now because that is simpler to ask this question in.
Please help if you can, thank you for reading.
python amazon-web-services post python-requests amazon-ses
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
This is the code for the get request I currently use to send an email through Amazon's Simple email service:
import datetime
import hashlib
import hmac
import urllib.parse
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'GET'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_headers = 'host:' + host + 'n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
canonical_querystring_part1 = 'Action=SendEmail'
'&Destination.ToAddresses.member.1='
'&Message.Body.Html.Charset=UTF-8'
'&Message.Body.Html.Data='
'&Message.Body.Text.Charset=UTF-8'
'&Message.Body.Text.Data='
'&Message.Subject.Charset=UTF-8'
'&Message.Subject.Data='
'&Source='.format(urllib.parse.quote(my_email, safe=''),
urllib.parse.quote('<b>Html Hello.</b>', safe=''),
urllib.parse.quote('Non Html hello.', safe=''),
urllib.parse.quote('Asyncio Subject line.', safe=''),
urllib.parse.quote(my_email, safe=''))
canonical_querystring_part2 = '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring_part2 += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring_part2 += '&X-Amz-Date=' + amz_date
canonical_querystring_part2 += '&X-Amz-Expires=30'
canonical_querystring_part2 += '&X-Amz-SignedHeaders=' + signed_headers
canonical_querystring = canonical_querystring_part1 + canonical_querystring_part2
payload_hash = hashlib.sha256(''.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature
request_url = endpoint + "?" + canonical_querystring
r = requests.get(request_url)
It is a bit long, but it runs fine. This is my attempt at doing the exact same thing, but with a post request:
import datetime
import hashlib
import hmac
import requests
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
method = 'POST'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
access_key = 'my_access_key'
secret_key = 'my_secret_key'
my_email = 'my_email_address'
content_type = 'application/x-www-form-urlencoded; charset=utf-8'
# Request parameters for CreateTable--passed in a JSON block.
request_parameters = ''
request_parameters += "'body': 'Action': 'SendEmail', 'Destination.ToAddresses.member.1': '%s', 'Message.Body.Html.Charset': 'UTF-8', "
"'Message.Body.Html.Data': 'HTMLMESSAGE', 'Message.Body.Text.Charset': 'UTF-8', 'Message.Body.Text.Data': 'nonHTMLmessage', 'Message.Subject.Charset': 'UTF-8', "
"'Message.Subject.Data': 'subject','Source': '%s', " % (my_email, my_email)
request_parameters += "'Content-Type': '%s', " % content_type
request_parameters += "'context': 'client_region': 'us-east-1'"
request_parameters += ''
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_querystring = ''
canonical_headers = 'content-type:' + content_type + 'n' + 'host:' + host + 'n' + 'x-amz-date:' + amz_date + 'n'
signed_headers = 'content-type;host;x-amz-date'
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()
canonical_request = method + 'n' + canonical_uri + 'n' + canonical_querystring + 'n' + canonical_headers + 'n' + signed_headers + 'n' + payload_hash
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + 'n' + amz_date + 'n' + credential_scope + 'n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, date_stamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode('utf-8'), hashlib.sha256).hexdigest()
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
headers = 'Content-Type': content_type, 'X-Amz-Date': amz_date, 'Authorization': authorization_header
r = requests.post(endpoint, params=request_parameters, headers=headers)
Im following Amazon's documentation as best I can for the v4 signing process here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
Im also using complete get and post request examples they have for SES found here: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/query-interface-examples.html
For some reason it is not working. I tried changing the kwarg in my post request from params to json to data.
r = requests.post(endpoint, params=request_parameters, headers=headers)
<IncompleteSignatureException>
<Message>When Content-Type:application/x-www-form-urlencoded, URL cannot include query-string parameters (after '?'): '/?%7B'body':%20%7B'Action':%20'SendEmail',%20'Destination.ToAddresses.member.1':%20'ArbiBushka717@gmail.com',%20'Message.Body.Html.Charset':%20'UTF-8',%20'Message.Body.Html.Data':%20'HTMLMESSAGE',%20'Message.Body.Text.Charset':%20'UTF-8',%20'Message.Body.Text.Data':%20'nonHTMLmessage',%20'Message.Subject.Charset':%20'UTF-8',%20'Message.Subject.Data':%20'subject','Source':%20'ArbiBushka717@gmail.com'%7D,%20'Content-Type':%20'application/x-www-form-urlencoded;%20charset=utf-8',%20'context':%20%7B'client_region':%20'us-east-1'%7D%7D'</Message>
</IncompleteSignatureException>
r = requests.post(endpoint, data=request_parameters, headers=headers)
<AccessDeniedException/>
r = requests.post(endpoint, json=request_parameters, headers=headers)
<InvalidSignatureException>
<Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
</InvalidSignatureException>
Each form of post request gives a different error. Does anyone with experience in structuring post requests (or requests to Amazon's API) know what I am doing wrong?
Before anyone writes, "Use their Python SDK." I can't. Their SDK is blocking and I have to do this asynchronously. I plan to move from requests to aiohttp. I am trying to get this in requests for now because that is simpler to ask this question in.
Please help if you can, thank you for reading.
python amazon-web-services post python-requests amazon-ses
python amazon-web-services post python-requests amazon-ses
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
asked Mar 26 at 17:26
Arbi BushkaArbi Bushka
456 bronze badges
456 bronze badges
It will be very helpful if you can post the actual request bodies
– bwest
Mar 26 at 18:30
add a comment |
It will be very helpful if you can post the actual request bodies
– bwest
Mar 26 at 18:30
It will be very helpful if you can post the actual request bodies
– bwest
Mar 26 at 18:30
It will be very helpful if you can post the actual request bodies
– bwest
Mar 26 at 18:30
add a comment |
1 Answer
1
active
oldest
votes
0
wait nvm I got to. To specify a body in requests you use the "data" kwarg.
Also my variable request_parameters needed to be in the format param1=This¶m2=that
Wow what a headache.
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55362998%2fwhy-does-amazon-ses-accept-my-get-requests-but-deny-my-similarly-structured-pos%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
wait nvm I got to. To specify a body in requests you use the "data" kwarg.
Also my variable request_parameters needed to be in the format param1=This¶m2=that
Wow what a headache.
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
add a comment |
0
wait nvm I got to. To specify a body in requests you use the "data" kwarg.
Also my variable request_parameters needed to be in the format param1=This¶m2=that
Wow what a headache.
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
add a comment |
0
0
0
wait nvm I got to. To specify a body in requests you use the "data" kwarg.
Also my variable request_parameters needed to be in the format param1=This¶m2=that
Wow what a headache.
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
wait nvm I got to. To specify a body in requests you use the "data" kwarg.
Also my variable request_parameters needed to be in the format param1=This¶m2=that
Wow what a headache.
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
answered Mar 26 at 18:46
Arbi BushkaArbi Bushka
456 bronze badges
456 bronze badges
add a comment |
add a comment |
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55362998%2fwhy-does-amazon-ses-accept-my-get-requests-but-deny-my-similarly-structured-pos%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It will be very helpful if you can post the actual request bodies
– bwest
Mar 26 at 18:30