Upload a file to s3 given a https s3 urlWriting files in Node.jsCheck synchronously if file/directory exists in Node.jsWhat's the difference between dependencies, devDependencies and peerDependencies in npm package.json file?Catching async exceptions from CORS requestPre-Signed S3 URL Signature Does Not MatchDownload AWS S3 file from EC2 instanceS3: No Access-Control-Allow-Origin header is present on the requested resourceupload file to S3 via cURL using presigned URLAWS S3 Signature Does Not MatchAWS Get Pre-Signed URL with custom domain
Why do private jets such as Gulfstream fly higher than other civilian jets?
Traveling from Germany to other countries by train?
Count number of occurences of particular numbers in list
Does this put me at risk for identity theft?
12V lead acid charger with LM317 not charging
Can chords be inferred from melody alone?
How to help new students accept function notation
What are these mathematical groups in U.S. universities?
Print only the last three columns from file
What is the bio-mechanical plausibility of a fox with venomous fangs?
Why don't the open notes matter in guitar chords?
Can I enter a rental property without giving notice if I'm afraid a tenant may be hurt?
How to continue a line in Latex in math mode?
Why is Chromosome 1 called Chromosome 1?
Based on what criteria do you add/not add icons to labels within a toolbar?
Are children a reason to be rejected for a job?
What word best describes someone who likes to do everything on his own?
Why do proponents of guns oppose gun competency tests?
Is there such thing as a "3-dimensional surface?"
What does VB stand for?
Is it a bad idea to offer variants of a final exam based on the type of allowed calculators?
Who is the god Ao?
I was contacted by a private bank overseas to get my inheritance
If someone else uploads my GPL'd code to Github without my permission, do they not put it at risk of being stolen by Microsoft?
Upload a file to s3 given a https s3 url
Writing files in Node.jsCheck synchronously if file/directory exists in Node.jsWhat's the difference between dependencies, devDependencies and peerDependencies in npm package.json file?Catching async exceptions from CORS requestPre-Signed S3 URL Signature Does Not MatchDownload AWS S3 file from EC2 instanceS3: No Access-Control-Allow-Origin header is present on the requested resourceupload file to S3 via cURL using presigned URLAWS S3 Signature Does Not MatchAWS Get Pre-Signed URL with custom domain
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I'm trying to copy an object from one an s3 bucket to my s3 bucket, but I'm given a http url of the file like this;
https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348
If i throw this is the browser i get a csv no worries at all (note: i've obscured parts of the real url so it won't work for you).
I'm using copyObject API in node with the following parameters:
var params =
Bucket: "me-buket",
CopySource: "https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348"
, Key: "data"
;
I keep getting the error:
"InvalidArgument: Unsupported copy source parameter."
What am I doing wrong??
node.js amazon-s3 aws-sdk
asked Mar 27 at 5:13
RedRed
133 bronze badges
add a comment |
I'm trying to copy an object from one an s3 bucket to my s3 bucket, but I'm given a http url of the file like this;
https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348
If i throw this is the browser i get a csv no worries at all (note: i've obscured parts of the real url so it won't work for you).
I'm using copyObject API in node with the following parameters:
var params =
Bucket: "me-buket",
CopySource: "https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348"
, Key: "data"
;
I keep getting the error:
"InvalidArgument: Unsupported copy source parameter."
What am I doing wrong??
node.js amazon-s3 aws-sdk
asked Mar 27 at 5:13
RedRed
133 bronze badges
add a comment |
I'm trying to copy an object from one an s3 bucket to my s3 bucket, but I'm given a http url of the file like this;
https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348
If i throw this is the browser i get a csv no worries at all (note: i've obscured parts of the real url so it won't work for you).
I'm using copyObject API in node with the following parameters:
var params =
Bucket: "me-buket",
CopySource: "https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348"
, Key: "data"
;
I keep getting the error:
"InvalidArgument: Unsupported copy source parameter."
What am I doing wrong??
node.js amazon-s3 aws-sdk
asked Mar 27 at 5:13
RedRed
133 bronze badges
I'm trying to copy an object from one an s3 bucket to my s3 bucket, but I'm given a http url of the file like this;
https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348
If i throw this is the browser i get a csv no worries at all (note: i've obscured parts of the real url so it won't work for you).
I'm using copyObject API in node with the following parameters:
var params =
Bucket: "me-buket",
CopySource: "https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348"
, Key: "data"
;
I keep getting the error:
"InvalidArgument: Unsupported copy source parameter."
What am I doing wrong??
node.js amazon-s3 aws-sdk
node.js amazon-s3 aws-sdk
asked Mar 27 at 5:13
RedRed
133 bronze badges
asked Mar 27 at 5:13
RedRed
133 bronze badges
asked Mar 27 at 5:13
RedRed
133 bronze badges
asked Mar 27 at 5:13
RedRed
133 bronze badges
asked Mar 27 at 5:13
RedRed
133 bronze badges
133 bronze badges
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
S3 does not support using a pre-signed URL (or any other URL) as CopySource. The value you want is /$bucket/$key which in this case is /not-my-bucket/1234567381...acZMLZ.csv.gz (the leading https:// replaced by a leading slash, the .s3.amazonaws.com removed, and the ? and everything following it removed).
But... that still isn't likely to work, unless the user invoking copyObject() has permission to read the source object directly, which is unlikely... otherwise the third party wouldn't have given you the pre-signed URL.
"Fixing" your request as described above will probably change the error to AccessDenied.
Without permission to read the object directly, you can't use copyObject(). You'll need to download the file and then upload it to your bucket in two steps.
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
add a comment |
The above error indicates that the CopySource paramenter which you are giving is incorrect. The correct way to give CopySource parameter will be as follows:
var params = Bucket: "your_bucket", CopySource: "/source_bucketname_url/sourceobjectkey";
Note : The name of the source bucket and key name of the source object must be separated by a slash (/).
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55370204%2fupload-a-file-to-s3-given-a-https-s3-url%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
S3 does not support using a pre-signed URL (or any other URL) as CopySource. The value you want is /$bucket/$key which in this case is /not-my-bucket/1234567381...acZMLZ.csv.gz (the leading https:// replaced by a leading slash, the .s3.amazonaws.com removed, and the ? and everything following it removed).
But... that still isn't likely to work, unless the user invoking copyObject() has permission to read the source object directly, which is unlikely... otherwise the third party wouldn't have given you the pre-signed URL.
"Fixing" your request as described above will probably change the error to AccessDenied.
Without permission to read the object directly, you can't use copyObject(). You'll need to download the file and then upload it to your bucket in two steps.
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
add a comment |
S3 does not support using a pre-signed URL (or any other URL) as CopySource. The value you want is /$bucket/$key which in this case is /not-my-bucket/1234567381...acZMLZ.csv.gz (the leading https:// replaced by a leading slash, the .s3.amazonaws.com removed, and the ? and everything following it removed).
But... that still isn't likely to work, unless the user invoking copyObject() has permission to read the source object directly, which is unlikely... otherwise the third party wouldn't have given you the pre-signed URL.
"Fixing" your request as described above will probably change the error to AccessDenied.
Without permission to read the object directly, you can't use copyObject(). You'll need to download the file and then upload it to your bucket in two steps.
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
add a comment |
S3 does not support using a pre-signed URL (or any other URL) as CopySource. The value you want is /$bucket/$key which in this case is /not-my-bucket/1234567381...acZMLZ.csv.gz (the leading https:// replaced by a leading slash, the .s3.amazonaws.com removed, and the ? and everything following it removed).
But... that still isn't likely to work, unless the user invoking copyObject() has permission to read the source object directly, which is unlikely... otherwise the third party wouldn't have given you the pre-signed URL.
"Fixing" your request as described above will probably change the error to AccessDenied.
Without permission to read the object directly, you can't use copyObject(). You'll need to download the file and then upload it to your bucket in two steps.
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
S3 does not support using a pre-signed URL (or any other URL) as CopySource. The value you want is /$bucket/$key which in this case is /not-my-bucket/1234567381...acZMLZ.csv.gz (the leading https:// replaced by a leading slash, the .s3.amazonaws.com removed, and the ? and everything following it removed).
But... that still isn't likely to work, unless the user invoking copyObject() has permission to read the source object directly, which is unlikely... otherwise the third party wouldn't have given you the pre-signed URL.
"Fixing" your request as described above will probably change the error to AccessDenied.
Without permission to read the object directly, you can't use copyObject(). You'll need to download the file and then upload it to your bucket in two steps.
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
answered Mar 27 at 21:46
Michael - sqlbotMichael - sqlbot
104k14 gold badges157 silver badges221 bronze badges
104k14 gold badges157 silver badges221 bronze badges
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
add a comment |
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
Yep this resolves it, I'm just going to have to etl some other way
– Red
Mar 28 at 7:45
add a comment |
The above error indicates that the CopySource paramenter which you are giving is incorrect. The correct way to give CopySource parameter will be as follows:
var params = Bucket: "your_bucket", CopySource: "/source_bucketname_url/sourceobjectkey";
Note : The name of the source bucket and key name of the source object must be separated by a slash (/).
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
add a comment |
The above error indicates that the CopySource paramenter which you are giving is incorrect. The correct way to give CopySource parameter will be as follows:
var params = Bucket: "your_bucket", CopySource: "/source_bucketname_url/sourceobjectkey";
Note : The name of the source bucket and key name of the source object must be separated by a slash (/).
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
add a comment |
The above error indicates that the CopySource paramenter which you are giving is incorrect. The correct way to give CopySource parameter will be as follows:
var params = Bucket: "your_bucket", CopySource: "/source_bucketname_url/sourceobjectkey";
Note : The name of the source bucket and key name of the source object must be separated by a slash (/).
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
The above error indicates that the CopySource paramenter which you are giving is incorrect. The correct way to give CopySource parameter will be as follows:
var params = Bucket: "your_bucket", CopySource: "/source_bucketname_url/sourceobjectkey";
Note : The name of the source bucket and key name of the source object must be separated by a slash (/).
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
answered Mar 27 at 6:24
Aress SupportAress Support
8171 silver badge11 bronze badges
8171 silver badge11 bronze badges
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
add a comment |
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
Thanks for that, but authentication still causes problems
– Red
Mar 28 at 7:44
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55370204%2fupload-a-file-to-s3-given-a-https-s3-url%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
