How to pass multiple ids in batches and than merge it in RHow to sort a dataframe by multiple column(s)How to join (merge) data frames (inner, outer, left, right)How to make a great R reproducible exampleSimultaneously merge multiple data.frames in a listWhy is `[` better than `subset`?How can we make xkcd style graphs?Group by multiple columns in dplyr, using string vector inputFixing a multiple warning “unknown column”Passing column names through multiple functions with dplyrPassing multiple arguments to function in dplyr::summarise_if
Ethernet, Wifi and a little human psychology
geschafft or geschaffen? which one is past participle of schaffen?
Wrong Schengen Visa exit stamp on my passport, who can I complain to?
Help with wheel lock
Permutations in Disguise
What are the advantages and disadvantages of tail wheels that cause modern airplanes to not use them?
Python web-scraper to download table of transistor counts from Wikipedia
Why is belonging not transitive?
What is the source of "You can achieve a lot with hate, but even more with love" (Shakespeare?)
Exam design: give maximum score per question or not?
Make 2019 with single digits
Why does an orbit become hyperbolic when total orbital energy is positive?
Is it appropriate to CC a lot of people on an email
Why is the UK still pressing on with Brexit?
Asked to Not Use Transactions and to Use A Workaround to Simulate One
What does this line from The hobbit mean?
Is there any reason to concentrate on the Thunderous Smite spell after using its effects?
What is a "major country" as named in Bernie Sanders' Healthcare debate answers?
Bit one of the Intel 8080's Flags register
How does a simple logistic regression model achieve a 92% classification accuracy on MNIST?
Answer Not A Fool, or Answer A Fool?
In what sequence should an advanced civilization teach technology to medieval society to maximize rate of adoption?
How can I use expandafter the expand the definition of this control sequence?
Why are some files not movable on Windows 10?
How to pass multiple ids in batches and than merge it in R
How to sort a dataframe by multiple column(s)How to join (merge) data frames (inner, outer, left, right)How to make a great R reproducible exampleSimultaneously merge multiple data.frames in a listWhy is `[` better than `subset`?How can we make xkcd style graphs?Group by multiple columns in dplyr, using string vector inputFixing a multiple warning “unknown column”Passing column names through multiple functions with dplyrPassing multiple arguments to function in dplyr::summarise_if
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have below mentioned dataframe in R.
DF1
ID Sales Cost Value
RTT-123 10 10000 15000
RTT-456 15 12000 17000
RTT-789 14 14000 19000
The dataframe containst almost ~30K unique Ids, while passing these ids to redshift using below mentioned query, I am getting error Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
How to pass these Ids automatically in the batch of 2K ids while querying and then merge the output to one single data frame in R.
Query:
df2<-paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (", paste(shQuote(DF1$ID , type = "sh"),collapse = ','),");")
output<-dbGetQuery(link,df2)
r dplyr tidyverse
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
add a comment
|
I have below mentioned dataframe in R.
DF1
ID Sales Cost Value
RTT-123 10 10000 15000
RTT-456 15 12000 17000
RTT-789 14 14000 19000
The dataframe containst almost ~30K unique Ids, while passing these ids to redshift using below mentioned query, I am getting error Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
How to pass these Ids automatically in the batch of 2K ids while querying and then merge the output to one single data frame in R.
Query:
df2<-paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (", paste(shQuote(DF1$ID , type = "sh"),collapse = ','),");")
output<-dbGetQuery(link,df2)
r dplyr tidyverse
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
add a comment
|
I have below mentioned dataframe in R.
DF1
ID Sales Cost Value
RTT-123 10 10000 15000
RTT-456 15 12000 17000
RTT-789 14 14000 19000
The dataframe containst almost ~30K unique Ids, while passing these ids to redshift using below mentioned query, I am getting error Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
How to pass these Ids automatically in the batch of 2K ids while querying and then merge the output to one single data frame in R.
Query:
df2<-paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (", paste(shQuote(DF1$ID , type = "sh"),collapse = ','),");")
output<-dbGetQuery(link,df2)
r dplyr tidyverse
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
I have below mentioned dataframe in R.
DF1
ID Sales Cost Value
RTT-123 10 10000 15000
RTT-456 15 12000 17000
RTT-789 14 14000 19000
The dataframe containst almost ~30K unique Ids, while passing these ids to redshift using below mentioned query, I am getting error Error in .verify.JDBC.result(r, "Unable to retrieve JDBC result set for ", :
How to pass these Ids automatically in the batch of 2K ids while querying and then merge the output to one single data frame in R.
Query:
df2<-paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (", paste(shQuote(DF1$ID , type = "sh"),collapse = ','),");")
output<-dbGetQuery(link,df2)
r dplyr tidyverse
r dplyr tidyverse
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
asked Mar 28 at 11:37
Vector JXVector JX
2472 silver badges15 bronze badges
2472 silver badges15 bronze badges
add a comment
|
add a comment
|
1 Answer
1
active
oldest
votes
Something like this (not tested), here we are using 1000 IDs at a time, adjust as per your needs:
library(data.table) # rbindlist
output <- rbindlist(
lapply(
# 1000 chunks
split(DF1$ID, ceiling(seq_along(DF1$ID)/1000)),
function(i)
df2 <- paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (",
paste(shQuote(i , type = "sh"), collapse = ','),
");")
dbGetQuery(link, df2)
))
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
add a comment
|
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55396644%2fhow-to-pass-multiple-ids-in-batches-and-than-merge-it-in-r%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Something like this (not tested), here we are using 1000 IDs at a time, adjust as per your needs:
library(data.table) # rbindlist
output <- rbindlist(
lapply(
# 1000 chunks
split(DF1$ID, ceiling(seq_along(DF1$ID)/1000)),
function(i)
df2 <- paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (",
paste(shQuote(i , type = "sh"), collapse = ','),
");")
dbGetQuery(link, df2)
))
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
add a comment
|
Something like this (not tested), here we are using 1000 IDs at a time, adjust as per your needs:
library(data.table) # rbindlist
output <- rbindlist(
lapply(
# 1000 chunks
split(DF1$ID, ceiling(seq_along(DF1$ID)/1000)),
function(i)
df2 <- paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (",
paste(shQuote(i , type = "sh"), collapse = ','),
");")
dbGetQuery(link, df2)
))
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
add a comment
|
Something like this (not tested), here we are using 1000 IDs at a time, adjust as per your needs:
library(data.table) # rbindlist
output <- rbindlist(
lapply(
# 1000 chunks
split(DF1$ID, ceiling(seq_along(DF1$ID)/1000)),
function(i)
df2 <- paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (",
paste(shQuote(i , type = "sh"), collapse = ','),
");")
dbGetQuery(link, df2)
))
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
Something like this (not tested), here we are using 1000 IDs at a time, adjust as per your needs:
library(data.table) # rbindlist
output <- rbindlist(
lapply(
# 1000 chunks
split(DF1$ID, ceiling(seq_along(DF1$ID)/1000)),
function(i)
df2 <- paste0("SELECT ID,list1,list2, date1 FROM table1 b
WHERE b.ID IN (",
paste(shQuote(i , type = "sh"), collapse = ','),
");")
dbGetQuery(link, df2)
))
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
answered Mar 28 at 11:48
zx8754zx8754
32.6k8 gold badges72 silver badges114 bronze badges
32.6k8 gold badges72 silver badges114 bronze badges
add a comment
|
add a comment
|
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55396644%2fhow-to-pass-multiple-ids-in-batches-and-than-merge-it-in-r%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
