Efficient ways to summarize array in R depending on other data The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) The Ask Question Wizard is Live! Data science time! April 2019 and salary with experiencebest way to store data locally and update from web from time to time?create sequence of numbers from time dependent panel dataWhat's the most secure way to create database views in a data layer environment for many consumers?Best way to store company and employee dataCustom data-dependent recoding to logicals in Rdatatable: add observations to all groups and assign value that depends on other rows
Create an outline of font
Sort a list of pairs representing an acyclic, partial automorphism
Windows 10: How to Lock (not sleep) laptop on lid close?
Is this wall load bearing? Blueprints and photos attached
Typeface like Times New Roman but with "tied" percent sign
Can withdrawing asylum be illegal?
Who or what is the being for whom Being is a question for Heidegger?
How to politely respond to generic emails requesting a PhD/job in my lab? Without wasting too much time
What information about me do stores get via my credit card?
How to delete random line from file using Unix command?
How can I protect witches in combat who wear limited clothing?
The variadic template constructor of my class cannot modify my class members, why is that so?
does high air pressure throw off wheel balance?
First use of “packing” as in carrying a gun
Why does the Event Horizon Telescope (EHT) not include telescopes from Africa, Asia or Australia?
How should I replace vector<uint8_t>::const_iterator in an API?
Semisimplicity of the category of coherent sheaves?
Is it ok to offer lower paid work as a trial period before negotiating for a full-time job?
How to stretch delimiters to envolve matrices inside of a kbordermatrix?
How are presidential pardons supposed to be used?
Working through the single responsibility principle (SRP) in Python when calls are expensive
"... to apply for a visa" or "... and applied for a visa"?
Change bounding box of math glyphs in LuaTeX
Scientific Reports - Significant Figures
Efficient ways to summarize array in R depending on other data
The 2019 Stack Overflow Developer Survey Results Are In
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
The Ask Question Wizard is Live!
Data science time! April 2019 and salary with experiencebest way to store data locally and update from web from time to time?create sequence of numbers from time dependent panel dataWhat's the most secure way to create database views in a data layer environment for many consumers?Best way to store company and employee dataCustom data-dependent recoding to logicals in Rdatatable: add observations to all groups and assign value that depends on other rows
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?
Below are two approaches I tried. Although they seem to work, they take too long to scale...
Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:
A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
X <- as.logical(rbinom(n.i, 1, 0.49))
Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)
In my case n.i <- 10^5
n.t <- 10^2
n.k <-10
Approach 1 — Vectorized approach:
result <- matrix(0, nrow=n.t, ncol=4)
count <- matrix(0, nrow=n.i, ncol=n.t)
heads <- A=="H"
for (x in 0:1) # male or female
for (y in 0:1) # dime or quarter
count <- 0
for (k in 1:n.k)
count <- count + 1*(X==x & Y[,k]==y & heads[,,k])
result[,1+x+2*(y-1)] <- colSums(count>0)
Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k))
. Then use apply():
for (x in 0:1)
for (y in 0:1)
result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)
Anyone any better solutions?
data-management
add a comment |
I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?
Below are two approaches I tried. Although they seem to work, they take too long to scale...
Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:
A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
X <- as.logical(rbinom(n.i, 1, 0.49))
Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)
In my case n.i <- 10^5
n.t <- 10^2
n.k <-10
Approach 1 — Vectorized approach:
result <- matrix(0, nrow=n.t, ncol=4)
count <- matrix(0, nrow=n.i, ncol=n.t)
heads <- A=="H"
for (x in 0:1) # male or female
for (y in 0:1) # dime or quarter
count <- 0
for (k in 1:n.k)
count <- count + 1*(X==x & Y[,k]==y & heads[,,k])
result[,1+x+2*(y-1)] <- colSums(count>0)
Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k))
. Then use apply():
for (x in 0:1)
for (y in 0:1)
result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)
Anyone any better solutions?
data-management
add a comment |
I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?
Below are two approaches I tried. Although they seem to work, they take too long to scale...
Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:
A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
X <- as.logical(rbinom(n.i, 1, 0.49))
Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)
In my case n.i <- 10^5
n.t <- 10^2
n.k <-10
Approach 1 — Vectorized approach:
result <- matrix(0, nrow=n.t, ncol=4)
count <- matrix(0, nrow=n.i, ncol=n.t)
heads <- A=="H"
for (x in 0:1) # male or female
for (y in 0:1) # dime or quarter
count <- 0
for (k in 1:n.k)
count <- count + 1*(X==x & Y[,k]==y & heads[,,k])
result[,1+x+2*(y-1)] <- colSums(count>0)
Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k))
. Then use apply():
for (x in 0:1)
for (y in 0:1)
result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)
Anyone any better solutions?
data-management
I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?
Below are two approaches I tried. Although they seem to work, they take too long to scale...
Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:
A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
X <- as.logical(rbinom(n.i, 1, 0.49))
Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)
In my case n.i <- 10^5
n.t <- 10^2
n.k <-10
Approach 1 — Vectorized approach:
result <- matrix(0, nrow=n.t, ncol=4)
count <- matrix(0, nrow=n.i, ncol=n.t)
heads <- A=="H"
for (x in 0:1) # male or female
for (y in 0:1) # dime or quarter
count <- 0
for (k in 1:n.k)
count <- count + 1*(X==x & Y[,k]==y & heads[,,k])
result[,1+x+2*(y-1)] <- colSums(count>0)
Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k))
. Then use apply():
for (x in 0:1)
for (y in 0:1)
result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)
Anyone any better solutions?
data-management
data-management
edited Mar 25 at 18:31
Reinier Meester
asked Mar 22 at 6:25
Reinier MeesterReinier Meester
12
12
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55293990%2fefficient-ways-to-summarize-array-in-r-depending-on-other-data%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55293990%2fefficient-ways-to-summarize-array-in-r-depending-on-other-data%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown