Efficient ways to summarize array in R depending on other data The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) The Ask Question Wizard is Live! Data science time! April 2019 and salary with experiencebest way to store data locally and update from web from time to time?create sequence of numbers from time dependent panel dataWhat's the most secure way to create database views in a data layer environment for many consumers?Best way to store company and employee dataCustom data-dependent recoding to logicals in Rdatatable: add observations to all groups and assign value that depends on other rows

Create an outline of font

Sort a list of pairs representing an acyclic, partial automorphism

Windows 10: How to Lock (not sleep) laptop on lid close?

Is this wall load bearing? Blueprints and photos attached

Typeface like Times New Roman but with "tied" percent sign

Can withdrawing asylum be illegal?

Who or what is the being for whom Being is a question for Heidegger?

How to politely respond to generic emails requesting a PhD/job in my lab? Without wasting too much time

What information about me do stores get via my credit card?

How to delete random line from file using Unix command?

How can I protect witches in combat who wear limited clothing?

The variadic template constructor of my class cannot modify my class members, why is that so?

does high air pressure throw off wheel balance?

First use of “packing” as in carrying a gun

Why does the Event Horizon Telescope (EHT) not include telescopes from Africa, Asia or Australia?

How should I replace vector<uint8_t>::const_iterator in an API?

Semisimplicity of the category of coherent sheaves?

Is it ok to offer lower paid work as a trial period before negotiating for a full-time job?

How to stretch delimiters to envolve matrices inside of a kbordermatrix?

How are presidential pardons supposed to be used?

Working through the single responsibility principle (SRP) in Python when calls are expensive

"... to apply for a visa" or "... and applied for a visa"?

Change bounding box of math glyphs in LuaTeX

Scientific Reports - Significant Figures



Efficient ways to summarize array in R depending on other data



The 2019 Stack Overflow Developer Survey Results Are In
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
The Ask Question Wizard is Live!
Data science time! April 2019 and salary with experiencebest way to store data locally and update from web from time to time?create sequence of numbers from time dependent panel dataWhat's the most secure way to create database views in a data layer environment for many consumers?Best way to store company and employee dataCustom data-dependent recoding to logicals in Rdatatable: add observations to all groups and assign value that depends on other rows



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








0















I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?



Below are two approaches I tried. Although they seem to work, they take too long to scale...



Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:



A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
X <- as.logical(rbinom(n.i, 1, 0.49))
Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)


In my case n.i <- 10^5 n.t <- 10^2 n.k <-10



Approach 1 — Vectorized approach:



result <- matrix(0, nrow=n.t, ncol=4)
count <- matrix(0, nrow=n.i, ncol=n.t)
heads <- A=="H"
for (x in 0:1) # male or female
for (y in 0:1) # dime or quarter
count <- 0
for (k in 1:n.k)
count <- count + 1*(X==x & Y[,k]==y & heads[,,k])

result[,1+x+2*(y-1)] <- colSums(count>0)




Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k)). Then use apply():



for (x in 0:1) 
for (y in 0:1)
result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)




Anyone any better solutions?










share|improve this question






























    0















    I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?



    Below are two approaches I tried. Although they seem to work, they take too long to scale...



    Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:



    A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
    X <- as.logical(rbinom(n.i, 1, 0.49))
    Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)


    In my case n.i <- 10^5 n.t <- 10^2 n.k <-10



    Approach 1 — Vectorized approach:



    result <- matrix(0, nrow=n.t, ncol=4)
    count <- matrix(0, nrow=n.i, ncol=n.t)
    heads <- A=="H"
    for (x in 0:1) # male or female
    for (y in 0:1) # dime or quarter
    count <- 0
    for (k in 1:n.k)
    count <- count + 1*(X==x & Y[,k]==y & heads[,,k])

    result[,1+x+2*(y-1)] <- colSums(count>0)




    Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k)). Then use apply():



    for (x in 0:1) 
    for (y in 0:1)
    result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)




    Anyone any better solutions?










    share|improve this question


























      0












      0








      0








      I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?



      Below are two approaches I tried. Although they seem to work, they take too long to scale...



      Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:



      A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
      X <- as.logical(rbinom(n.i, 1, 0.49))
      Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)


      In my case n.i <- 10^5 n.t <- 10^2 n.k <-10



      Approach 1 — Vectorized approach:



      result <- matrix(0, nrow=n.t, ncol=4)
      count <- matrix(0, nrow=n.i, ncol=n.t)
      heads <- A=="H"
      for (x in 0:1) # male or female
      for (y in 0:1) # dime or quarter
      count <- 0
      for (k in 1:n.k)
      count <- count + 1*(X==x & Y[,k]==y & heads[,,k])

      result[,1+x+2*(y-1)] <- colSums(count>0)




      Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k)). Then use apply():



      for (x in 0:1) 
      for (y in 0:1)
      result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)




      Anyone any better solutions?










      share|improve this question
















      I want to summarize outcomes from a 3-dimensional array contingent on information from two other datasets. Say, the number of individuals i who after flipping k coins in year t (array) have at least 1 head, with results organized by the individual's sex (vector) and the coin they used for each flip, dime or quarter (matrix). What is the best way to achieve this?



      Below are two approaches I tried. Although they seem to work, they take too long to scale...



      Let A be the array storing the coin flips, X the vector storing people's sex, and Y the matrix storing the coins used:



      A <- array(sample(c("H","T"), size=n.i*n.t*n.k, replace=T), dim=c(n.i, n.t, n.k))
      X <- as.logical(rbinom(n.i, 1, 0.49))
      Y <- matrix(as.logical(rbinom(n.i*n.p, 1, 0.3)), nrow=n.i, ncol=n.k)


      In my case n.i <- 10^5 n.t <- 10^2 n.k <-10



      Approach 1 — Vectorized approach:



      result <- matrix(0, nrow=n.t, ncol=4)
      count <- matrix(0, nrow=n.i, ncol=n.t)
      heads <- A=="H"
      for (x in 0:1) # male or female
      for (y in 0:1) # dime or quarter
      count <- 0
      for (k in 1:n.k)
      count <- count + 1*(X==x & Y[,k]==y & heads[,,k])

      result[,1+x+2*(y-1)] <- colSums(count>0)




      Approach 2 — Expand X and Y to similar dimensions as A, fixing values along the k and t axis, eg: X <- array(X, dim=c(n.i, n.t, n.k)). Then use apply():



      for (x in 0:1) 
      for (y in 0:1)
      result[,1+x+2*(y-1)] <- apply(apply(X==x & Y==y & heads, 3, sum)>0, 2, sum)




      Anyone any better solutions?







      data-management






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Mar 25 at 18:31







      Reinier Meester

















      asked Mar 22 at 6:25









      Reinier MeesterReinier Meester

      12




      12






















          0






          active

          oldest

          votes












          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55293990%2fefficient-ways-to-summarize-array-in-r-depending-on-other-data%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55293990%2fefficient-ways-to-summarize-array-in-r-depending-on-other-data%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

          Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

          Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript