Changing Dates in R from webscraper but not able to convertR: Reshaping irregular time series data without explicit unique datesDate format for plotting x axis ticks of time series dataWhy is a date being reformated to numeric value when passed into data frame?Search for subheader text and subset a range of time series dataDeal with dates in RHow to convert string to POSIX in R when the string has abbreviated weekday and monthdayR - Sort all days by monthSub-setting by partial date rReorganizing data by date and count in rMatching a file name with a dynamic variable in r
Should we run PBKDF2 for every plaintext to be protected or should we run PBKDF2 only once?
Fishing from underwater domes
How does Query decide the order in which the functions are applied?
The 7-numbers crossword
Cheap oscilloscope showing 16 MHz square wave
Datasets of Large Molecules
Understanding GFCI configuration in basement
Ways you can end up paying interest on a credit card if you pay the full amount back in due time
Could these polynomials be identified?
What happens if you just start drawing from the Deck of Many Things without declaring any number of cards?
Function of the separated, individual solar cells on Telstar 1 and 2? Why were they "special"?
How secure are public hashed passwords (with a salt)?
How to correctly set logical high level on PS/2 port?
Is Chuck the Evil Sandwich Making Guy's head actually a sandwich?
How are the cards determined in an incomplete deck of many things?
Given a specific computer system, is it possible to estimate the actual precise run time of a piece of Assembly code
What caused the end of cybernetic implants?
Squares inside a square
Can some one give a step-by-step computation for an empirical distribution with this concrete example
An alternative to "two column" geometry proofs
Do universities maintain secret textbooks?
How to save money by shopping at a variety of grocery stores?
How to run a command 1 out of N times in Bash
Sum and average calculator
Changing Dates in R from webscraper but not able to convert
R: Reshaping irregular time series data without explicit unique datesDate format for plotting x axis ticks of time series dataWhy is a date being reformated to numeric value when passed into data frame?Search for subheader text and subset a range of time series dataDeal with dates in RHow to convert string to POSIX in R when the string has abbreviated weekday and monthdayR - Sort all days by monthSub-setting by partial date rReorganizing data by date and count in rMatching a file name with a dynamic variable in r
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I am trying to complete a problem that pulls from two data sets that need to be combined into one data set. To get to this point, I need to rbind both data sets by the year-month information. Unfortunately, the first data set needs to be tallied by year-month info, and I can't seem to figure out how to change the date so I can have month-year info rather than month-day-year info.
This is data on avalanches and I need to write code totally the number of avalanches each moth for the Snow Season, defined as Dec-Mar. How do I do that?
I keep trying to convert the format of the date to month-year but after I change it with
as.Date(avalancheslc$Date, format="%y-%m")
all the values for Date turn to NA's....help!
# write the webscraper
library(XML)
library(RCurl)
avalanche<-data.frame()
avalanche.url<-"https://utahavalanchecenter.org/observations?page="
all.pages<-0:202
for(page in all.pages)
this.url<-paste(avalanche.url, page, sep=" ")
this.webpage<-htmlParse(getURL(this.url))
thispage.avalanche<-readHTMLTable(this.webpage, which=1, header=T)
avalanche<-rbind(avalanche,thispage.avalanche)
# subset the data to the Salt Lake Region
avalancheslc<-subset(avalanche, Region=="Salt Lake")
str(avalancheslc)
avalancheslc$monthyear<-format(as.Date(avalancheslc$Date),"%Y-%m")
# How can I tally the number of avalanches?
The final output of my dataset should be something like:
date avalanches
2000-1 18
2000-2 4
2000-3 10
2000-12 12
2001-1 52
r
add a comment |
I am trying to complete a problem that pulls from two data sets that need to be combined into one data set. To get to this point, I need to rbind both data sets by the year-month information. Unfortunately, the first data set needs to be tallied by year-month info, and I can't seem to figure out how to change the date so I can have month-year info rather than month-day-year info.
This is data on avalanches and I need to write code totally the number of avalanches each moth for the Snow Season, defined as Dec-Mar. How do I do that?
I keep trying to convert the format of the date to month-year but after I change it with
as.Date(avalancheslc$Date, format="%y-%m")
all the values for Date turn to NA's....help!
# write the webscraper
library(XML)
library(RCurl)
avalanche<-data.frame()
avalanche.url<-"https://utahavalanchecenter.org/observations?page="
all.pages<-0:202
for(page in all.pages)
this.url<-paste(avalanche.url, page, sep=" ")
this.webpage<-htmlParse(getURL(this.url))
thispage.avalanche<-readHTMLTable(this.webpage, which=1, header=T)
avalanche<-rbind(avalanche,thispage.avalanche)
# subset the data to the Salt Lake Region
avalancheslc<-subset(avalanche, Region=="Salt Lake")
str(avalancheslc)
avalancheslc$monthyear<-format(as.Date(avalancheslc$Date),"%Y-%m")
# How can I tally the number of avalanches?
The final output of my dataset should be something like:
date avalanches
2000-1 18
2000-2 4
2000-3 10
2000-12 12
2001-1 52
r
add a comment |
I am trying to complete a problem that pulls from two data sets that need to be combined into one data set. To get to this point, I need to rbind both data sets by the year-month information. Unfortunately, the first data set needs to be tallied by year-month info, and I can't seem to figure out how to change the date so I can have month-year info rather than month-day-year info.
This is data on avalanches and I need to write code totally the number of avalanches each moth for the Snow Season, defined as Dec-Mar. How do I do that?
I keep trying to convert the format of the date to month-year but after I change it with
as.Date(avalancheslc$Date, format="%y-%m")
all the values for Date turn to NA's....help!
# write the webscraper
library(XML)
library(RCurl)
avalanche<-data.frame()
avalanche.url<-"https://utahavalanchecenter.org/observations?page="
all.pages<-0:202
for(page in all.pages)
this.url<-paste(avalanche.url, page, sep=" ")
this.webpage<-htmlParse(getURL(this.url))
thispage.avalanche<-readHTMLTable(this.webpage, which=1, header=T)
avalanche<-rbind(avalanche,thispage.avalanche)
# subset the data to the Salt Lake Region
avalancheslc<-subset(avalanche, Region=="Salt Lake")
str(avalancheslc)
avalancheslc$monthyear<-format(as.Date(avalancheslc$Date),"%Y-%m")
# How can I tally the number of avalanches?
The final output of my dataset should be something like:
date avalanches
2000-1 18
2000-2 4
2000-3 10
2000-12 12
2001-1 52
r
I am trying to complete a problem that pulls from two data sets that need to be combined into one data set. To get to this point, I need to rbind both data sets by the year-month information. Unfortunately, the first data set needs to be tallied by year-month info, and I can't seem to figure out how to change the date so I can have month-year info rather than month-day-year info.
This is data on avalanches and I need to write code totally the number of avalanches each moth for the Snow Season, defined as Dec-Mar. How do I do that?
I keep trying to convert the format of the date to month-year but after I change it with
as.Date(avalancheslc$Date, format="%y-%m")
all the values for Date turn to NA's....help!
# write the webscraper
library(XML)
library(RCurl)
avalanche<-data.frame()
avalanche.url<-"https://utahavalanchecenter.org/observations?page="
all.pages<-0:202
for(page in all.pages)
this.url<-paste(avalanche.url, page, sep=" ")
this.webpage<-htmlParse(getURL(this.url))
thispage.avalanche<-readHTMLTable(this.webpage, which=1, header=T)
avalanche<-rbind(avalanche,thispage.avalanche)
# subset the data to the Salt Lake Region
avalancheslc<-subset(avalanche, Region=="Salt Lake")
str(avalancheslc)
avalancheslc$monthyear<-format(as.Date(avalancheslc$Date),"%Y-%m")
# How can I tally the number of avalanches?
The final output of my dataset should be something like:
date avalanches
2000-1 18
2000-2 4
2000-3 10
2000-12 12
2001-1 52
r
r
edited Mar 28 at 0:39
neilfws
20.8k5 gold badges38 silver badges49 bronze badges
20.8k5 gold badges38 silver badges49 bronze badges
asked Mar 28 at 0:38
crichcrich
325 bronze badges
325 bronze badges
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
This should work (I tried it on only 1 page, not all 203). Note the use of the option stringsAsFactors = F in the readHTMLTable function, and the need to add names because 1 column does not automatically get one.
library(XML)
library(RCurl)
library(dplyr)
avalanche <- data.frame()
avalanche.url <- "https://utahavalanchecenter.org/observations?page="
all.pages <- 0:202
for(page in all.pages)
this.url <- paste(avalanche.url, page, sep=" ")
this.webpage <- htmlParse(getURL(this.url))
thispage.avalanche <- readHTMLTable(this.webpage, which = 1, header = T,
stringsAsFactors = F)
names(thispage.avalanche) <- c('Date','Region','Location','Observer')
avalanche <- rbind(avalanche,thispage.avalanche)
avalancheslc <- subset(avalanche, Region == "Salt Lake")
str(avalancheslc)
avalancheslc <- mutate(avalancheslc, Date = as.Date(Date, format = "%m/%d/%Y"),
monthyear = paste(year(Date), month(Date), sep = "-"))
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
By "sum across each of the monthyears into a dataframe" do you just mean this:as_tibble(with(avalancheslc, table(monthyear)))?
– ericOss
Mar 28 at 18:01
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
I didn't remember which package theyearfunction was from so I Googled "R function year," which revealed that it's from thelubridatepackage, which I always load. I highly recommend it for dealing with dates and times.
– ericOss
Mar 28 at 22:17
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55388529%2fchanging-dates-in-r-from-webscraper-but-not-able-to-convert%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
This should work (I tried it on only 1 page, not all 203). Note the use of the option stringsAsFactors = F in the readHTMLTable function, and the need to add names because 1 column does not automatically get one.
library(XML)
library(RCurl)
library(dplyr)
avalanche <- data.frame()
avalanche.url <- "https://utahavalanchecenter.org/observations?page="
all.pages <- 0:202
for(page in all.pages)
this.url <- paste(avalanche.url, page, sep=" ")
this.webpage <- htmlParse(getURL(this.url))
thispage.avalanche <- readHTMLTable(this.webpage, which = 1, header = T,
stringsAsFactors = F)
names(thispage.avalanche) <- c('Date','Region','Location','Observer')
avalanche <- rbind(avalanche,thispage.avalanche)
avalancheslc <- subset(avalanche, Region == "Salt Lake")
str(avalancheslc)
avalancheslc <- mutate(avalancheslc, Date = as.Date(Date, format = "%m/%d/%Y"),
monthyear = paste(year(Date), month(Date), sep = "-"))
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
By "sum across each of the monthyears into a dataframe" do you just mean this:as_tibble(with(avalancheslc, table(monthyear)))?
– ericOss
Mar 28 at 18:01
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
I didn't remember which package theyearfunction was from so I Googled "R function year," which revealed that it's from thelubridatepackage, which I always load. I highly recommend it for dealing with dates and times.
– ericOss
Mar 28 at 22:17
add a comment |
This should work (I tried it on only 1 page, not all 203). Note the use of the option stringsAsFactors = F in the readHTMLTable function, and the need to add names because 1 column does not automatically get one.
library(XML)
library(RCurl)
library(dplyr)
avalanche <- data.frame()
avalanche.url <- "https://utahavalanchecenter.org/observations?page="
all.pages <- 0:202
for(page in all.pages)
this.url <- paste(avalanche.url, page, sep=" ")
this.webpage <- htmlParse(getURL(this.url))
thispage.avalanche <- readHTMLTable(this.webpage, which = 1, header = T,
stringsAsFactors = F)
names(thispage.avalanche) <- c('Date','Region','Location','Observer')
avalanche <- rbind(avalanche,thispage.avalanche)
avalancheslc <- subset(avalanche, Region == "Salt Lake")
str(avalancheslc)
avalancheslc <- mutate(avalancheslc, Date = as.Date(Date, format = "%m/%d/%Y"),
monthyear = paste(year(Date), month(Date), sep = "-"))
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
By "sum across each of the monthyears into a dataframe" do you just mean this:as_tibble(with(avalancheslc, table(monthyear)))?
– ericOss
Mar 28 at 18:01
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
I didn't remember which package theyearfunction was from so I Googled "R function year," which revealed that it's from thelubridatepackage, which I always load. I highly recommend it for dealing with dates and times.
– ericOss
Mar 28 at 22:17
add a comment |
This should work (I tried it on only 1 page, not all 203). Note the use of the option stringsAsFactors = F in the readHTMLTable function, and the need to add names because 1 column does not automatically get one.
library(XML)
library(RCurl)
library(dplyr)
avalanche <- data.frame()
avalanche.url <- "https://utahavalanchecenter.org/observations?page="
all.pages <- 0:202
for(page in all.pages)
this.url <- paste(avalanche.url, page, sep=" ")
this.webpage <- htmlParse(getURL(this.url))
thispage.avalanche <- readHTMLTable(this.webpage, which = 1, header = T,
stringsAsFactors = F)
names(thispage.avalanche) <- c('Date','Region','Location','Observer')
avalanche <- rbind(avalanche,thispage.avalanche)
avalancheslc <- subset(avalanche, Region == "Salt Lake")
str(avalancheslc)
avalancheslc <- mutate(avalancheslc, Date = as.Date(Date, format = "%m/%d/%Y"),
monthyear = paste(year(Date), month(Date), sep = "-"))
This should work (I tried it on only 1 page, not all 203). Note the use of the option stringsAsFactors = F in the readHTMLTable function, and the need to add names because 1 column does not automatically get one.
library(XML)
library(RCurl)
library(dplyr)
avalanche <- data.frame()
avalanche.url <- "https://utahavalanchecenter.org/observations?page="
all.pages <- 0:202
for(page in all.pages)
this.url <- paste(avalanche.url, page, sep=" ")
this.webpage <- htmlParse(getURL(this.url))
thispage.avalanche <- readHTMLTable(this.webpage, which = 1, header = T,
stringsAsFactors = F)
names(thispage.avalanche) <- c('Date','Region','Location','Observer')
avalanche <- rbind(avalanche,thispage.avalanche)
avalancheslc <- subset(avalanche, Region == "Salt Lake")
str(avalancheslc)
avalancheslc <- mutate(avalancheslc, Date = as.Date(Date, format = "%m/%d/%Y"),
monthyear = paste(year(Date), month(Date), sep = "-"))
answered Mar 28 at 1:16
ericOssericOss
1515 bronze badges
1515 bronze badges
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
By "sum across each of the monthyears into a dataframe" do you just mean this:as_tibble(with(avalancheslc, table(monthyear)))?
– ericOss
Mar 28 at 18:01
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
I didn't remember which package theyearfunction was from so I Googled "R function year," which revealed that it's from thelubridatepackage, which I always load. I highly recommend it for dealing with dates and times.
– ericOss
Mar 28 at 22:17
add a comment |
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
By "sum across each of the monthyears into a dataframe" do you just mean this:as_tibble(with(avalancheslc, table(monthyear)))?
– ericOss
Mar 28 at 18:01
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
I didn't remember which package theyearfunction was from so I Googled "R function year," which revealed that it's from thelubridatepackage, which I always load. I highly recommend it for dealing with dates and times.
– ericOss
Mar 28 at 22:17
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
that worked! Thank you! Could you help me figure out how to sum across each of the monthyears into a dataframe?
– crich
Mar 28 at 1:39
By "sum across each of the monthyears into a dataframe" do you just mean this:
as_tibble(with(avalancheslc, table(monthyear)))?– ericOss
Mar 28 at 18:01
By "sum across each of the monthyears into a dataframe" do you just mean this:
as_tibble(with(avalancheslc, table(monthyear)))?– ericOss
Mar 28 at 18:01
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
Now when I run the code, I come up with a new error saing "Error in mutat_impl(.data,dots): Evaluation error: could not find function "year".
– crich
Mar 28 at 20:13
I didn't remember which package the
year function was from so I Googled "R function year," which revealed that it's from the lubridate package, which I always load. I highly recommend it for dealing with dates and times.– ericOss
Mar 28 at 22:17
I didn't remember which package the
year function was from so I Googled "R function year," which revealed that it's from the lubridate package, which I always load. I highly recommend it for dealing with dates and times.– ericOss
Mar 28 at 22:17
add a comment |
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55388529%2fchanging-dates-in-r-from-webscraper-but-not-able-to-convert%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown