Styjun

I am trying to read the urls from the first column in a csv file. In the csv file, there are 6051 urls in total which I want to read. To do so, I tried the following codes:

 urls = []
 with open("C:/Users/hyoungm/Downloads/urls.csv") as csvfile:
 blogurl = csv.reader(csvfile)
 for row in blogurl:
 row = row[0]
 print(row)

len(row)

However, the number of urls that are shown is only 65. I have no idea why the total number of urls appears differently from the csv file.

Can anybody help me with figuring out how to read all urls (6051 in total) from the csv file?

To read all the urls from the csv file, I also tried several different codes that resulted in the same number of urls (i.e., 65 urls) or failure, such as:
1)

 openfile = open("C:/Users/hyoungm/Downloads/urls.csv")
 r = csv.reader(openfile)
 for i in r:
 #the urls are in the first column ... 0 refers to the first column
 blogurls = i[0]
 print (blogurls)
 len(blogurls)

2)

 urls = pd.read_csv("C:/Users/hyoungm/Downloads/urls.csv")
 with closing(requests.get(urls, stream = True)) as r:
 reader = csv.reader(r.iter_lines(), delimiter = ',', quotechar = '""')
 for row in reader:
 print(row)
 len(row)

3)

 with open("C:/Users/hyoungm/Downloads/urls.csv") as csvfile:
 lines = csv.reader(csvfile)
 for i, line in enumerate(lines):
 if i == 0:
 for line in csvfile:
 print(line[1:])
 len(line)

4) and

 blogurls = []
 with open("C:/Users/hyoungm/Downloads/urls.csv") as csvfile:
 r = csv.reader(csvfile)
 for i in r:
 blogurl = i[0]
 r = requests.get(blogurl)
 blogurls.append(blogurl)

 for url in blogurls:
 page = urlopen(url[0]).read()
 soup = BeautifulSoup(page, "html.parser")
 len(blogurls)

I expect the output of 6051 urls as originally collected in the csv file, instead of 65 urls.

After reading all the urls, I am going to scrawl down the textual data from each url. I supposed to get the following textual data using all 6051 urls. Please click the following link for the image:

the codes and the outcomes based on 65 urls so far

The following two approaches work for me:

import requests

r = requests.get('https://raw.githubusercontent.com/GemmyMoon/MultipleUrls/master/urls.csv')
urls = r.text.splitlines()

print(len(urls)) # Returns 6051

and

import csv
import requests
from io import StringIO

r = requests.get('https://raw.githubusercontent.com/GemmyMoon/MultipleUrls/master/urls.csv')
reader = csv.reader(StringIO(r.text))
urls = [line[0] for line in reader]

print(len(urls)) # Returns 6051