Styjun

Are there any symmetric cryptosystems based on computational complexity assumptions?

Can more than one instance of Bend Luck be applied to the same roll by multiple Wild Magic sorcerers?

Windows reverting changes made by Linux to FAT32 partion

Have the writers and actors of GOT responded to its poor reception?

Can I get the output of a command line program with TeX (using e.g. read18)?

Why does the U.S military use mercenaries?

How to pipe results multiple results into a command?

Can ThermodynamicData be used with NSolve?

Why would company (decision makers) wait for someone to retire, rather than lay them off, when their role is no longer needed?

Would a "ring language" be possible?

Physically unpleasant work environment

How do you cope with rejection?

How would fantasy dwarves exist, realistically?

What do you call bracelets you wear around the legs?

Why wear sunglasses in indoor velodromes?

on the truth quest vs in the quest for truth

In Dutch history two people are referred to as "William III"; are there any more cases where this happens?

Why is so much ransomware breakable?

What should I wear to go and sign an employment contract?

How do I balance a campaign consisting of four kobold PCs?

Told to apply for UK visa before other visas, on UK-Spain-etc. visit

Failing students when it might cause them economic ruin

How to laser-level close to a surface

Hotel booking: Why is Agoda much cheaper than booking.com?

Pandas read_csv not reading all rows?

Pandas cannot load data, csv encoding mysteryHow to upgrade all Python packages with pip?How do I list all files of a directory?How to read a file line-by-line into a list?Add one row to pandas DataFrameRenaming columns in pandasDelete column from pandas DataFrame by column namePandas read_csv and UTF-16“Large data” work flows using pandasHow to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandas

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;

0

I have a text file containing 7 millions rows of text ~ and encoded in utf-16.

70357719 new.file

new.file: text/plain; charset=utf-16le

When I use pandas read_csv encoding to utf-16 it only imports a percentage of the rows.

Using the following test code;

import pandas as pd 
data = pd.read_csv('new.file',names=['Text'],sep="n")
print "Plain:",len(data)

data = pd.read_csv('new.file',names=['Text'],encoding="utf-16",sep="n")
print "utf-16",len(data)

Provides the following output;

'Plain:', 215585254
'utf-16', 65446415

I'm using python 2.7, and have already tested for empty rows in the file (of which there are none).

Basically, I'm at a lost for what to try next, I need all rows of this file to be imported.

python pandas

share|improve this question

asked Mar 23 at 17:28

F.DF.D

7411

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Take a look: stackoverflow.com/questions/38728366/… and stackoverflow.com/questions/55316476/…
  
  – rafaelc
  Mar 23 at 17:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Why are you using sep="n"?
  
  – Benitok
  Mar 23 at 17:53
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  RafaelC, the second links goes back to this question. | Benitok, to separate each line = row, I'm aware names= would also do this.
  
  – F.D
  Mar 23 at 17:56
  

  
  
  
  

add a comment | 

0

I have a text file containing 7 millions rows of text ~ and encoded in utf-16.

70357719 new.file

new.file: text/plain; charset=utf-16le

When I use pandas read_csv encoding to utf-16 it only imports a percentage of the rows.

Using the following test code;

import pandas as pd 
data = pd.read_csv('new.file',names=['Text'],sep="n")
print "Plain:",len(data)

data = pd.read_csv('new.file',names=['Text'],encoding="utf-16",sep="n")
print "utf-16",len(data)

Provides the following output;

'Plain:', 215585254
'utf-16', 65446415

I'm using python 2.7, and have already tested for empty rows in the file (of which there are none).

Basically, I'm at a lost for what to try next, I need all rows of this file to be imported.

python pandas

share|improve this question

asked Mar 23 at 17:28

F.DF.D

7411

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Take a look: stackoverflow.com/questions/38728366/… and stackoverflow.com/questions/55316476/…
  
  – rafaelc
  Mar 23 at 17:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Why are you using sep="n"?
  
  – Benitok
  Mar 23 at 17:53
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  RafaelC, the second links goes back to this question. | Benitok, to separate each line = row, I'm aware names= would also do this.
  
  – F.D
  Mar 23 at 17:56
  

  
  
  
  

add a comment | 

I have a text file containing 7 millions rows of text ~ and encoded in utf-16.

70357719 new.file

new.file: text/plain; charset=utf-16le

When I use pandas read_csv encoding to utf-16 it only imports a percentage of the rows.

Using the following test code;

import pandas as pd 
data = pd.read_csv('new.file',names=['Text'],sep="n")
print "Plain:",len(data)

data = pd.read_csv('new.file',names=['Text'],encoding="utf-16",sep="n")
print "utf-16",len(data)

Provides the following output;

'Plain:', 215585254
'utf-16', 65446415

I'm using python 2.7, and have already tested for empty rows in the file (of which there are none).

Basically, I'm at a lost for what to try next, I need all rows of this file to be imported.

python pandas

share|improve this question

asked Mar 23 at 17:28

F.DF.D

7411

70357719 new.file

new.file: text/plain; charset=utf-16le

import pandas as pd 
data = pd.read_csv('new.file',names=['Text'],sep="n")
print "Plain:",len(data)

data = pd.read_csv('new.file',names=['Text'],encoding="utf-16",sep="n")
print "utf-16",len(data)

'Plain:', 215585254
'utf-16', 65446415

share|improve this question

asked Mar 23 at 17:28

F.DF.D

7411

share|improve this question

asked Mar 23 at 17:28

F.DF.D

7411

asked Mar 23 at 17:28

F.DF.D

7411

asked Mar 23 at 17:28

F.DF.D

7411