Pandas Dataframe replace Nan from a row when a column value matchesAdd one row to pandas DataFrameSelecting multiple columns in a pandas dataframeAdding new column to existing DataFrame in Python pandasDelete column from pandas DataFrameHow to drop rows of Pandas DataFrame whose value in a certain column is NaN“Large data” work flows using pandasHow to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandasGet list from pandas DataFrame column headersHow to check if any value is NaN in a Pandas DataFrame
Looking for a big fantasy novel about scholarly monks that sort of worship math?
What quests do you need to stop at before you make an enemy of a faction for each faction?
Why does the seven segment display have decimal point at the right?
I won a car in a poker game
How to create large inductors (1H) for audio use?
What is the source of the fear in the Hallow spell's extra Fear effect?
Why would image resources loaded from different origins triggering HTTP authentication dialogs be harmful?
Why there is no wireless switch?
How to calculate the power level of a Commander deck?
What does it mean to count a group of numbers with their multiplicity?
How do German speakers decide what should be on the left side of the verb?
Why did Boris Johnson call for new elections?
Golfball Dimples on spaceships (and planes)?
How quickly would a wooden treasure chest rot?
Why are UK MPs allowed to not vote (but it counts as a no)?
What is the purpose of the rotating plate in front of the lock?
What is the material of snubber brakes?
Translate English to Pig Latin | PIG_LATIN.PY
If I sell my PS4 game disc and buy a digital version, can I still access my saved game?
Never make public members virtual/abstract - really?
How to measure the statistical "distance" between two frequency distributions?
"syntax error near unexpected token" after editing .bashrc
How do English-speaking kids loudly request something?
Pronunciation of "sincero" and "sinceramente"
Pandas Dataframe replace Nan from a row when a column value matches
Add one row to pandas DataFrameSelecting multiple columns in a pandas dataframeAdding new column to existing DataFrame in Python pandasDelete column from pandas DataFrameHow to drop rows of Pandas DataFrame whose value in a certain column is NaN“Large data” work flows using pandasHow to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandasGet list from pandas DataFrame column headersHow to check if any value is NaN in a Pandas DataFrame
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have dataframe i.e.,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 Nan salem
3 III A Eng 80 gphss Nan
4 III A Mat 45 Nan salem
5 III A Eng 40 gphss Nan
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss Nan
I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
Can anyone help me out in this?
python python-3.x pandas nan
add a comment |
I have dataframe i.e.,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 Nan salem
3 III A Eng 80 gphss Nan
4 III A Mat 45 Nan salem
5 III A Eng 40 gphss Nan
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss Nan
I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
Can anyone help me out in this?
python python-3.x pandas nan
add a comment |
I have dataframe i.e.,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 Nan salem
3 III A Eng 80 gphss Nan
4 III A Mat 45 Nan salem
5 III A Eng 40 gphss Nan
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss Nan
I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
Can anyone help me out in this?
python python-3.x pandas nan
I have dataframe i.e.,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 Nan salem
3 III A Eng 80 gphss Nan
4 III A Mat 45 Nan salem
5 III A Eng 40 gphss Nan
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss Nan
I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
Input Dataframe
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
Can anyone help me out in this?
python python-3.x pandas nan
python python-3.x pandas nan
edited Mar 28 at 5:35
AkshayNevrekar
6,93312 gold badges23 silver badges45 bronze badges
6,93312 gold badges23 silver badges45 bronze badges
asked Mar 28 at 4:42
Mahamutha MMahamutha M
3841 silver badge14 bronze badges
3841 silver badge14 bronze badges
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
Use forward and back filling missing values per groups with lambda function
in columns specified in list with DataFrame.groupby
- is necessary for each combination same values per groups:
cols = ['school','city']
df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
print (df)
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
@MahamuthaM - Not sure if understand, it is solution for creatingDataFrame
? And there is some problem?
– jezrael
Mar 28 at 7:30
3
@MahamuthaM - Can you explain more? No replace? Try usedf = df.replace(['Nan', 'NaN'], np.nan)
before my solution.
– jezrael
Mar 28 at 7:32
add a comment |
Assuming that each pair of class
and section
corresponds to a unique pair of school
and city
, we can use groupby
:
# create a dictionary of class and section with school and city
# here we assume that for each pair and class there's a row with both school and city
# if that's not the case, we can separate the two series
school_city_dict = df[['class', 'section','school','city']].dropna().
groupby(['class', 'section'])[['school','city']].
max().to_dict()
# school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
# 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'
# set index, prepare for map function
df.set_index(['class','section'], inplace=True)
df.loc[:,'school'] = df.index.map(school_city_dict['school'])
df.loc[:,'city'] = df.index.map(school_city_dict['city'])
# reset index to the original
df.reset_index()
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55390298%2fpandas-dataframe-replace-nan-from-a-row-when-a-column-value-matches%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Use forward and back filling missing values per groups with lambda function
in columns specified in list with DataFrame.groupby
- is necessary for each combination same values per groups:
cols = ['school','city']
df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
print (df)
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
@MahamuthaM - Not sure if understand, it is solution for creatingDataFrame
? And there is some problem?
– jezrael
Mar 28 at 7:30
3
@MahamuthaM - Can you explain more? No replace? Try usedf = df.replace(['Nan', 'NaN'], np.nan)
before my solution.
– jezrael
Mar 28 at 7:32
add a comment |
Use forward and back filling missing values per groups with lambda function
in columns specified in list with DataFrame.groupby
- is necessary for each combination same values per groups:
cols = ['school','city']
df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
print (df)
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
@MahamuthaM - Not sure if understand, it is solution for creatingDataFrame
? And there is some problem?
– jezrael
Mar 28 at 7:30
3
@MahamuthaM - Can you explain more? No replace? Try usedf = df.replace(['Nan', 'NaN'], np.nan)
before my solution.
– jezrael
Mar 28 at 7:32
add a comment |
Use forward and back filling missing values per groups with lambda function
in columns specified in list with DataFrame.groupby
- is necessary for each combination same values per groups:
cols = ['school','city']
df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
print (df)
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
Use forward and back filling missing values per groups with lambda function
in columns specified in list with DataFrame.groupby
- is necessary for each combination same values per groups:
cols = ['school','city']
df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
print (df)
class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem
answered Mar 28 at 6:06
jezraeljezrael
407k32 gold badges423 silver badges486 bronze badges
407k32 gold badges423 silver badges486 bronze badges
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
@MahamuthaM - Not sure if understand, it is solution for creatingDataFrame
? And there is some problem?
– jezrael
Mar 28 at 7:30
3
@MahamuthaM - Can you explain more? No replace? Try usedf = df.replace(['Nan', 'NaN'], np.nan)
before my solution.
– jezrael
Mar 28 at 7:32
add a comment |
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
@MahamuthaM - Not sure if understand, it is solution for creatingDataFrame
? And there is some problem?
– jezrael
Mar 28 at 7:30
3
@MahamuthaM - Can you explain more? No replace? Try usedf = df.replace(['Nan', 'NaN'], np.nan)
before my solution.
– jezrael
Mar 28 at 7:32
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
I have tried your suggestion, Whereas I am unable to get the result
– Mahamutha M
Mar 28 at 7:29
@MahamuthaM - Not sure if understand, it is solution for creating
DataFrame
? And there is some problem?– jezrael
Mar 28 at 7:30
@MahamuthaM - Not sure if understand, it is solution for creating
DataFrame
? And there is some problem?– jezrael
Mar 28 at 7:30
3
3
@MahamuthaM - Can you explain more? No replace? Try use
df = df.replace(['Nan', 'NaN'], np.nan)
before my solution.– jezrael
Mar 28 at 7:32
@MahamuthaM - Can you explain more? No replace? Try use
df = df.replace(['Nan', 'NaN'], np.nan)
before my solution.– jezrael
Mar 28 at 7:32
add a comment |
Assuming that each pair of class
and section
corresponds to a unique pair of school
and city
, we can use groupby
:
# create a dictionary of class and section with school and city
# here we assume that for each pair and class there's a row with both school and city
# if that's not the case, we can separate the two series
school_city_dict = df[['class', 'section','school','city']].dropna().
groupby(['class', 'section'])[['school','city']].
max().to_dict()
# school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
# 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'
# set index, prepare for map function
df.set_index(['class','section'], inplace=True)
df.loc[:,'school'] = df.index.map(school_city_dict['school'])
df.loc[:,'city'] = df.index.map(school_city_dict['city'])
# reset index to the original
df.reset_index()
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
add a comment |
Assuming that each pair of class
and section
corresponds to a unique pair of school
and city
, we can use groupby
:
# create a dictionary of class and section with school and city
# here we assume that for each pair and class there's a row with both school and city
# if that's not the case, we can separate the two series
school_city_dict = df[['class', 'section','school','city']].dropna().
groupby(['class', 'section'])[['school','city']].
max().to_dict()
# school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
# 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'
# set index, prepare for map function
df.set_index(['class','section'], inplace=True)
df.loc[:,'school'] = df.index.map(school_city_dict['school'])
df.loc[:,'city'] = df.index.map(school_city_dict['city'])
# reset index to the original
df.reset_index()
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
add a comment |
Assuming that each pair of class
and section
corresponds to a unique pair of school
and city
, we can use groupby
:
# create a dictionary of class and section with school and city
# here we assume that for each pair and class there's a row with both school and city
# if that's not the case, we can separate the two series
school_city_dict = df[['class', 'section','school','city']].dropna().
groupby(['class', 'section'])[['school','city']].
max().to_dict()
# school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
# 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'
# set index, prepare for map function
df.set_index(['class','section'], inplace=True)
df.loc[:,'school'] = df.index.map(school_city_dict['school'])
df.loc[:,'city'] = df.index.map(school_city_dict['city'])
# reset index to the original
df.reset_index()
Assuming that each pair of class
and section
corresponds to a unique pair of school
and city
, we can use groupby
:
# create a dictionary of class and section with school and city
# here we assume that for each pair and class there's a row with both school and city
# if that's not the case, we can separate the two series
school_city_dict = df[['class', 'section','school','city']].dropna().
groupby(['class', 'section'])[['school','city']].
max().to_dict()
# school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
# 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'
# set index, prepare for map function
df.set_index(['class','section'], inplace=True)
df.loc[:,'school'] = df.index.map(school_city_dict['school'])
df.loc[:,'city'] = df.index.map(school_city_dict['city'])
# reset index to the original
df.reset_index()
edited Mar 28 at 10:42
answered Mar 28 at 6:01
Quang HoangQuang Hoang
16.2k2 gold badges14 silver badges26 bronze badges
16.2k2 gold badges14 silver badges26 bronze badges
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
add a comment |
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
AttributeError: 'list' object has no attribute 'dropna'
– Mahamutha M
Mar 28 at 7:41
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55390298%2fpandas-dataframe-replace-nan-from-a-row-when-a-column-value-matches%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown