Pandas Dataframe replace Nan from a row when a column value matchesAdd one row to pandas DataFrameSelecting multiple columns in a pandas dataframeAdding new column to existing DataFrame in Python pandasDelete column from pandas DataFrameHow to drop rows of Pandas DataFrame whose value in a certain column is NaN“Large data” work flows using pandasHow to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandasGet list from pandas DataFrame column headersHow to check if any value is NaN in a Pandas DataFrame

Looking for a big fantasy novel about scholarly monks that sort of worship math?

What quests do you need to stop at before you make an enemy of a faction for each faction?

Why does the seven segment display have decimal point at the right?

I won a car in a poker game

How to create large inductors (1H) for audio use?

What is the source of the fear in the Hallow spell's extra Fear effect?

Why would image resources loaded from different origins triggering HTTP authentication dialogs be harmful?

Why there is no wireless switch?

How to calculate the power level of a Commander deck?

What does it mean to count a group of numbers with their multiplicity?

How do German speakers decide what should be on the left side of the verb?

Why did Boris Johnson call for new elections?

Golfball Dimples on spaceships (and planes)?

How quickly would a wooden treasure chest rot?

Why are UK MPs allowed to not vote (but it counts as a no)?

What is the purpose of the rotating plate in front of the lock?

What is the material of snubber brakes?

Translate English to Pig Latin | PIG_LATIN.PY

If I sell my PS4 game disc and buy a digital version, can I still access my saved game?

Never make public members virtual/abstract - really?

How to measure the statistical "distance" between two frequency distributions?

"syntax error near unexpected token" after editing .bashrc

How do English-speaking kids loudly request something?

Pronunciation of "sincero" and "sinceramente"



Pandas Dataframe replace Nan from a row when a column value matches


Add one row to pandas DataFrameSelecting multiple columns in a pandas dataframeAdding new column to existing DataFrame in Python pandasDelete column from pandas DataFrameHow to drop rows of Pandas DataFrame whose value in a certain column is NaN“Large data” work flows using pandasHow to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandasGet list from pandas DataFrame column headersHow to check if any value is NaN in a Pandas DataFrame






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








6















I have dataframe i.e.,



Input Dataframe

class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 Nan salem
3 III A Eng 80 gphss Nan
4 III A Mat 45 Nan salem
5 III A Eng 40 gphss Nan
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss Nan


I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
Input Dataframe



 class section sub marks school city
0 I A Eng 80 jghss salem
1 I A Mat 90 jghss salem
2 I A Eng 50 jghss salem
3 III A Eng 80 gphss salem
4 III A Mat 45 gphss salem
5 III A Eng 40 gphss salem
6 III A Eng 20 gphss salem
7 III A Mat 55 gphss salem


Can anyone help me out in this?










share|improve this question
































    6















    I have dataframe i.e.,



    Input Dataframe

    class section sub marks school city
    0 I A Eng 80 jghss salem
    1 I A Mat 90 jghss salem
    2 I A Eng 50 Nan salem
    3 III A Eng 80 gphss Nan
    4 III A Mat 45 Nan salem
    5 III A Eng 40 gphss Nan
    6 III A Eng 20 gphss salem
    7 III A Mat 55 gphss Nan


    I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
    Input Dataframe



     class section sub marks school city
    0 I A Eng 80 jghss salem
    1 I A Mat 90 jghss salem
    2 I A Eng 50 jghss salem
    3 III A Eng 80 gphss salem
    4 III A Mat 45 gphss salem
    5 III A Eng 40 gphss salem
    6 III A Eng 20 gphss salem
    7 III A Mat 55 gphss salem


    Can anyone help me out in this?










    share|improve this question




























      6












      6








      6


      2






      I have dataframe i.e.,



      Input Dataframe

      class section sub marks school city
      0 I A Eng 80 jghss salem
      1 I A Mat 90 jghss salem
      2 I A Eng 50 Nan salem
      3 III A Eng 80 gphss Nan
      4 III A Mat 45 Nan salem
      5 III A Eng 40 gphss Nan
      6 III A Eng 20 gphss salem
      7 III A Mat 55 gphss Nan


      I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
      Input Dataframe



       class section sub marks school city
      0 I A Eng 80 jghss salem
      1 I A Mat 90 jghss salem
      2 I A Eng 50 jghss salem
      3 III A Eng 80 gphss salem
      4 III A Mat 45 gphss salem
      5 III A Eng 40 gphss salem
      6 III A Eng 20 gphss salem
      7 III A Mat 55 gphss salem


      Can anyone help me out in this?










      share|improve this question
















      I have dataframe i.e.,



      Input Dataframe

      class section sub marks school city
      0 I A Eng 80 jghss salem
      1 I A Mat 90 jghss salem
      2 I A Eng 50 Nan salem
      3 III A Eng 80 gphss Nan
      4 III A Mat 45 Nan salem
      5 III A Eng 40 gphss Nan
      6 III A Eng 20 gphss salem
      7 III A Mat 55 gphss Nan


      I need to replace the "Nan" in "school" and "city" when a value in "class" and "section" column matches. The resultant outcome suppose to be,
      Input Dataframe



       class section sub marks school city
      0 I A Eng 80 jghss salem
      1 I A Mat 90 jghss salem
      2 I A Eng 50 jghss salem
      3 III A Eng 80 gphss salem
      4 III A Mat 45 gphss salem
      5 III A Eng 40 gphss salem
      6 III A Eng 20 gphss salem
      7 III A Mat 55 gphss salem


      Can anyone help me out in this?







      python python-3.x pandas nan






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Mar 28 at 5:35









      AkshayNevrekar

      6,93312 gold badges23 silver badges45 bronze badges




      6,93312 gold badges23 silver badges45 bronze badges










      asked Mar 28 at 4:42









      Mahamutha MMahamutha M

      3841 silver badge14 bronze badges




      3841 silver badge14 bronze badges

























          2 Answers
          2






          active

          oldest

          votes


















          7
















          Use forward and back filling missing values per groups with lambda function in columns specified in list with DataFrame.groupby - is necessary for each combination same values per groups:



          cols = ['school','city']
          df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
          print (df)
          class section sub marks school city
          0 I A Eng 80 jghss salem
          1 I A Mat 90 jghss salem
          2 I A Eng 50 jghss salem
          3 III A Eng 80 gphss salem
          4 III A Mat 45 gphss salem
          5 III A Eng 40 gphss salem
          6 III A Eng 20 gphss salem
          7 III A Mat 55 gphss salem





          share|improve this answer

























          • I have tried your suggestion, Whereas I am unable to get the result

            – Mahamutha M
            Mar 28 at 7:29












          • @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

            – jezrael
            Mar 28 at 7:30






          • 3





            @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

            – jezrael
            Mar 28 at 7:32



















          1
















          Assuming that each pair of class and section corresponds to a unique pair of school and city, we can use groupby:



          # create a dictionary of class and section with school and city
          # here we assume that for each pair and class there's a row with both school and city
          # if that's not the case, we can separate the two series
          school_city_dict = df[['class', 'section','school','city']].dropna().
          groupby(['class', 'section'])[['school','city']].
          max().to_dict()
          # school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
          # 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'

          # set index, prepare for map function
          df.set_index(['class','section'], inplace=True)

          df.loc[:,'school'] = df.index.map(school_city_dict['school'])
          df.loc[:,'city'] = df.index.map(school_city_dict['city'])

          # reset index to the original
          df.reset_index()





          share|improve this answer



























          • AttributeError: 'list' object has no attribute 'dropna'

            – Mahamutha M
            Mar 28 at 7:41













          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );














          draft saved

          draft discarded
















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55390298%2fpandas-dataframe-replace-nan-from-a-row-when-a-column-value-matches%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          2 Answers
          2






          active

          oldest

          votes








          2 Answers
          2






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          7
















          Use forward and back filling missing values per groups with lambda function in columns specified in list with DataFrame.groupby - is necessary for each combination same values per groups:



          cols = ['school','city']
          df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
          print (df)
          class section sub marks school city
          0 I A Eng 80 jghss salem
          1 I A Mat 90 jghss salem
          2 I A Eng 50 jghss salem
          3 III A Eng 80 gphss salem
          4 III A Mat 45 gphss salem
          5 III A Eng 40 gphss salem
          6 III A Eng 20 gphss salem
          7 III A Mat 55 gphss salem





          share|improve this answer

























          • I have tried your suggestion, Whereas I am unable to get the result

            – Mahamutha M
            Mar 28 at 7:29












          • @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

            – jezrael
            Mar 28 at 7:30






          • 3





            @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

            – jezrael
            Mar 28 at 7:32
















          7
















          Use forward and back filling missing values per groups with lambda function in columns specified in list with DataFrame.groupby - is necessary for each combination same values per groups:



          cols = ['school','city']
          df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
          print (df)
          class section sub marks school city
          0 I A Eng 80 jghss salem
          1 I A Mat 90 jghss salem
          2 I A Eng 50 jghss salem
          3 III A Eng 80 gphss salem
          4 III A Mat 45 gphss salem
          5 III A Eng 40 gphss salem
          6 III A Eng 20 gphss salem
          7 III A Mat 55 gphss salem





          share|improve this answer

























          • I have tried your suggestion, Whereas I am unable to get the result

            – Mahamutha M
            Mar 28 at 7:29












          • @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

            – jezrael
            Mar 28 at 7:30






          • 3





            @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

            – jezrael
            Mar 28 at 7:32














          7














          7










          7









          Use forward and back filling missing values per groups with lambda function in columns specified in list with DataFrame.groupby - is necessary for each combination same values per groups:



          cols = ['school','city']
          df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
          print (df)
          class section sub marks school city
          0 I A Eng 80 jghss salem
          1 I A Mat 90 jghss salem
          2 I A Eng 50 jghss salem
          3 III A Eng 80 gphss salem
          4 III A Mat 45 gphss salem
          5 III A Eng 40 gphss salem
          6 III A Eng 20 gphss salem
          7 III A Mat 55 gphss salem





          share|improve this answer













          Use forward and back filling missing values per groups with lambda function in columns specified in list with DataFrame.groupby - is necessary for each combination same values per groups:



          cols = ['school','city']
          df[cols] = df.groupby(['class','section'])[cols].apply(lambda x: x.ffill().bfill())
          print (df)
          class section sub marks school city
          0 I A Eng 80 jghss salem
          1 I A Mat 90 jghss salem
          2 I A Eng 50 jghss salem
          3 III A Eng 80 gphss salem
          4 III A Mat 45 gphss salem
          5 III A Eng 40 gphss salem
          6 III A Eng 20 gphss salem
          7 III A Mat 55 gphss salem






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Mar 28 at 6:06









          jezraeljezrael

          407k32 gold badges423 silver badges486 bronze badges




          407k32 gold badges423 silver badges486 bronze badges















          • I have tried your suggestion, Whereas I am unable to get the result

            – Mahamutha M
            Mar 28 at 7:29












          • @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

            – jezrael
            Mar 28 at 7:30






          • 3





            @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

            – jezrael
            Mar 28 at 7:32


















          • I have tried your suggestion, Whereas I am unable to get the result

            – Mahamutha M
            Mar 28 at 7:29












          • @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

            – jezrael
            Mar 28 at 7:30






          • 3





            @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

            – jezrael
            Mar 28 at 7:32

















          I have tried your suggestion, Whereas I am unable to get the result

          – Mahamutha M
          Mar 28 at 7:29






          I have tried your suggestion, Whereas I am unable to get the result

          – Mahamutha M
          Mar 28 at 7:29














          @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

          – jezrael
          Mar 28 at 7:30





          @MahamuthaM - Not sure if understand, it is solution for creating DataFrame? And there is some problem?

          – jezrael
          Mar 28 at 7:30




          3




          3





          @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

          – jezrael
          Mar 28 at 7:32






          @MahamuthaM - Can you explain more? No replace? Try use df = df.replace(['Nan', 'NaN'], np.nan) before my solution.

          – jezrael
          Mar 28 at 7:32














          1
















          Assuming that each pair of class and section corresponds to a unique pair of school and city, we can use groupby:



          # create a dictionary of class and section with school and city
          # here we assume that for each pair and class there's a row with both school and city
          # if that's not the case, we can separate the two series
          school_city_dict = df[['class', 'section','school','city']].dropna().
          groupby(['class', 'section'])[['school','city']].
          max().to_dict()
          # school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
          # 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'

          # set index, prepare for map function
          df.set_index(['class','section'], inplace=True)

          df.loc[:,'school'] = df.index.map(school_city_dict['school'])
          df.loc[:,'city'] = df.index.map(school_city_dict['city'])

          # reset index to the original
          df.reset_index()





          share|improve this answer



























          • AttributeError: 'list' object has no attribute 'dropna'

            – Mahamutha M
            Mar 28 at 7:41















          1
















          Assuming that each pair of class and section corresponds to a unique pair of school and city, we can use groupby:



          # create a dictionary of class and section with school and city
          # here we assume that for each pair and class there's a row with both school and city
          # if that's not the case, we can separate the two series
          school_city_dict = df[['class', 'section','school','city']].dropna().
          groupby(['class', 'section'])[['school','city']].
          max().to_dict()
          # school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
          # 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'

          # set index, prepare for map function
          df.set_index(['class','section'], inplace=True)

          df.loc[:,'school'] = df.index.map(school_city_dict['school'])
          df.loc[:,'city'] = df.index.map(school_city_dict['city'])

          # reset index to the original
          df.reset_index()





          share|improve this answer



























          • AttributeError: 'list' object has no attribute 'dropna'

            – Mahamutha M
            Mar 28 at 7:41













          1














          1










          1









          Assuming that each pair of class and section corresponds to a unique pair of school and city, we can use groupby:



          # create a dictionary of class and section with school and city
          # here we assume that for each pair and class there's a row with both school and city
          # if that's not the case, we can separate the two series
          school_city_dict = df[['class', 'section','school','city']].dropna().
          groupby(['class', 'section'])[['school','city']].
          max().to_dict()
          # school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
          # 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'

          # set index, prepare for map function
          df.set_index(['class','section'], inplace=True)

          df.loc[:,'school'] = df.index.map(school_city_dict['school'])
          df.loc[:,'city'] = df.index.map(school_city_dict['city'])

          # reset index to the original
          df.reset_index()





          share|improve this answer















          Assuming that each pair of class and section corresponds to a unique pair of school and city, we can use groupby:



          # create a dictionary of class and section with school and city
          # here we assume that for each pair and class there's a row with both school and city
          # if that's not the case, we can separate the two series
          school_city_dict = df[['class', 'section','school','city']].dropna().
          groupby(['class', 'section'])[['school','city']].
          max().to_dict()
          # school_city_dict = 'school': ('I', 'A'): 'jghss', ('III', 'A'): 'gphss',
          # 'city': ('I', 'A'): 'salem', ('III', 'A'): 'salem'

          # set index, prepare for map function
          df.set_index(['class','section'], inplace=True)

          df.loc[:,'school'] = df.index.map(school_city_dict['school'])
          df.loc[:,'city'] = df.index.map(school_city_dict['city'])

          # reset index to the original
          df.reset_index()






          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Mar 28 at 10:42

























          answered Mar 28 at 6:01









          Quang HoangQuang Hoang

          16.2k2 gold badges14 silver badges26 bronze badges




          16.2k2 gold badges14 silver badges26 bronze badges















          • AttributeError: 'list' object has no attribute 'dropna'

            – Mahamutha M
            Mar 28 at 7:41

















          • AttributeError: 'list' object has no attribute 'dropna'

            – Mahamutha M
            Mar 28 at 7:41
















          AttributeError: 'list' object has no attribute 'dropna'

          – Mahamutha M
          Mar 28 at 7:41





          AttributeError: 'list' object has no attribute 'dropna'

          – Mahamutha M
          Mar 28 at 7:41


















          draft saved

          draft discarded















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55390298%2fpandas-dataframe-replace-nan-from-a-row-when-a-column-value-matches%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

          Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

          Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript