XArray: add a “layer” of data to NetCDFAdd new keys to a dictionary?Extend dimensions in netCDF file using Rxarray writing to netCDF from Pandas - dimension issueHow can I save a 3 column data frame into a NetCDF file in R?Converting raster stack extent from meters to lat/lon coordinatesxarray - cannot serialize coordinatesxarray DataArray.where() reduced coordinate when maskingXarray Data Array from netcdf returns numpy grid array larger than inputxarray - resample time series data from daily to hourlyUsing xarray to change coordinate system in order to Slice operation

Multi tool use
Multi tool use

Is it more effective to add yeast before or after kneading?

Which museums have artworks of all four Ninja Turtles' namesakes?

What do you do if you have developments on your paper during the long peer review process?

What is the meaning of word 'crack' in chapter 33 of A Game of Thrones?

reverse a list of generic type

Where are they calling from?

A high quality contribution but an annoying error is present in my published article

Is It Possible to Have Different Sea Levels, Eventually Causing New Landforms to Appear?

Where Does VDD+0.3V Input Limit Come From on IC chips?

Do the villains know Batman has no superpowers?

Can the U.S. president make military decisions without consulting anyone?

Why is there is no screening for Ovarian Cancer?

can my paper be excluded from the conference proceedings after attending and presenting?

Do all creatures have souls?

Examples of "unsuccessful" theories with afterlives

Is it impolite to ask for halal food when traveling to and in Thailand?

Way of the bicycle

I reverse the source code, you negate the output!

Are there non JavaScript ways to hide HTML source code?

Allocating credit card points

How does IBM's 53-bit quantum computer compare to classical ones for cryptanalytic tasks?

How is the problem, ⟨G⟩ in Logspace?

Safely hang a mirror that does not have hooks

Late 1970's and 6502 chip facilities for operating systems



XArray: add a “layer” of data to NetCDF


Add new keys to a dictionary?Extend dimensions in netCDF file using Rxarray writing to netCDF from Pandas - dimension issueHow can I save a 3 column data frame into a NetCDF file in R?Converting raster stack extent from meters to lat/lon coordinatesxarray - cannot serialize coordinatesxarray DataArray.where() reduced coordinate when maskingXarray Data Array from netcdf returns numpy grid array larger than inputxarray - resample time series data from daily to hourlyUsing xarray to change coordinate system in order to Slice operation






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








3















Set up



suppose I have a NetCDF file that stores a number of rasters indexed by date, longitudes and latitudes, loaded in memory with XArray with name "stack":



<xarray.Dataset>
Dimensions: (date: 1, lat: 2000, lon: 7200)
Coordinates:
* date (date) datetime64[ns] 2000-01-01
* lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
* lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
Data variables:
rainfall (date, lat, lon) float64 ...


task



to add a new date to the stack.



method



my approach is to create a Dataset "new" from the raster with the same indices as the NetCDF loaded:



xr.DataArray(
<some numpy data>,
dims=['date', 'lat', 'lon'],
coords=
'date': [<some datetime64>],
'lat': <same list of latitudes>,
'lon': <same list of longitudes>
,
name='rainfall'
).to_dataset()


and then concatenate:



merged = xr.concat([stack, new], dim='date')


This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date and data; something like:



stack[<new_date>] = <some numpy data>









share|improve this question






























    3















    Set up



    suppose I have a NetCDF file that stores a number of rasters indexed by date, longitudes and latitudes, loaded in memory with XArray with name "stack":



    <xarray.Dataset>
    Dimensions: (date: 1, lat: 2000, lon: 7200)
    Coordinates:
    * date (date) datetime64[ns] 2000-01-01
    * lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
    * lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
    Data variables:
    rainfall (date, lat, lon) float64 ...


    task



    to add a new date to the stack.



    method



    my approach is to create a Dataset "new" from the raster with the same indices as the NetCDF loaded:



    xr.DataArray(
    <some numpy data>,
    dims=['date', 'lat', 'lon'],
    coords=
    'date': [<some datetime64>],
    'lat': <same list of latitudes>,
    'lon': <same list of longitudes>
    ,
    name='rainfall'
    ).to_dataset()


    and then concatenate:



    merged = xr.concat([stack, new], dim='date')


    This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date and data; something like:



    stack[<new_date>] = <some numpy data>









    share|improve this question


























      3












      3








      3








      Set up



      suppose I have a NetCDF file that stores a number of rasters indexed by date, longitudes and latitudes, loaded in memory with XArray with name "stack":



      <xarray.Dataset>
      Dimensions: (date: 1, lat: 2000, lon: 7200)
      Coordinates:
      * date (date) datetime64[ns] 2000-01-01
      * lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
      * lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
      Data variables:
      rainfall (date, lat, lon) float64 ...


      task



      to add a new date to the stack.



      method



      my approach is to create a Dataset "new" from the raster with the same indices as the NetCDF loaded:



      xr.DataArray(
      <some numpy data>,
      dims=['date', 'lat', 'lon'],
      coords=
      'date': [<some datetime64>],
      'lat': <same list of latitudes>,
      'lon': <same list of longitudes>
      ,
      name='rainfall'
      ).to_dataset()


      and then concatenate:



      merged = xr.concat([stack, new], dim='date')


      This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date and data; something like:



      stack[<new_date>] = <some numpy data>









      share|improve this question














      Set up



      suppose I have a NetCDF file that stores a number of rasters indexed by date, longitudes and latitudes, loaded in memory with XArray with name "stack":



      <xarray.Dataset>
      Dimensions: (date: 1, lat: 2000, lon: 7200)
      Coordinates:
      * date (date) datetime64[ns] 2000-01-01
      * lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
      * lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
      Data variables:
      rainfall (date, lat, lon) float64 ...


      task



      to add a new date to the stack.



      method



      my approach is to create a Dataset "new" from the raster with the same indices as the NetCDF loaded:



      xr.DataArray(
      <some numpy data>,
      dims=['date', 'lat', 'lon'],
      coords=
      'date': [<some datetime64>],
      'lat': <same list of latitudes>,
      'lon': <same list of longitudes>
      ,
      name='rainfall'
      ).to_dataset()


      and then concatenate:



      merged = xr.concat([stack, new], dim='date')


      This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date and data; something like:



      stack[<new_date>] = <some numpy data>






      python netcdf python-xarray






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Mar 28 at 15:50









      lorenzorilorenzori

      5613 silver badges16 bronze badges




      5613 silver badges16 bronze badges

























          2 Answers
          2






          active

          oldest

          votes


















          1
















          After some work I have a workaround using netCDF4-python which is useful if you are looking for a way not to load the entire netCDF in memory at once.



          The original netCDF file is processed with XArray but I fallback to netCDF4 for this particular operation.



          An example follows where I want to add a timestep to the unlimited variable date. The other 2 variables are longitudes and latitudes.



          First off I open the netCDF using netCDF4 and read the variables date that I will extend as well as the data



          d = Dataset('dataset.nc', 'a')
          dt = d.variables['date']
          data = d.variables['data']


          after that I add the numpy array to a slice:



          data[len(dt):len(dt)+1, :,:] = <some numpy data>


          and finally add the extra time step:



          from datetime import datetime
          from netCDF4 import date2num
          dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)


          Hope for it to be useful to others.






          share|improve this answer
































            0
















            Try to use reindex to extend the original DataArray, and then assign value with indexing.



            extra_date = <some datetime64>
            date_extended = np.concatenate([stack.date, [extra_date]]
            # this will extend the arrays and place NaNs in the new position
            stack_extended = stack.reindex('date': date_extended)
            # now assign to that position
            stack_extended.loc[dict(date=extra_data)] = <some numpy data>





            share|improve this answer

























            • hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

              – lorenzori
              Apr 3 at 7:03











            • loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

              – Ryan
              Apr 4 at 14:58












            • but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

              – lorenzori
              Apr 5 at 8:06













            Your Answer






            StackExchange.ifUsing("editor", function ()
            StackExchange.using("externalEditor", function ()
            StackExchange.using("snippets", function ()
            StackExchange.snippets.init();
            );
            );
            , "code-snippets");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "1"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );














            draft saved

            draft discarded
















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55401863%2fxarray-add-a-layer-of-data-to-netcdf%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1
















            After some work I have a workaround using netCDF4-python which is useful if you are looking for a way not to load the entire netCDF in memory at once.



            The original netCDF file is processed with XArray but I fallback to netCDF4 for this particular operation.



            An example follows where I want to add a timestep to the unlimited variable date. The other 2 variables are longitudes and latitudes.



            First off I open the netCDF using netCDF4 and read the variables date that I will extend as well as the data



            d = Dataset('dataset.nc', 'a')
            dt = d.variables['date']
            data = d.variables['data']


            after that I add the numpy array to a slice:



            data[len(dt):len(dt)+1, :,:] = <some numpy data>


            and finally add the extra time step:



            from datetime import datetime
            from netCDF4 import date2num
            dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)


            Hope for it to be useful to others.






            share|improve this answer





























              1
















              After some work I have a workaround using netCDF4-python which is useful if you are looking for a way not to load the entire netCDF in memory at once.



              The original netCDF file is processed with XArray but I fallback to netCDF4 for this particular operation.



              An example follows where I want to add a timestep to the unlimited variable date. The other 2 variables are longitudes and latitudes.



              First off I open the netCDF using netCDF4 and read the variables date that I will extend as well as the data



              d = Dataset('dataset.nc', 'a')
              dt = d.variables['date']
              data = d.variables['data']


              after that I add the numpy array to a slice:



              data[len(dt):len(dt)+1, :,:] = <some numpy data>


              and finally add the extra time step:



              from datetime import datetime
              from netCDF4 import date2num
              dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)


              Hope for it to be useful to others.






              share|improve this answer



























                1














                1










                1









                After some work I have a workaround using netCDF4-python which is useful if you are looking for a way not to load the entire netCDF in memory at once.



                The original netCDF file is processed with XArray but I fallback to netCDF4 for this particular operation.



                An example follows where I want to add a timestep to the unlimited variable date. The other 2 variables are longitudes and latitudes.



                First off I open the netCDF using netCDF4 and read the variables date that I will extend as well as the data



                d = Dataset('dataset.nc', 'a')
                dt = d.variables['date']
                data = d.variables['data']


                after that I add the numpy array to a slice:



                data[len(dt):len(dt)+1, :,:] = <some numpy data>


                and finally add the extra time step:



                from datetime import datetime
                from netCDF4 import date2num
                dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)


                Hope for it to be useful to others.






                share|improve this answer













                After some work I have a workaround using netCDF4-python which is useful if you are looking for a way not to load the entire netCDF in memory at once.



                The original netCDF file is processed with XArray but I fallback to netCDF4 for this particular operation.



                An example follows where I want to add a timestep to the unlimited variable date. The other 2 variables are longitudes and latitudes.



                First off I open the netCDF using netCDF4 and read the variables date that I will extend as well as the data



                d = Dataset('dataset.nc', 'a')
                dt = d.variables['date']
                data = d.variables['data']


                after that I add the numpy array to a slice:



                data[len(dt):len(dt)+1, :,:] = <some numpy data>


                and finally add the extra time step:



                from datetime import datetime
                from netCDF4 import date2num
                dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)


                Hope for it to be useful to others.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Apr 24 at 13:16









                lorenzorilorenzori

                5613 silver badges16 bronze badges




                5613 silver badges16 bronze badges


























                    0
















                    Try to use reindex to extend the original DataArray, and then assign value with indexing.



                    extra_date = <some datetime64>
                    date_extended = np.concatenate([stack.date, [extra_date]]
                    # this will extend the arrays and place NaNs in the new position
                    stack_extended = stack.reindex('date': date_extended)
                    # now assign to that position
                    stack_extended.loc[dict(date=extra_data)] = <some numpy data>





                    share|improve this answer

























                    • hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

                      – lorenzori
                      Apr 3 at 7:03











                    • loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

                      – Ryan
                      Apr 4 at 14:58












                    • but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

                      – lorenzori
                      Apr 5 at 8:06















                    0
















                    Try to use reindex to extend the original DataArray, and then assign value with indexing.



                    extra_date = <some datetime64>
                    date_extended = np.concatenate([stack.date, [extra_date]]
                    # this will extend the arrays and place NaNs in the new position
                    stack_extended = stack.reindex('date': date_extended)
                    # now assign to that position
                    stack_extended.loc[dict(date=extra_data)] = <some numpy data>





                    share|improve this answer

























                    • hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

                      – lorenzori
                      Apr 3 at 7:03











                    • loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

                      – Ryan
                      Apr 4 at 14:58












                    • but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

                      – lorenzori
                      Apr 5 at 8:06













                    0














                    0










                    0









                    Try to use reindex to extend the original DataArray, and then assign value with indexing.



                    extra_date = <some datetime64>
                    date_extended = np.concatenate([stack.date, [extra_date]]
                    # this will extend the arrays and place NaNs in the new position
                    stack_extended = stack.reindex('date': date_extended)
                    # now assign to that position
                    stack_extended.loc[dict(date=extra_data)] = <some numpy data>





                    share|improve this answer













                    Try to use reindex to extend the original DataArray, and then assign value with indexing.



                    extra_date = <some datetime64>
                    date_extended = np.concatenate([stack.date, [extra_date]]
                    # this will extend the arrays and place NaNs in the new position
                    stack_extended = stack.reindex('date': date_extended)
                    # now assign to that position
                    stack_extended.loc[dict(date=extra_data)] = <some numpy data>






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Apr 3 at 2:39









                    RyanRyan

                    4362 silver badges11 bronze badges




                    4362 silver badges11 bronze badges















                    • hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

                      – lorenzori
                      Apr 3 at 7:03











                    • loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

                      – Ryan
                      Apr 4 at 14:58












                    • but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

                      – lorenzori
                      Apr 5 at 8:06

















                    • hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

                      – lorenzori
                      Apr 3 at 7:03











                    • loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

                      – Ryan
                      Apr 4 at 14:58












                    • but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

                      – lorenzori
                      Apr 5 at 8:06
















                    hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

                    – lorenzori
                    Apr 3 at 7:03





                    hey thanks for this. I did try to use .loc before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment

                    – lorenzori
                    Apr 3 at 7:03













                    loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

                    – Ryan
                    Apr 4 at 14:58






                    loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your xarray.__version__?

                    – Ryan
                    Apr 4 at 14:58














                    but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

                    – lorenzori
                    Apr 5 at 8:06





                    but only if I have the x=1 already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5 ? I am running the conda distribution v 0.11.

                    – lorenzori
                    Apr 5 at 8:06


















                    draft saved

                    draft discarded















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55401863%2fxarray-add-a-layer-of-data-to-netcdf%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    lONZq7Do,KjV A3sM9FNesB6QY4Pl iQ,SydPyisK NYPPajw8,3X k5Jy,NyZP63Fz503opEI0,WpeViA,zcKN 6VXCDUxU6zMjGn9e0,4,L0KV
                    O,P XaLQCsTYei6vd51,dJIVnOJ9OApn RuO0

                    Popular posts from this blog

                    Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

                    SQL error code 1064 with creating Laravel foreign keysForeign key constraints: When to use ON UPDATE and ON DELETEDropping column with foreign key Laravel error: General error: 1025 Error on renameLaravel SQL Can't create tableLaravel Migration foreign key errorLaravel php artisan migrate:refresh giving a syntax errorSQLSTATE[42S01]: Base table or view already exists or Base table or view already exists: 1050 Tableerror in migrating laravel file to xampp serverSyntax error or access violation: 1064:syntax to use near 'unsigned not null, modelName varchar(191) not null, title varchar(191) not nLaravel cannot create new table field in mysqlLaravel 5.7:Last migration creates table but is not registered in the migration table

                    은진 송씨 목차 역사 본관 분파 인물 조선 왕실과의 인척 관계 집성촌 항렬자 인구 같이 보기 각주 둘러보기 메뉴은진 송씨세종실록 149권, 지리지 충청도 공주목 은진현