XArray: add a “layer” of data to NetCDFAdd new keys to a dictionary?Extend dimensions in netCDF file using Rxarray writing to netCDF from Pandas - dimension issueHow can I save a 3 column data frame into a NetCDF file in R?Converting raster stack extent from meters to lat/lon coordinatesxarray - cannot serialize coordinatesxarray DataArray.where() reduced coordinate when maskingXarray Data Array from netcdf returns numpy grid array larger than inputxarray - resample time series data from daily to hourlyUsing xarray to change coordinate system in order to Slice operation
Multi tool use
Is it more effective to add yeast before or after kneading?
Which museums have artworks of all four Ninja Turtles' namesakes?
What do you do if you have developments on your paper during the long peer review process?
What is the meaning of word 'crack' in chapter 33 of A Game of Thrones?
reverse a list of generic type
Where are they calling from?
A high quality contribution but an annoying error is present in my published article
Is It Possible to Have Different Sea Levels, Eventually Causing New Landforms to Appear?
Where Does VDD+0.3V Input Limit Come From on IC chips?
Do the villains know Batman has no superpowers?
Can the U.S. president make military decisions without consulting anyone?
Why is there is no screening for Ovarian Cancer?
can my paper be excluded from the conference proceedings after attending and presenting?
Do all creatures have souls?
Examples of "unsuccessful" theories with afterlives
Is it impolite to ask for halal food when traveling to and in Thailand?
Way of the bicycle
I reverse the source code, you negate the output!
Are there non JavaScript ways to hide HTML source code?
Allocating credit card points
How does IBM's 53-bit quantum computer compare to classical ones for cryptanalytic tasks?
How is the problem, ⟨G⟩ in Logspace?
Safely hang a mirror that does not have hooks
Late 1970's and 6502 chip facilities for operating systems
XArray: add a “layer” of data to NetCDF
Add new keys to a dictionary?Extend dimensions in netCDF file using Rxarray writing to netCDF from Pandas - dimension issueHow can I save a 3 column data frame into a NetCDF file in R?Converting raster stack extent from meters to lat/lon coordinatesxarray - cannot serialize coordinatesxarray DataArray.where() reduced coordinate when maskingXarray Data Array from netcdf returns numpy grid array larger than inputxarray - resample time series data from daily to hourlyUsing xarray to change coordinate system in order to Slice operation
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
Set up
suppose I have a NetCDF file that stores a number of rasters indexed by date
, longitudes
and latitudes
, loaded in memory with XArray
with name "stack":
<xarray.Dataset>
Dimensions: (date: 1, lat: 2000, lon: 7200)
Coordinates:
* date (date) datetime64[ns] 2000-01-01
* lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
* lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
Data variables:
rainfall (date, lat, lon) float64 ...
task
to add a new date
to the stack.
method
my approach is to create a Dataset
"new" from the raster with the same indices as the NetCDF loaded:
xr.DataArray(
<some numpy data>,
dims=['date', 'lat', 'lon'],
coords=
'date': [<some datetime64>],
'lat': <same list of latitudes>,
'lon': <same list of longitudes>
,
name='rainfall'
).to_dataset()
and then concatenate:
merged = xr.concat([stack, new], dim='date')
This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date
and data; something like:
stack[<new_date>] = <some numpy data>
python netcdf python-xarray
add a comment
|
Set up
suppose I have a NetCDF file that stores a number of rasters indexed by date
, longitudes
and latitudes
, loaded in memory with XArray
with name "stack":
<xarray.Dataset>
Dimensions: (date: 1, lat: 2000, lon: 7200)
Coordinates:
* date (date) datetime64[ns] 2000-01-01
* lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
* lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
Data variables:
rainfall (date, lat, lon) float64 ...
task
to add a new date
to the stack.
method
my approach is to create a Dataset
"new" from the raster with the same indices as the NetCDF loaded:
xr.DataArray(
<some numpy data>,
dims=['date', 'lat', 'lon'],
coords=
'date': [<some datetime64>],
'lat': <same list of latitudes>,
'lon': <same list of longitudes>
,
name='rainfall'
).to_dataset()
and then concatenate:
merged = xr.concat([stack, new], dim='date')
This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date
and data; something like:
stack[<new_date>] = <some numpy data>
python netcdf python-xarray
add a comment
|
Set up
suppose I have a NetCDF file that stores a number of rasters indexed by date
, longitudes
and latitudes
, loaded in memory with XArray
with name "stack":
<xarray.Dataset>
Dimensions: (date: 1, lat: 2000, lon: 7200)
Coordinates:
* date (date) datetime64[ns] 2000-01-01
* lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
* lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
Data variables:
rainfall (date, lat, lon) float64 ...
task
to add a new date
to the stack.
method
my approach is to create a Dataset
"new" from the raster with the same indices as the NetCDF loaded:
xr.DataArray(
<some numpy data>,
dims=['date', 'lat', 'lon'],
coords=
'date': [<some datetime64>],
'lat': <same list of latitudes>,
'lon': <same list of longitudes>
,
name='rainfall'
).to_dataset()
and then concatenate:
merged = xr.concat([stack, new], dim='date')
This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date
and data; something like:
stack[<new_date>] = <some numpy data>
python netcdf python-xarray
Set up
suppose I have a NetCDF file that stores a number of rasters indexed by date
, longitudes
and latitudes
, loaded in memory with XArray
with name "stack":
<xarray.Dataset>
Dimensions: (date: 1, lat: 2000, lon: 7200)
Coordinates:
* date (date) datetime64[ns] 2000-01-01
* lat (lat) float64 49.97 49.92 49.87 49.82 ... -49.88 -49.93 -49.98
* lon (lon) float64 -180.0 -179.9 -179.9 -179.8 ... 179.9 179.9 180.0
Data variables:
rainfall (date, lat, lon) float64 ...
task
to add a new date
to the stack.
method
my approach is to create a Dataset
"new" from the raster with the same indices as the NetCDF loaded:
xr.DataArray(
<some numpy data>,
dims=['date', 'lat', 'lon'],
coords=
'date': [<some datetime64>],
'lat': <same list of latitudes>,
'lon': <same list of longitudes>
,
name='rainfall'
).to_dataset()
and then concatenate:
merged = xr.concat([stack, new], dim='date')
This works but is not very elegant and being new to XArray maybe there is a better way to go about this, for example just with some indexing routines say adding a new date
and data; something like:
stack[<new_date>] = <some numpy data>
python netcdf python-xarray
python netcdf python-xarray
asked Mar 28 at 15:50
lorenzorilorenzori
5613 silver badges16 bronze badges
5613 silver badges16 bronze badges
add a comment
|
add a comment
|
2 Answers
2
active
oldest
votes
After some work I have a workaround using netCDF4-python
which is useful if you are looking for a way not to load the entire netCDF in memory at once.
The original netCDF file is processed with XArray
but I fallback to netCDF4
for this particular operation.
An example follows where I want to add a timestep to the unlimited variable date
. The other 2 variables are longitudes and latitudes.
First off I open the netCDF using netCDF4 and read the variables date
that I will extend as well as the data
d = Dataset('dataset.nc', 'a')
dt = d.variables['date']
data = d.variables['data']
after that I add the numpy array to a slice:
data[len(dt):len(dt)+1, :,:] = <some numpy data>
and finally add the extra time step:
from datetime import datetime
from netCDF4 import date2num
dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)
Hope for it to be useful to others.
add a comment
|
Try to use reindex
to extend the original DataArray, and then assign value with indexing.
extra_date = <some datetime64>
date_extended = np.concatenate([stack.date, [extra_date]]
# this will extend the arrays and place NaNs in the new position
stack_extended = stack.reindex('date': date_extended)
# now assign to that position
stack_extended.loc[dict(date=extra_data)] = <some numpy data>
hey thanks for this. I did try to use.loc
before but actually I don't think you can assign data to it, i.e. you getTypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is yourxarray.__version__
?
– Ryan
Apr 4 at 14:58
but only if I have thex=1
already in there. What if I want to add a new one, sayda.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.
– lorenzori
Apr 5 at 8:06
add a comment
|
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55401863%2fxarray-add-a-layer-of-data-to-netcdf%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
After some work I have a workaround using netCDF4-python
which is useful if you are looking for a way not to load the entire netCDF in memory at once.
The original netCDF file is processed with XArray
but I fallback to netCDF4
for this particular operation.
An example follows where I want to add a timestep to the unlimited variable date
. The other 2 variables are longitudes and latitudes.
First off I open the netCDF using netCDF4 and read the variables date
that I will extend as well as the data
d = Dataset('dataset.nc', 'a')
dt = d.variables['date']
data = d.variables['data']
after that I add the numpy array to a slice:
data[len(dt):len(dt)+1, :,:] = <some numpy data>
and finally add the extra time step:
from datetime import datetime
from netCDF4 import date2num
dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)
Hope for it to be useful to others.
add a comment
|
After some work I have a workaround using netCDF4-python
which is useful if you are looking for a way not to load the entire netCDF in memory at once.
The original netCDF file is processed with XArray
but I fallback to netCDF4
for this particular operation.
An example follows where I want to add a timestep to the unlimited variable date
. The other 2 variables are longitudes and latitudes.
First off I open the netCDF using netCDF4 and read the variables date
that I will extend as well as the data
d = Dataset('dataset.nc', 'a')
dt = d.variables['date']
data = d.variables['data']
after that I add the numpy array to a slice:
data[len(dt):len(dt)+1, :,:] = <some numpy data>
and finally add the extra time step:
from datetime import datetime
from netCDF4 import date2num
dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)
Hope for it to be useful to others.
add a comment
|
After some work I have a workaround using netCDF4-python
which is useful if you are looking for a way not to load the entire netCDF in memory at once.
The original netCDF file is processed with XArray
but I fallback to netCDF4
for this particular operation.
An example follows where I want to add a timestep to the unlimited variable date
. The other 2 variables are longitudes and latitudes.
First off I open the netCDF using netCDF4 and read the variables date
that I will extend as well as the data
d = Dataset('dataset.nc', 'a')
dt = d.variables['date']
data = d.variables['data']
after that I add the numpy array to a slice:
data[len(dt):len(dt)+1, :,:] = <some numpy data>
and finally add the extra time step:
from datetime import datetime
from netCDF4 import date2num
dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)
Hope for it to be useful to others.
After some work I have a workaround using netCDF4-python
which is useful if you are looking for a way not to load the entire netCDF in memory at once.
The original netCDF file is processed with XArray
but I fallback to netCDF4
for this particular operation.
An example follows where I want to add a timestep to the unlimited variable date
. The other 2 variables are longitudes and latitudes.
First off I open the netCDF using netCDF4 and read the variables date
that I will extend as well as the data
d = Dataset('dataset.nc', 'a')
dt = d.variables['date']
data = d.variables['data']
after that I add the numpy array to a slice:
data[len(dt):len(dt)+1, :,:] = <some numpy data>
and finally add the extra time step:
from datetime import datetime
from netCDF4 import date2num
dt[len(dt)-1] = date2num(datetime(<year>, <month>, <day>), dt.units)
Hope for it to be useful to others.
answered Apr 24 at 13:16
lorenzorilorenzori
5613 silver badges16 bronze badges
5613 silver badges16 bronze badges
add a comment
|
add a comment
|
Try to use reindex
to extend the original DataArray, and then assign value with indexing.
extra_date = <some datetime64>
date_extended = np.concatenate([stack.date, [extra_date]]
# this will extend the arrays and place NaNs in the new position
stack_extended = stack.reindex('date': date_extended)
# now assign to that position
stack_extended.loc[dict(date=extra_data)] = <some numpy data>
hey thanks for this. I did try to use.loc
before but actually I don't think you can assign data to it, i.e. you getTypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is yourxarray.__version__
?
– Ryan
Apr 4 at 14:58
but only if I have thex=1
already in there. What if I want to add a new one, sayda.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.
– lorenzori
Apr 5 at 8:06
add a comment
|
Try to use reindex
to extend the original DataArray, and then assign value with indexing.
extra_date = <some datetime64>
date_extended = np.concatenate([stack.date, [extra_date]]
# this will extend the arrays and place NaNs in the new position
stack_extended = stack.reindex('date': date_extended)
# now assign to that position
stack_extended.loc[dict(date=extra_data)] = <some numpy data>
hey thanks for this. I did try to use.loc
before but actually I don't think you can assign data to it, i.e. you getTypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is yourxarray.__version__
?
– Ryan
Apr 4 at 14:58
but only if I have thex=1
already in there. What if I want to add a new one, sayda.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.
– lorenzori
Apr 5 at 8:06
add a comment
|
Try to use reindex
to extend the original DataArray, and then assign value with indexing.
extra_date = <some datetime64>
date_extended = np.concatenate([stack.date, [extra_date]]
# this will extend the arrays and place NaNs in the new position
stack_extended = stack.reindex('date': date_extended)
# now assign to that position
stack_extended.loc[dict(date=extra_data)] = <some numpy data>
Try to use reindex
to extend the original DataArray, and then assign value with indexing.
extra_date = <some datetime64>
date_extended = np.concatenate([stack.date, [extra_date]]
# this will extend the arrays and place NaNs in the new position
stack_extended = stack.reindex('date': date_extended)
# now assign to that position
stack_extended.loc[dict(date=extra_data)] = <some numpy data>
answered Apr 3 at 2:39
RyanRyan
4362 silver badges11 bronze badges
4362 silver badges11 bronze badges
hey thanks for this. I did try to use.loc
before but actually I don't think you can assign data to it, i.e. you getTypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is yourxarray.__version__
?
– Ryan
Apr 4 at 14:58
but only if I have thex=1
already in there. What if I want to add a new one, sayda.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.
– lorenzori
Apr 5 at 8:06
add a comment
|
hey thanks for this. I did try to use.loc
before but actually I don't think you can assign data to it, i.e. you getTypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is yourxarray.__version__
?
– Ryan
Apr 4 at 14:58
but only if I have thex=1
already in there. What if I want to add a new one, sayda.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.
– lorenzori
Apr 5 at 8:06
hey thanks for this. I did try to use
.loc
before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
hey thanks for this. I did try to use
.loc
before but actually I don't think you can assign data to it, i.e. you get TypeError: '_LocIndexer' object does not support item assignment
– lorenzori
Apr 3 at 7:03
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your
xarray.__version__
?– Ryan
Apr 4 at 14:58
loc does in general support item assignment ` da = xr.DataArray([0, 0, 0], dims=['x'], coords='x': [0, 1, 2]) da.loc[dict(x=1)] = 5 ` What is your
xarray.__version__
?– Ryan
Apr 4 at 14:58
but only if I have the
x=1
already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.– lorenzori
Apr 5 at 8:06
but only if I have the
x=1
already in there. What if I want to add a new one, say da.loc[dict(x=3)] = 5
? I am running the conda distribution v 0.11.– lorenzori
Apr 5 at 8:06
add a comment
|
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55401863%2fxarray-add-a-layer-of-data-to-netcdf%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
lONZq7Do,KjV A3sM9FNesB6QY4Pl iQ,SydPyisK NYPPajw8,3X k5Jy,NyZP63Fz503opEI0,WpeViA,zcKN 6VXCDUxU6zMjGn9e0,4,L0KV