Styjun

Why would people reject a god's purely beneficial blessing?

Why does the numerical solution of an ODE move away from an unstable equilibrium?

How to append a matrix element by element

Go Get the Six Six-Pack

Require advice on power conservation for backpacking trip

Can the US president have someone sent to jail?

When is the original BFGS algorithm still better than the Limited-Memory version?

90s (or earlier) cross-world fantasy book with a circular river and character-class tattoos

Do flight schools typically have dress codes or expectations?

What is the legal status of travelling with (unprescribed) methadone in your carry-on?

First-year PhD giving a talk among well-established researchers in the field

What are the benefits of using the X Card safety tool in comparison to plain communication?

Why is the Turkish president's surname spelt in Russian as Эрдоган, with г?

Why is the G major to Bb major resolution so strong?

Analog is Obtuse!

Should I include salary information on my CV?

How risky is real estate?

Impossible darts scores

Fetch and print all properties of an object graph as string

Through the Looking-Glass

Why aren't (poly-)cotton tents more popular?

Alphabet completion rate

Animation advice please

Going to get married soon, should I do it on Dec 31 or Jan 1?

H5PY problem saving composite numpy arrays

h5py: convert numpy data to native python typesHow can I copy a multidimensional h5py dataset to a flat 1D Python list without making any intermediate copies?Writing a multidimensional structured numpy array to hdf5 one field at a time with h5py raises a numpy broadcasting errorAttempt to open h5py file, returns errorno = 17, error message = 'file exists'Can't import numpy from CEstimator with numpy array input_fn“ValueError: Not a location id (Invalid object id)” while creating HDF5 datasetsStore ndarray in a PyTable (and how to define the Col()-type)Does pandas.HDFStore support MPI parallel writing to the HDF5 file?Error executing rnn model . How to fix it?

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

1

In an attempt to reverse-engineer a file format, I have arrived at a following minimal example for creating a composite numpy datatype and saving it to HDF5. The original file seems to be storing datasets of the below data type. However, I do not seem to be able to write such datasets to a file.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.array(data, dtype=data_type)
print(arr)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", arr, dtype=data_type)
h5f.close()

This code crashes with the error

Traceback (most recent call last):
File "test.py", line 13, in
dset = h5f.create_dataset("data", arr, dtype=data_type)
File "/opt/anaconda3/lib/python3.7/site-packages/h5py/_hl/group.py", line
116, in create_dataset
dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
File "/opt/anaconda3/lib/python3.7/site-packages/h5py/_hl/dataset.py", line
75, in make_new_dset
shape = tuple(shape)
TypeError: iteration over a 0-d array

How can I overcome this issue?

python numpy h5py

share|improve this question

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You need to use f.create_dataset('foo1', data=arr) syntax. WIthout a keyword, the second argument is assumed to the shape. So always use data= when providing the actual array.
  
  – hpaulj
  Mar 25 at 16:05
  

  
  
  
  

add a comment | 

1

In an attempt to reverse-engineer a file format, I have arrived at a following minimal example for creating a composite numpy datatype and saving it to HDF5. The original file seems to be storing datasets of the below data type. However, I do not seem to be able to write such datasets to a file.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.array(data, dtype=data_type)
print(arr)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", arr, dtype=data_type)
h5f.close()

This code crashes with the error

Traceback (most recent call last):
File "test.py", line 13, in
dset = h5f.create_dataset("data", arr, dtype=data_type)
File "/opt/anaconda3/lib/python3.7/site-packages/h5py/_hl/group.py", line
116, in create_dataset
dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
File "/opt/anaconda3/lib/python3.7/site-packages/h5py/_hl/dataset.py", line
75, in make_new_dset
shape = tuple(shape)
TypeError: iteration over a 0-d array

How can I overcome this issue?

python numpy h5py

share|improve this question

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You need to use f.create_dataset('foo1', data=arr) syntax. WIthout a keyword, the second argument is assumed to the shape. So always use data= when providing the actual array.
  
  – hpaulj
  Mar 25 at 16:05
  

  
  
  
  

add a comment | 

In an attempt to reverse-engineer a file format, I have arrived at a following minimal example for creating a composite numpy datatype and saving it to HDF5. The original file seems to be storing datasets of the below data type. However, I do not seem to be able to write such datasets to a file.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.array(data, dtype=data_type)
print(arr)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", arr, dtype=data_type)
h5f.close()

This code crashes with the error

Traceback (most recent call last):
File "test.py", line 13, in
dset = h5f.create_dataset("data", arr, dtype=data_type)
File "/opt/anaconda3/lib/python3.7/site-packages/h5py/_hl/group.py", line
116, in create_dataset
dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
File "/opt/anaconda3/lib/python3.7/site-packages/h5py/_hl/dataset.py", line
75, in make_new_dset
shape = tuple(shape)
TypeError: iteration over a 0-d array

How can I overcome this issue?

python numpy h5py

share|improve this question

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.array(data, dtype=data_type)
print(arr)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", arr, dtype=data_type)
h5f.close()

share|improve this question

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

share|improve this question

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

edited Mar 25 at 11:35

Chamila Maddumage

89011 gold badge1010 silver badges2424 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

asked Mar 25 at 10:24

Aleksejs FominsAleksejs Fomins

25822 silver badges1414 bronze badges

1 Answer
1

active

oldest

votes

0

I restructured/reordered your code to get it to work with h5py.
The code below works for 1 row. You will have to adjust to make the number of rows a variable.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.zeros((1,), dtype=data_type)
arr[0]['index'] = "Many cats".encode()
arr[0]['values'] = np.linspace(0, 1, 20)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", data=arr)

h5f.close()

share|improve this answer

edited Mar 25 at 16:37

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I don't think there was a problem in creating arr. A scalar, 0d, array is fine. The problem was in calling create_dataset. You use data=arr, he didn't.
  
  – hpaulj
  Mar 25 at 16:06
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55335664%2fh5py-problem-saving-composite-numpy-arrays%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

I restructured/reordered your code to get it to work with h5py.
The code below works for 1 row. You will have to adjust to make the number of rows a variable.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.zeros((1,), dtype=data_type)
arr[0]['index'] = "Many cats".encode()
arr[0]['values'] = np.linspace(0, 1, 20)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", data=arr)

h5f.close()

share|improve this answer

edited Mar 25 at 16:37

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I don't think there was a problem in creating arr. A scalar, 0d, array is fine. The problem was in calling create_dataset. You use data=arr, he didn't.
  
  – hpaulj
  Mar 25 at 16:06
  

  
  
  
  

add a comment | 

0

I restructured/reordered your code to get it to work with h5py.
The code below works for 1 row. You will have to adjust to make the number of rows a variable.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.zeros((1,), dtype=data_type)
arr[0]['index'] = "Many cats".encode()
arr[0]['values'] = np.linspace(0, 1, 20)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", data=arr)

h5f.close()

share|improve this answer

edited Mar 25 at 16:37

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I don't think there was a problem in creating arr. A scalar, 0d, array is fine. The problem was in calling create_dataset. You use data=arr, he didn't.
  
  – hpaulj
  Mar 25 at 16:06
  

  
  
  
  

add a comment | 

I restructured/reordered your code to get it to work with h5py.
The code below works for 1 row. You will have to adjust to make the number of rows a variable.

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.zeros((1,), dtype=data_type)
arr[0]['index'] = "Many cats".encode()
arr[0]['values'] = np.linspace(0, 1, 20)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", data=arr)

h5f.close()

share|improve this answer

edited Mar 25 at 16:37

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges

import numpy as np
import h5py

data = ("Many cats".encode(), np.linspace(0, 1, 20))
data_type = [('index', 'S' + str(len(data[0]))), ('values', '<f8', (20,))]

arr = np.zeros((1,), dtype=data_type)
arr[0]['index'] = "Many cats".encode()
arr[0]['values'] = np.linspace(0, 1, 20)

h5f = h5py.File("lol.h5", 'w')
dset = h5f.create_dataset("data", data=arr)

h5f.close()

share|improve this answer

edited Mar 25 at 16:37

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges

answered Mar 25 at 15:21

kcw78kcw78

70911 gold badge33 silver badges1515 bronze badges