read a JSON file and format it to CSVHow do I check whether a file exists without exceptions?How do I copy a file in Python?How do you read from stdin?Why can't Python parse this JSON data?How do I list all files of a directory?How to read a file line-by-line into a list?String formatting: % vs. .formatDelete a file or folderWhy is reading lines from stdin much slower in C++ than Python?How do I write JSON data to a file?
Could flaps be raised upward to serve as spoilers / lift dumpers?
The grades of the students in a class
How can flights operated by the same company have such different prices when marketed by another?
Is this mechanically safe?
mv Command Deleted Files In Source Directory and Target Directory
UX writing: When to use "we"?
Accurately recalling the key - can everyone do it?
How to innovate in OR
How to structure presentation to avoid getting questions that will be answered later in the presentation?
Why do MS SQL Server SEQUENCEs not have an ORDER parameter like Oracle?
How to let cacti grow even if no player is near?
Why are sugars in whole fruits not digested the same way sugars in juice are?
How to compare files with diffrent extensions and delete extra files?
How do I safety check that there is no light in Darkroom / Darkbag?
Should students have access to past exams or an exam bank?
Is this popular optical illusion made of a grey-scale image with coloured lines?
Should 2FA be enabled on service accounts?
"DDoouubbllee ssppeeaakk!!"
Delete the following space
How is Sword Coast North governed?
Has the US government provided details on plans to deal with AIDS and childhood cancer?
What is my clock telling me to do?
Password management for kids - what's a good way to start?
How do discovery writers hibernate?
read a JSON file and format it to CSV
How do I check whether a file exists without exceptions?How do I copy a file in Python?How do you read from stdin?Why can't Python parse this JSON data?How do I list all files of a directory?How to read a file line-by-line into a list?String formatting: % vs. .formatDelete a file or folderWhy is reading lines from stdin much slower in C++ than Python?How do I write JSON data to a file?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have to read a json file and extract the data to generate a CSV file.
Server is Redhat 7, python is Python 2.7.5
import time
import os
import sys
import json
with open('abcdc04_abcd11_ig_Host_metrics.json') as data_file:
data = json.load(data_file)
with open('abcdc04_abcd11_ig_Host_metrics.txt', 'w') as f:
for row in data:
symmetrixID= row['symmetrixID']
HostID= row['HostID']
HostMBReads= row['HostMBReads']
timestamp= row['timestamp']
joined = ",".join([symmetrixID , HostID, HostMBReads , timestamp])
f.write(joined)
The result is:
Traceback (most recent call last):
File "./json_scv", line 23, in <module>
symmetrixID= row['symmetrixID']
TypeError: string indices must be integers
My input json file is this:
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
I want to have a csv format look like this:
SymmID,HostName,TimeStamp,HostIOs,HostMBs,ResponseTime,Reads,Writes,HostMBReads,HostMBWrites,ReadResponseTime,WriteResponseTime SyscallCount
000123401234,jupiter_ig,1553637600000,0.12666667,0.000494792,0.15257895,0.12666667,0,0.000494792,0,0.15257895,0,0.21333334
000123401234,jupiter_ig, 1553637600000,0.1264559,0.000493968,0.15828949,0.1264559,0,0.000493968,0,0.15828949,0,0.123128116
000123401234,jupiter_ig,1553637600000,0 ,0,0,0,0,0,0,0,0,0.2
python python-2.x
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
add a comment |
I have to read a json file and extract the data to generate a CSV file.
Server is Redhat 7, python is Python 2.7.5
import time
import os
import sys
import json
with open('abcdc04_abcd11_ig_Host_metrics.json') as data_file:
data = json.load(data_file)
with open('abcdc04_abcd11_ig_Host_metrics.txt', 'w') as f:
for row in data:
symmetrixID= row['symmetrixID']
HostID= row['HostID']
HostMBReads= row['HostMBReads']
timestamp= row['timestamp']
joined = ",".join([symmetrixID , HostID, HostMBReads , timestamp])
f.write(joined)
The result is:
Traceback (most recent call last):
File "./json_scv", line 23, in <module>
symmetrixID= row['symmetrixID']
TypeError: string indices must be integers
My input json file is this:
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
I want to have a csv format look like this:
SymmID,HostName,TimeStamp,HostIOs,HostMBs,ResponseTime,Reads,Writes,HostMBReads,HostMBWrites,ReadResponseTime,WriteResponseTime SyscallCount
000123401234,jupiter_ig,1553637600000,0.12666667,0.000494792,0.15257895,0.12666667,0,0.000494792,0,0.15257895,0,0.21333334
000123401234,jupiter_ig, 1553637600000,0.1264559,0.000493968,0.15828949,0.1264559,0,0.000493968,0,0.15828949,0,0.123128116
000123401234,jupiter_ig,1553637600000,0 ,0,0,0,0,0,0,0,0,0.2
python python-2.x
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
1
Afterdata = json.load(data_file), the variabledatacontains a Python dictionary, so thefor row in data:is iterating of the _keys` of that dictionary — in other words, it's not a list.
– martineau
Mar 26 at 23:40
Max: Glad you understood my mangled comment — as often happens when I type too quickly.
– martineau
Mar 27 at 0:13
add a comment |
I have to read a json file and extract the data to generate a CSV file.
Server is Redhat 7, python is Python 2.7.5
import time
import os
import sys
import json
with open('abcdc04_abcd11_ig_Host_metrics.json') as data_file:
data = json.load(data_file)
with open('abcdc04_abcd11_ig_Host_metrics.txt', 'w') as f:
for row in data:
symmetrixID= row['symmetrixID']
HostID= row['HostID']
HostMBReads= row['HostMBReads']
timestamp= row['timestamp']
joined = ",".join([symmetrixID , HostID, HostMBReads , timestamp])
f.write(joined)
The result is:
Traceback (most recent call last):
File "./json_scv", line 23, in <module>
symmetrixID= row['symmetrixID']
TypeError: string indices must be integers
My input json file is this:
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
I want to have a csv format look like this:
SymmID,HostName,TimeStamp,HostIOs,HostMBs,ResponseTime,Reads,Writes,HostMBReads,HostMBWrites,ReadResponseTime,WriteResponseTime SyscallCount
000123401234,jupiter_ig,1553637600000,0.12666667,0.000494792,0.15257895,0.12666667,0,0.000494792,0,0.15257895,0,0.21333334
000123401234,jupiter_ig, 1553637600000,0.1264559,0.000493968,0.15828949,0.1264559,0,0.000493968,0,0.15828949,0,0.123128116
000123401234,jupiter_ig,1553637600000,0 ,0,0,0,0,0,0,0,0,0.2
python python-2.x
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
I have to read a json file and extract the data to generate a CSV file.
Server is Redhat 7, python is Python 2.7.5
import time
import os
import sys
import json
with open('abcdc04_abcd11_ig_Host_metrics.json') as data_file:
data = json.load(data_file)
with open('abcdc04_abcd11_ig_Host_metrics.txt', 'w') as f:
for row in data:
symmetrixID= row['symmetrixID']
HostID= row['HostID']
HostMBReads= row['HostMBReads']
timestamp= row['timestamp']
joined = ",".join([symmetrixID , HostID, HostMBReads , timestamp])
f.write(joined)
The result is:
Traceback (most recent call last):
File "./json_scv", line 23, in <module>
symmetrixID= row['symmetrixID']
TypeError: string indices must be integers
My input json file is this:
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
I want to have a csv format look like this:
SymmID,HostName,TimeStamp,HostIOs,HostMBs,ResponseTime,Reads,Writes,HostMBReads,HostMBWrites,ReadResponseTime,WriteResponseTime SyscallCount
000123401234,jupiter_ig,1553637600000,0.12666667,0.000494792,0.15257895,0.12666667,0,0.000494792,0,0.15257895,0,0.21333334
000123401234,jupiter_ig, 1553637600000,0.1264559,0.000493968,0.15828949,0.1264559,0,0.000493968,0,0.15828949,0,0.123128116
000123401234,jupiter_ig,1553637600000,0 ,0,0,0,0,0,0,0,0,0.2
python python-2.x
python python-2.x
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
edited Mar 26 at 23:34
martineau
74.3k11 gold badges101 silver badges193 bronze badges
74.3k11 gold badges101 silver badges193 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
asked Mar 26 at 23:29
MaxMax
155 bronze badges
155 bronze badges
1
Afterdata = json.load(data_file), the variabledatacontains a Python dictionary, so thefor row in data:is iterating of the _keys` of that dictionary — in other words, it's not a list.
– martineau
Mar 26 at 23:40
Max: Glad you understood my mangled comment — as often happens when I type too quickly.
– martineau
Mar 27 at 0:13
add a comment |
1
Afterdata = json.load(data_file), the variabledatacontains a Python dictionary, so thefor row in data:is iterating of the _keys` of that dictionary — in other words, it's not a list.
– martineau
Mar 26 at 23:40
Max: Glad you understood my mangled comment — as often happens when I type too quickly.
– martineau
Mar 27 at 0:13
1
1
After
data = json.load(data_file), the variable data contains a Python dictionary, so the for row in data: is iterating of the _keys` of that dictionary — in other words, it's not a list.– martineau
Mar 26 at 23:40
After
data = json.load(data_file), the variable data contains a Python dictionary, so the for row in data: is iterating of the _keys` of that dictionary — in other words, it's not a list.– martineau
Mar 26 at 23:40
Max: Glad you understood my mangled comment — as often happens when I type too quickly.
– martineau
Mar 27 at 0:13
Max: Glad you understood my mangled comment — as often happens when I type too quickly.
– martineau
Mar 27 at 0:13
add a comment |
1 Answer
1
active
oldest
votes
Your variable by the name of data should end up being a dictionary, not a list. So when you try to do "for row in data:", you're saying "Do the following for each key in the dictionary", not for items in a list! Dictionaries are not ordered, but regardless of which key first gets picked as row, the command fails because it can't find anything called "symmetrixID" inside of it. If HostID for example was the first key picked in the loop, then row['symmetrixID'] means data['HostID']['symmetrixID'].
If you look more closely, there's only one list in the dictionary to iterate through, and that's data["perf_data"]. So try the loop there.
So sticking your data in a string for the moment:
s = """
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
"""
Here's how I got the data to format:
import json
data = json.loads(s)
symmetrixID= data['symmetrixID']
HostID= data['HostID']
for row in data['perf_data']:
HostMBReads = row['HostMBReads']
timestamp = row['timestamp']
joined = ",".join([str(c) for c in [symmetrixID, HostID, HostMBReads, timestamp]])
print(joined)
Notice that I changed your joined expression. If you don't change all of those float values to strings first, the join won't work. Regardless, you can replace the print command with the writing command you need.
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
1
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
1
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
|
show 7 more comments
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55367645%2fread-a-json-file-and-format-it-to-csv%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your variable by the name of data should end up being a dictionary, not a list. So when you try to do "for row in data:", you're saying "Do the following for each key in the dictionary", not for items in a list! Dictionaries are not ordered, but regardless of which key first gets picked as row, the command fails because it can't find anything called "symmetrixID" inside of it. If HostID for example was the first key picked in the loop, then row['symmetrixID'] means data['HostID']['symmetrixID'].
If you look more closely, there's only one list in the dictionary to iterate through, and that's data["perf_data"]. So try the loop there.
So sticking your data in a string for the moment:
s = """
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
"""
Here's how I got the data to format:
import json
data = json.loads(s)
symmetrixID= data['symmetrixID']
HostID= data['HostID']
for row in data['perf_data']:
HostMBReads = row['HostMBReads']
timestamp = row['timestamp']
joined = ",".join([str(c) for c in [symmetrixID, HostID, HostMBReads, timestamp]])
print(joined)
Notice that I changed your joined expression. If you don't change all of those float values to strings first, the join won't work. Regardless, you can replace the print command with the writing command you need.
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
1
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
1
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
|
show 7 more comments
Your variable by the name of data should end up being a dictionary, not a list. So when you try to do "for row in data:", you're saying "Do the following for each key in the dictionary", not for items in a list! Dictionaries are not ordered, but regardless of which key first gets picked as row, the command fails because it can't find anything called "symmetrixID" inside of it. If HostID for example was the first key picked in the loop, then row['symmetrixID'] means data['HostID']['symmetrixID'].
If you look more closely, there's only one list in the dictionary to iterate through, and that's data["perf_data"]. So try the loop there.
So sticking your data in a string for the moment:
s = """
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
"""
Here's how I got the data to format:
import json
data = json.loads(s)
symmetrixID= data['symmetrixID']
HostID= data['HostID']
for row in data['perf_data']:
HostMBReads = row['HostMBReads']
timestamp = row['timestamp']
joined = ",".join([str(c) for c in [symmetrixID, HostID, HostMBReads, timestamp]])
print(joined)
Notice that I changed your joined expression. If you don't change all of those float values to strings first, the join won't work. Regardless, you can replace the print command with the writing command you need.
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
1
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
1
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
|
show 7 more comments
Your variable by the name of data should end up being a dictionary, not a list. So when you try to do "for row in data:", you're saying "Do the following for each key in the dictionary", not for items in a list! Dictionaries are not ordered, but regardless of which key first gets picked as row, the command fails because it can't find anything called "symmetrixID" inside of it. If HostID for example was the first key picked in the loop, then row['symmetrixID'] means data['HostID']['symmetrixID'].
If you look more closely, there's only one list in the dictionary to iterate through, and that's data["perf_data"]. So try the loop there.
So sticking your data in a string for the moment:
s = """
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
"""
Here's how I got the data to format:
import json
data = json.loads(s)
symmetrixID= data['symmetrixID']
HostID= data['HostID']
for row in data['perf_data']:
HostMBReads = row['HostMBReads']
timestamp = row['timestamp']
joined = ",".join([str(c) for c in [symmetrixID, HostID, HostMBReads, timestamp]])
print(joined)
Notice that I changed your joined expression. If you don't change all of those float values to strings first, the join won't work. Regardless, you can replace the print command with the writing command you need.
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
Your variable by the name of data should end up being a dictionary, not a list. So when you try to do "for row in data:", you're saying "Do the following for each key in the dictionary", not for items in a list! Dictionaries are not ordered, but regardless of which key first gets picked as row, the command fails because it can't find anything called "symmetrixID" inside of it. If HostID for example was the first key picked in the loop, then row['symmetrixID'] means data['HostID']['symmetrixID'].
If you look more closely, there's only one list in the dictionary to iterate through, and that's data["perf_data"]. So try the loop there.
So sticking your data in a string for the moment:
s = """
"symmetrixID": "000123401234",
"HostID": "jupiter_ig",
"perf_data": [
"HostMBReads": 0.00024720083,
"timestamp": 1553637300000,
"Writes": 0.0,
"ReadResponseTime": 0.15273508,
"Reads": 0.06328341,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15273508,
"SyscallCount": 0.09326678,
"HostMBWrites": 0.0,
"HostIOs": 0.06328341,
"MBs": 0.00024720083
,
"HostMBReads": 0.0004939684,
"timestamp": 1553637600000,
"Writes": 0.0,
"ReadResponseTime": 0.15828949,
"Reads": 0.1264559,
"WriteResponseTime": 0.0,
"ResponseTime": 0.15828949,
"SyscallCount": 0.123128116,
"HostMBWrites": 0.0,
"HostIOs": 0.1264559,
"MBs": 0.0004939684
,
"HostMBReads": 0.0,
"timestamp": 1553637900000,
"Writes": 0.0,
"ReadResponseTime": 0.0,
"Reads": 0.0,
"WriteResponseTime": 0.0,
"ResponseTime": 0.0,
"SyscallCount": 0.2,
"HostMBWrites": 0.0,
"HostIOs": 0.0,
"MBs": 0.0
],
"reporting_level": "Host"
"""
Here's how I got the data to format:
import json
data = json.loads(s)
symmetrixID= data['symmetrixID']
HostID= data['HostID']
for row in data['perf_data']:
HostMBReads = row['HostMBReads']
timestamp = row['timestamp']
joined = ",".join([str(c) for c in [symmetrixID, HostID, HostMBReads, timestamp]])
print(joined)
Notice that I changed your joined expression. If you don't change all of those float values to strings first, the join won't work. Regardless, you can replace the print command with the writing command you need.
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
answered Mar 26 at 23:47
Bill M.Bill M.
1,0211 gold badge1 silver badge13 bronze badges
1,0211 gold badge1 silver badge13 bronze badges
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
1
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
1
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
|
show 7 more comments
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
1
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
1
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but still errors
– Max
Mar 27 at 0:07
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
Thanks Bill, but now i have this : Traceback (most recent call last): File "./1json_scv", line 21, in <module> data = json.loads(s) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
– Max
Mar 27 at 0:09
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
There's a difference between json.load() and json.loads(). One takes a file path as input, whereas the other takes a string as input. If you're using json.loads() and not json.load(), are you sure your input is a string?
– Bill M.
Mar 27 at 0:13
1
1
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
Bill it worked thanks , i started python only 3 weeks a go the result is good now 000197802257,namecws11_ig,0.00024720083,1553637300000 000197802257,namecws11_ig,0.0004939684,1553637600000 000197802257,namecws11_ig,0.0,1553637900000
– Max
Mar 27 at 0:49
1
1
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
you are right i open a new one thanks for your precious help
– Max
Mar 28 at 16:11
|
show 7 more comments
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55367645%2fread-a-json-file-and-format-it-to-csv%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
After
data = json.load(data_file), the variabledatacontains a Python dictionary, so thefor row in data:is iterating of the _keys` of that dictionary — in other words, it's not a list.– martineau
Mar 26 at 23:40
Max: Glad you understood my mangled comment — as often happens when I type too quickly.
– martineau
Mar 27 at 0:13