Find any cell values >= x in Pandas dataframe and return cell value, column header, row and neighbouring cell value Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) Data science time! April 2019 and salary with experience The Ask Question Wizard is Live!Add one row to pandas DataFrameSelecting multiple columns in a pandas dataframeAdding new column to existing DataFrame in Python pandasDelete column from pandas DataFrame by column nameHow to drop rows of Pandas DataFrame whose value in certain columns is NaNHow do I get the row count of a pandas DataFrame?How to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandasDeleting DataFrame row in Pandas based on column valueGet list from pandas DataFrame column headers
Table formatting with tabularx?
Short story about astronauts fertilizing soil with their own bodies
Is the time—manner—place ordering of adverbials an oversimplification?
Does the main washing effect of soap come from foam?
What is a more techy Technical Writer job title that isn't cutesy or confusing?
Sally's older brother
.bashrc alias for a command with fixed second parameter
What did Turing mean when saying that "machines cannot give rise to surprises" is due to a fallacy?
latest version of QGIS fails to edit attribute table of GeoJSON file
New Order #6: Easter Egg
How to achieve cat-like agility?
Is a copyright notice with a non-existent name be invalid?
Are there any irrational/transcendental numbers for which the distribution of decimal digits is not uniform?
The bible of geometry: Is there a modern treatment of geometries from the most primitive to the most advanced?
"Destructive power" carried by a B-52?
Did John Wesley plagiarize Matthew Henry...?
How much damage would a cupful of neutron star matter do to the Earth?
Centre cell contents vertically
Find general formula for the terms
Statistical analysis applied to methods coming out of Machine Learning
As a dual citizen, my US passport will expire one day after traveling to the US. Will this work?
Is there a spell that can create a permanent fire?
newbie Q : How to read an output file in one command line
Relating to the President and obstruction, were Mueller's conclusions preordained?
Find any cell values >= x in Pandas dataframe and return cell value, column header, row and neighbouring cell value
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)
Data science time! April 2019 and salary with experience
The Ask Question Wizard is Live!Add one row to pandas DataFrameSelecting multiple columns in a pandas dataframeAdding new column to existing DataFrame in Python pandasDelete column from pandas DataFrame by column nameHow to drop rows of Pandas DataFrame whose value in certain columns is NaNHow do I get the row count of a pandas DataFrame?How to iterate over rows in a DataFrame in Pandas?Select rows from a DataFrame based on values in a column in pandasDeleting DataFrame row in Pandas based on column valueGet list from pandas DataFrame column headers
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I realise this is quite a lengthy ask, but I have been trying to solve this for days now with no success and wondered if anyone might have some ideas.
Consider a spreadsheet like so:
apple1 grape1 apple2 grape2 apple3 grape3
1 0 4 -0.2 2 0 4
2 0 4 0 6 0 3
3 -0.1 2 0 4 0 4
4 -0.5 5 0 6 -0.2 5
5 -0.4 4 0 5 0 2
6 0 6 -0.1 5 0 3
I would like to search my dataframe for any cell with a value less than -0.1, and write the value, column header, row number, and neighbouring value out.
At the start, I though it might be as simple as something along the lines of:
Newlist()
if df >= -0.1:
Newlist.append(cell.value)
Newlist.append(row.value)
Newlist.append(column.value)
Newlist.append(cell.value.shift(1))
I fully realise the above makes no sense, but I hope it conveys the idea of what I've been trying to do.
Next, I could convert the df to a list and work from there( using an ifnot >= -0.1 to delete objects?), but this seems inelegant and far from ideal. I am however open to this if anyone can get it to work.
I must have looked at every stack exchange question ever posted on this without managing anything so apologies if I've overlooked something very obvious.
Thanks!
python pandas
add a comment |
I realise this is quite a lengthy ask, but I have been trying to solve this for days now with no success and wondered if anyone might have some ideas.
Consider a spreadsheet like so:
apple1 grape1 apple2 grape2 apple3 grape3
1 0 4 -0.2 2 0 4
2 0 4 0 6 0 3
3 -0.1 2 0 4 0 4
4 -0.5 5 0 6 -0.2 5
5 -0.4 4 0 5 0 2
6 0 6 -0.1 5 0 3
I would like to search my dataframe for any cell with a value less than -0.1, and write the value, column header, row number, and neighbouring value out.
At the start, I though it might be as simple as something along the lines of:
Newlist()
if df >= -0.1:
Newlist.append(cell.value)
Newlist.append(row.value)
Newlist.append(column.value)
Newlist.append(cell.value.shift(1))
I fully realise the above makes no sense, but I hope it conveys the idea of what I've been trying to do.
Next, I could convert the df to a list and work from there( using an ifnot >= -0.1 to delete objects?), but this seems inelegant and far from ideal. I am however open to this if anyone can get it to work.
I must have looked at every stack exchange question ever posted on this without managing anything so apologies if I've overlooked something very obvious.
Thanks!
python pandas
2
Welcome to StackOverflow. Please post your expected output to make Minimal, Complete and Verifiable example. Also, your title and question contradict each other with cell value >=x in title and cell with a value less than -0.1, which one is it?
– Chris
Mar 22 at 12:24
add a comment |
I realise this is quite a lengthy ask, but I have been trying to solve this for days now with no success and wondered if anyone might have some ideas.
Consider a spreadsheet like so:
apple1 grape1 apple2 grape2 apple3 grape3
1 0 4 -0.2 2 0 4
2 0 4 0 6 0 3
3 -0.1 2 0 4 0 4
4 -0.5 5 0 6 -0.2 5
5 -0.4 4 0 5 0 2
6 0 6 -0.1 5 0 3
I would like to search my dataframe for any cell with a value less than -0.1, and write the value, column header, row number, and neighbouring value out.
At the start, I though it might be as simple as something along the lines of:
Newlist()
if df >= -0.1:
Newlist.append(cell.value)
Newlist.append(row.value)
Newlist.append(column.value)
Newlist.append(cell.value.shift(1))
I fully realise the above makes no sense, but I hope it conveys the idea of what I've been trying to do.
Next, I could convert the df to a list and work from there( using an ifnot >= -0.1 to delete objects?), but this seems inelegant and far from ideal. I am however open to this if anyone can get it to work.
I must have looked at every stack exchange question ever posted on this without managing anything so apologies if I've overlooked something very obvious.
Thanks!
python pandas
I realise this is quite a lengthy ask, but I have been trying to solve this for days now with no success and wondered if anyone might have some ideas.
Consider a spreadsheet like so:
apple1 grape1 apple2 grape2 apple3 grape3
1 0 4 -0.2 2 0 4
2 0 4 0 6 0 3
3 -0.1 2 0 4 0 4
4 -0.5 5 0 6 -0.2 5
5 -0.4 4 0 5 0 2
6 0 6 -0.1 5 0 3
I would like to search my dataframe for any cell with a value less than -0.1, and write the value, column header, row number, and neighbouring value out.
At the start, I though it might be as simple as something along the lines of:
Newlist()
if df >= -0.1:
Newlist.append(cell.value)
Newlist.append(row.value)
Newlist.append(column.value)
Newlist.append(cell.value.shift(1))
I fully realise the above makes no sense, but I hope it conveys the idea of what I've been trying to do.
Next, I could convert the df to a list and work from there( using an ifnot >= -0.1 to delete objects?), but this seems inelegant and far from ideal. I am however open to this if anyone can get it to work.
I must have looked at every stack exchange question ever posted on this without managing anything so apologies if I've overlooked something very obvious.
Thanks!
python pandas
python pandas
asked Mar 22 at 12:16
Alex BrownAlex Brown
31
31
2
Welcome to StackOverflow. Please post your expected output to make Minimal, Complete and Verifiable example. Also, your title and question contradict each other with cell value >=x in title and cell with a value less than -0.1, which one is it?
– Chris
Mar 22 at 12:24
add a comment |
2
Welcome to StackOverflow. Please post your expected output to make Minimal, Complete and Verifiable example. Also, your title and question contradict each other with cell value >=x in title and cell with a value less than -0.1, which one is it?
– Chris
Mar 22 at 12:24
2
2
Welcome to StackOverflow. Please post your expected output to make Minimal, Complete and Verifiable example. Also, your title and question contradict each other with cell value >=x in title and cell with a value less than -0.1, which one is it?
– Chris
Mar 22 at 12:24
Welcome to StackOverflow. Please post your expected output to make Minimal, Complete and Verifiable example. Also, your title and question contradict each other with cell value >=x in title and cell with a value less than -0.1, which one is it?
– Chris
Mar 22 at 12:24
add a comment |
1 Answer
1
active
oldest
votes
First, to filter your dataframe you can use boolean indexing like this :
df[df >= -0.1]
This way, all the data that is not superior to -0.1 will be displayed as nan, you can then use Pandas.isnull() to identify them.
To get the row and columns of the data you want, you could turn your dataframe into an array with df.to_numpy() and iterate over the rows and columns with enumerate to keep the id of row/column you are currently iterating through :
my_data = df[df >= -0.1].to_numpy()
for idrow, row in enumerate(my_data):
for idcol, col in enumerate(row):
if not pd.isnull(col):
print("Value :"+str(col)+" column:"+str(idcol)+" row:"+str(idrow))
This will result in something like this :
Value :0.0 column:0 row:0
Value :4.0 column:1 row:0
Value :2.0 column:3 row:0
You can get columns name by using this in the loop :
df.columns[idcol]
Once you got those ids, you can get the neighbouring values by direct access ie.
my_data[x][y]
Just remember to set a condition to not access value that are not in the array !
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55299420%2ffind-any-cell-values-x-in-pandas-dataframe-and-return-cell-value-column-head%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
First, to filter your dataframe you can use boolean indexing like this :
df[df >= -0.1]
This way, all the data that is not superior to -0.1 will be displayed as nan, you can then use Pandas.isnull() to identify them.
To get the row and columns of the data you want, you could turn your dataframe into an array with df.to_numpy() and iterate over the rows and columns with enumerate to keep the id of row/column you are currently iterating through :
my_data = df[df >= -0.1].to_numpy()
for idrow, row in enumerate(my_data):
for idcol, col in enumerate(row):
if not pd.isnull(col):
print("Value :"+str(col)+" column:"+str(idcol)+" row:"+str(idrow))
This will result in something like this :
Value :0.0 column:0 row:0
Value :4.0 column:1 row:0
Value :2.0 column:3 row:0
You can get columns name by using this in the loop :
df.columns[idcol]
Once you got those ids, you can get the neighbouring values by direct access ie.
my_data[x][y]
Just remember to set a condition to not access value that are not in the array !
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
add a comment |
First, to filter your dataframe you can use boolean indexing like this :
df[df >= -0.1]
This way, all the data that is not superior to -0.1 will be displayed as nan, you can then use Pandas.isnull() to identify them.
To get the row and columns of the data you want, you could turn your dataframe into an array with df.to_numpy() and iterate over the rows and columns with enumerate to keep the id of row/column you are currently iterating through :
my_data = df[df >= -0.1].to_numpy()
for idrow, row in enumerate(my_data):
for idcol, col in enumerate(row):
if not pd.isnull(col):
print("Value :"+str(col)+" column:"+str(idcol)+" row:"+str(idrow))
This will result in something like this :
Value :0.0 column:0 row:0
Value :4.0 column:1 row:0
Value :2.0 column:3 row:0
You can get columns name by using this in the loop :
df.columns[idcol]
Once you got those ids, you can get the neighbouring values by direct access ie.
my_data[x][y]
Just remember to set a condition to not access value that are not in the array !
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
add a comment |
First, to filter your dataframe you can use boolean indexing like this :
df[df >= -0.1]
This way, all the data that is not superior to -0.1 will be displayed as nan, you can then use Pandas.isnull() to identify them.
To get the row and columns of the data you want, you could turn your dataframe into an array with df.to_numpy() and iterate over the rows and columns with enumerate to keep the id of row/column you are currently iterating through :
my_data = df[df >= -0.1].to_numpy()
for idrow, row in enumerate(my_data):
for idcol, col in enumerate(row):
if not pd.isnull(col):
print("Value :"+str(col)+" column:"+str(idcol)+" row:"+str(idrow))
This will result in something like this :
Value :0.0 column:0 row:0
Value :4.0 column:1 row:0
Value :2.0 column:3 row:0
You can get columns name by using this in the loop :
df.columns[idcol]
Once you got those ids, you can get the neighbouring values by direct access ie.
my_data[x][y]
Just remember to set a condition to not access value that are not in the array !
First, to filter your dataframe you can use boolean indexing like this :
df[df >= -0.1]
This way, all the data that is not superior to -0.1 will be displayed as nan, you can then use Pandas.isnull() to identify them.
To get the row and columns of the data you want, you could turn your dataframe into an array with df.to_numpy() and iterate over the rows and columns with enumerate to keep the id of row/column you are currently iterating through :
my_data = df[df >= -0.1].to_numpy()
for idrow, row in enumerate(my_data):
for idcol, col in enumerate(row):
if not pd.isnull(col):
print("Value :"+str(col)+" column:"+str(idcol)+" row:"+str(idrow))
This will result in something like this :
Value :0.0 column:0 row:0
Value :4.0 column:1 row:0
Value :2.0 column:3 row:0
You can get columns name by using this in the loop :
df.columns[idcol]
Once you got those ids, you can get the neighbouring values by direct access ie.
my_data[x][y]
Just remember to set a condition to not access value that are not in the array !
answered Mar 22 at 12:52
Bruce SwainBruce Swain
1316
1316
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
add a comment |
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
Thank you so much for this bruce, it's working brilliantly. The only section I'm having trouble with is right at the end with the my_data[x][y]. I would like to print this under your original phrase so e.g. print("some_word :+my_data[str(idrow)[str(idcol)+1]). This however is throwing up some funny errors. The main problem is sorted however so thank you a tonne for that!
– Alex Brown
Mar 22 at 13:31
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
The errors might be because you try to access element that are not in the array. For example if you have an array like this : x = [0,1,2,3], accessing x[4] will result in error as the array don't have a 5th element. This works in both dimensions. Also you must use an integer to access data contained in an array. with my_data[str(idrow)][str(idcol)+1] you are using string ! so just do my_data[idrow][idcol+1]
– Bruce Swain
Mar 22 at 13:45
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
Working perfectly, thank you again.
– Alex Brown
Mar 22 at 14:11
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55299420%2ffind-any-cell-values-x-in-pandas-dataframe-and-return-cell-value-column-head%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Welcome to StackOverflow. Please post your expected output to make Minimal, Complete and Verifiable example. Also, your title and question contradict each other with cell value >=x in title and cell with a value less than -0.1, which one is it?
– Chris
Mar 22 at 12:24