[Python] How to handle inf and NaN in numpy mean, standard deviation, maximum / minimum

Problems that occurred

For the output of image processing (image evaluation) software using numpy in python3, Inf and NaN are included in the mean, standard deviation, maximum, and minimum.

sample_code.py


import numpy as np
img = #Somehow an array of images

print(np.max(img))
print(np.min(img))
print(np.mean(img))
print(np.std(img))

>inf
>-inf
>NaN
>NaN

Like this I was quite addicted to it

What are inf and NaN?

In the first place, what is inf or NaN **? (Gekiuma Gaku) When I looked it up, a lot of information came out. inf Abbreviation for infinity. Represents infinity. -inf is negative infinity, isn't it?

NaN

Floating-point numbers have a special number called NaN (Not a Number) that represents anomalous real numbers. This happens when you perform calculations that cannot be represented by real numbers, such as infinity-infinity, indeterminate forms such as 0.0 / 0.0, square roots of negative numbers, and logarithms of negative numbers.

From NaN Story

My understanding is that values (numbers) that you do not want to consider or exclude during calculation are recognized.

Countermeasure

1. Replace inf with NaN

sample_code.py


import numpy as np
img = #Somehow an array of images

img[img == -np.inf] = np.nan
img[img == np.inf] = np.nan

Now the inf disappears from the array. Let's move on

2. Use a function that considers NaN

sample_code.py


import numpy as np
img = #Somehow an array of images

img[img == -np.inf] = np.nan
img[img == np.inf] = np.nan

minimum,maximum= np.nanmax(img), np.nanmin(img)
average= np.nanmean(img)
standard deviation= np.nanstd(img)

#There are also totals and variances

Reference: https://note.nkmk.me/python-numpy-nansum/

3. A story to deepen understanding

I'm talking about what kind of processing this is done. When calculating the maximum and average, ** exclude the NaN element ** and calculate. For example, in the case of average calculation, it is not included in the total value of the whole and is not included in the number of elements.

An image that deletes NaN from the array. Then you may think that you should delete it, but that is not the case with image processing. If you delete it, it will be the same as the vertical and horizontal size of the image.

For the time being, if you divide by 0 or divide by 0, inf and NaN will be included, so Is it a feeling that software that does such processing should basically perform the above processing?

Afterword

It was quite difficult to notice this. I often automate image processing work like I do with imageJ. I noticed that the output result of the software and the result processed by imageJ were different. I felt that inf would be excluded. Is it common sense in the world of image processing?

Recommended Posts

[Python] How to handle inf and NaN in numpy mean, standard deviation, maximum / minimum
How to handle Japanese in Python
How to use is and == in Python
How to generate permutations in Python and C ++
How to output "Ketsumaimo" as standard output in Python
How to handle datetime type in python sqlite3
How to plot autocorrelation and partial autocorrelation in python
Calculation of standard deviation and correlation coefficient in Python
[Python] How to sort dict in list and instance in list
Calculate mean, median, mode, variance, standard deviation in Python
How to handle JSON in Ruby, Python, JavaScript, PHP
How to debug the Python standard library in Visual Studio
How to swap elements in an array in Python, and how to reverse an array.
[Introduction to Udemy Python 3 + Application] 36. How to use In and Not
To go back and forth between standard python, numpy, pandas ①
A standard way to develop and distribute packages in Python
Comparison of how to use higher-order functions in Python 2 and 3
[Blender] How to handle mouse and keyboard events in Blender scripts
How to execute external shell scripts and commands in python
How to log in to AtCoder with Python and submit automatically
[Python] How to do PCA in Python
How to handle session in SQLAlchemy
How to install OpenCV on Cloud9 and run it in Python
How to collect images in Python
How to use SQLite in Python
Difference in how to write if statement between ruby ​​and python
How to use Mysql in python
[ROS2] How to describe remap and parameter in python format launch
How to wrap C in Python
How to use ChemSpider in Python
How to use PubChem in Python
How to display bytes in the same way in Java and Python
How to write the correct shebang in Perl, Python and Ruby scripts
How to get the date and time difference in seconds with python
[Python] How to put any number of standard inputs in a list
How to put a half-width space before letters and numbers in Python.
[Python] How to use list 2 Reference of list value, number of elements, maximum value, minimum value
How to sort by specifying a column in the Python Numpy array.
How to stop a program in python until a specific date and time
How to package and distribute Python scripts
[Introduction to Python] How to use class in Python?
How to access environment variables in Python
How to dynamically define variables in Python
How to install and use pandas_datareader [Python]
How to do R chartr () in Python
[Itertools.permutations] How to put permutations in Python
How to work with BigQuery in Python
How to get a stacktrace in python
How to display multiplication table in python
How to handle consecutive values in MySQL
How to switch python versions in cloud9
How to adjust image contrast in Python
How to use __slots__ in Python class
How to dynamically zero pad in Python
[Python] How to calculate MAE and RMSE
How to use regular expressions in Python
How to display Hello world in python
Stock price and statistics (mean, standard deviation)
2. Mean and standard deviation with neural network!
How to write Ruby to_s in Python
[Python] Precautions when finding the maximum and minimum values in a numpy array with a small number of elements