The “ValueError: input contains NaN, infinity or a value too large for dtype(‘float64’)” error occurs when there are missing values or infinity values in your dataset. These values can cause issues when performing operations or calculations on the data.
Here are some steps you can take to fix this error:
- Identify the columns or variables that contain NaN, infinity, or large values. You can use the
isna() or isnull() functions to find missing values, and the np.isinf() function to find infinity values.
- Depending on the number of missing or infinity values, you may choose to drop the rows or columns containing these values using the
dropna() function.
- Alternatively, you can fill in missing values with an appropriate value using the
fillna() function. For example, you can fill in missing values with the mean or median of the column.
- If you have large values, you may need to scale your data using normalization or standardization techniques. You can use libraries such as scikit-learn to perform these operations.
- Lastly, you can convert the data type of your dataframe to a larger data type using the
astype() function. For example, you can convert your data type to float128.
Here is an example code snippet that demonstrates how to fill in missing values with the mean of the column:
import numpy as np
import pandas as pd
# create a dataframe with NaN values
df = pd.DataFrame({‘A’: [1, 2, np.nan, 4], ‘B’: [5, np.nan, np.inf, 8]})
# fill NaN values with the mean of the column
df = df.fillna(df.mean())
# print the resulting dataframe
print(df)
.