One error you may encounter when using pandas is:
ValueError: Cannot mask with non-boolean array containing NA / NaN values
This error usually occurs when you attempt to find the rows in a pandas DataFrame that contain a particular string, but the column you’re searching in has NaN values.
The following example shows how to fix this error in practice.
How to Reproduce the Error
Suppose we have the following pandas DataFrame:
import pandas as pd
import numpy as np
#create DataFrame
df = pd.DataFrame({'team': ['A', 'A', 'A', 'B', 'B'],
'position': ['Guard', 'Guard', np.nan, 'Guard', 'Forward'],
'points': [22, 28, 14, 13, 19]})
#view DataFrame
print(df)
team position points
0 A Guard 22
1 A Guard 28
2 A NaN 14
3 B Guard 13
4 B Forward 19
Now suppose we attempt to access all rows in the DataFrame where the position column contains the string “Guard”:
#access all rows where position column contains 'Guard'
df[df['position'].str.contains('Guard')]
ValueError: Cannot mask with non-boolean array containing NA / NaN values
We receive an error because there is a NaN value in the position column.
How to Fix the Error
To avoid this error, we simply need to use the argument na=False within the str.contains() function:
#access all rows where position column contains 'Guard', ignore NaN
df[df['position'].str.contains('Guard', na=False)]
team position points
0 A Guard 22
1 A Guard 28
3 B Guard 13
This time we’re able to access all rows that contain “Guard” in the position column without any errors.
Another way to avoid this error is to use .fillna(False) as follows:
#access all rows where position column contains 'Guard', ignore NaN
df[df['position'].str.contains('Guard').fillna(False)]
team position points
0 A Guard 22
1 A Guard 28
3 B Guard 13
Once again we’re able to access all rows that contain “Guard” in the position column without any errors.
Additional Resources
The following tutorials explain how to fix other common errors in Python:
How to Fix KeyError in Pandas
How to Fix: ValueError: cannot convert float NaN to integer
How to Fix: ValueError: operands could not be broadcast together with shapes