[pandas] Replace None with NaN in pandas dataframe

I have table x:

        website
0   http://www.google.com/
1   http://www.yahoo.com
2   None

I want to replace python None with pandas NaN. I tried:

x.replace(to_replace=None, value=np.nan)

But I got:

TypeError: 'regex' must be a string or a compiled regular expression or a list or dict of strings or regular expressions, you passed a 'bool'

How should I go about it?

This question is related to pandas dataframe replace nan nonetype

The answer is


The following line replaces None with NaN:

df['column'].replace('None', np.nan, inplace=True)

DataFrame['Col_name'].replace("None", np.nan, inplace=True)

If you use df.replace([None], np.nan, inplace=True), this changed all datetime objects with missing data to object dtypes. So now you may have broken queries unless you change them back to datetime which can be taxing depending on the size of your data.

If you want to use this method, you can first identify the object dtype fields in your df and then replace the None:

obj_columns = list(df.select_dtypes(include=['object']).columns.values)
df[obj_columns] = df[obj_columns].replace([None], np.nan)

Here's another option:

df.replace(to_replace=[None], value=np.nan, inplace=True)

Examples related to pandas

xlrd.biffh.XLRDError: Excel xlsx file; not supported Pandas Merging 101 How to increase image size of pandas.DataFrame.plot in jupyter notebook? Trying to merge 2 dataframes but get ValueError Python Pandas User Warning: Sorting because non-concatenation axis is not aligned How to show all of columns name on pandas dataframe? Pandas/Python: Set value of one column based on value in another column Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Python convert object to float

Examples related to dataframe

Trying to merge 2 dataframes but get ValueError How to show all of columns name on pandas dataframe? Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Display all dataframe columns in a Jupyter Python Notebook How to convert column with string type to int form in pyspark data frame? Display/Print one column from a DataFrame of Series in Pandas Binning column with python pandas Selection with .loc in python Set value to an entire column of a pandas dataframe

Examples related to replace

How do I find and replace all occurrences (in all files) in Visual Studio Code? How to find and replace with regex in excel How to replace text in a column of a Pandas dataframe? How to replace negative numbers in Pandas Data Frame by zero Replacing few values in a pandas dataframe column with another value How to replace multiple patterns at once with sed? Using tr to replace newline with space replace special characters in a string python Replace None with NaN in pandas dataframe Batch script to find and replace a string in text file within a minute for files up to 12 MB

Examples related to nan

Display rows with one or more NaN values in pandas dataframe How to find which columns contain any NaN value in Pandas dataframe How to set a cell to NaN in a pandas dataframe Elegant way to create empty pandas DataFrame with NaN of type float How to check if any value is NaN in a Pandas DataFrame How to replace NaNs by preceding values in pandas DataFrame? Pandas Replace NaN with blank/empty string Convert pandas.Series from dtype object to float, and errors to nans How to filter in NaN (pandas)? Replace None with NaN in pandas dataframe

Examples related to nonetype

Why is this printing 'None' in the output? Replace None with NaN in pandas dataframe How to "test" NoneType in python? What is a 'NoneType' object? 'NoneType' object is not subscriptable? remove None value from a list without removing the 0 value Python None comparison: should I use "is" or ==? Python NoneType object is not callable (beginner) Why do I get AttributeError: 'NoneType' object has no attribute 'something'? Return None if Dictionary key is not available