如何从整个数据框中删除所有非数字数字：调试

Question

我正在尝试使用一行代码从我的数据框中删除所有非数字字符（即 ]$^M# 等字符）。数据框是 Google Play 商店应用数据集。

df = pd.read_csv("googleplaystore.csv")

df['Rating'].fillna(value = '0.0', inplace = True)

#sample data#

   Rating    Reviews                Size     Installs  Type Price  \
0        4.1     159                 19M      10,000+  Free     0   
1        3.9     967                 14M     500,000+  Free     0   
2        4.7   87510                8.7M   5,000,000+  Free     0   
3        4.5  215644                 25M  50,000,000+  Free     0   
4        4.3     967                2.8M     100,000+  Free     0   
...      ...     ...                 ...          ...   ...   ...   
10836    4.5      38                 53M       5,000+  Free     0   
10837      5       4                3.6M         100+  Free     0   
10838    0.0       3                9.5M       1,000+  Free     0   
10839    4.5     114  Varies with device       1,000+  Free     0   
10840    4.5  398307                 19M  10,000,000+  Free     0   


Content Rating                     Genres      Last Updated  \
0           Everyone               Art & Design   January 7, 2018   
1           Everyone  Art & Design;Pretend Play  January 15, 2018   
2           Everyone               Art & Design    August 1, 2018   
3               Teen               Art & Design      June 8, 2018   
4           Everyone    Art & Design;Creativity     June 20, 2018   
...              ...                        ...               ...   
10836       Everyone                  Education     July 25, 2017   
10837       Everyone                  Education      July 6, 2018   
10838       Everyone                    Medical  January 20, 2017   
10839     Mature 17+          Books & Reference  January 19, 2015   
10840       Everyone                  Lifestyle     July 25, 2018

clean_data = df.replace('[^\d.]', '', regex = True).astype(float)

基本上，我试图从“大小”列中删除数字后面的“M”以及“安装”列中的“+”号。

但我收到此错误消息；

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-325-887d47a9889e> in <module>
----> 1 data_ = df.replace('[^\d.]', '', regex = True).astype(float)

~\anaconda3\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors)
   5696         else:
   5697             # else, only a single dtype is given
-> 5698             new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors)
   5699             return self._constructor(new_data).__finalize__(self)
   5700 

~\anaconda3\lib\site-packages\pandas\core\internals\managers.py in astype(self, dtype, copy, errors)
    580 
    581     def astype(self, dtype, copy: bool = False, errors: str = "raise"):
--> 582         return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
    583 
    584     def convert(self, **kwargs):

~\anaconda3\lib\site-packages\pandas\core\internals\managers.py in apply(self, f, filter, **kwargs)
    440                 applied = b.apply(f, **kwargs)
    441             else:
--> 442                 applied = getattr(b, f)(**kwargs)
    443             result_blocks = _extend_blocks(applied, result_blocks)
    444 

~\anaconda3\lib\site-packages\pandas\core\internals\blocks.py in astype(self, dtype, copy, errors)
    623             vals1d = values.ravel()
    624             try:
--> 625                 values = astype_nansafe(vals1d, dtype, copy=True)
    626             except (ValueError, TypeError):
    627                 # e.g. astype_nansafe can fail on object-dtype of strings

~\anaconda3\lib\site-packages\pandas\core\dtypes\cast.py in astype_nansafe(arr, dtype, copy, skipna)
    895     if copy or is_object_dtype(arr) or is_object_dtype(dtype):
    896         # Explicit copy, or required since NumPy can't view from / to object.
--> 897         return arr.astype(dtype, copy=True)
    898 
    899     return arr.view(dtype)

ValueError: could not convert string to float: 

如果可能，请协助调试。我真的很想将它保留在整个数据框的一行代码中。提前谢谢你。

Answer 1

我认为问题是需要指定替换列并将空值替换为 NaN 或 0 如果不是倒数第二个 Size 值的数字：

cols = ['Size','Installs']
df[cols] = df[cols].replace('[^\d.]', '', regex = True).replace('',np.nan).astype(float)

print (df)
       Rating  Reviews  Size    Installs  Type  Price
0         4.1      159  19.0     10000.0  Free      0
1         3.9      967  14.0    500000.0  Free      0
2         4.7    87510   8.7   5000000.0  Free      0
3         4.5   215644  25.0  50000000.0  Free      0
4         4.3      967   2.8    100000.0  Free      0
10836     4.5       38  53.0      5000.0  Free      0
10837     5.0        4   3.6       100.0  Free      0
10838     0.0        3   9.5      1000.0  Free      0
10839     4.5      114   NaN      1000.0  Free      0
10840     4.5   398307  19.0  10000000.0  Free      0

Answer 2

问题是您将数据框中的所有非数字字符替换为“”。

这意味着非数字字符串以“”结尾 - 一个零长度字符串。这不能解释为浮点数，所以会出现错误。

如果您只在评级列上运行替换

df["Rating"].replace('[^\d.]', '', regex = True).astype(float)

然后它会起作用，因为从该列中删除非数字字符会导致一列仅填充可以转换为数字的字符。

但是，在整个数据帧上运行它是行不通的，因为您的许多值都是纯非数字的。例如，流派列最终将只是一列空字符串，从而引发错误。