如何将 pip 命令的结果存储到 Pandas Dataframe 中

Question

为了获取已安装库的列表，我在 Jupyter Notebook 中运行以下命令：

!pip list

我得到以下输出：

Package                            Version
---------------------------------- ---------
alabaster                          0.7.12
altair                             4.1.0
anaconda-client                    1.7.2
anaconda-navigator                 1.9.7
anaconda-project                   0.8.3
asn1crypto                         0.24.0
astroid                            2.2.5
astropy                            3.2.1
atomicwrites                       1.3.0
attrs                              20.3.0
alabaster                          0.7.12
altair                             4.1.0
anaconda-client                    1.7.2
anaconda-navigator                 1.9.7
anaconda-project                   0.8.3
asn1crypto                         0.24.0
astroid                            2.2.5
astropy                            3.2.1
atomicwrites                       1.3.0
attrs                              20.3.0
Automat                            20.2.0
azureml                            0.2.7
Babel                              2.7.0
backcall                           0.1.0
backports.functools-lru-cache      1.5
backports.os                       0.1.1
backports.shutil-get-terminal-size 1.0.0
backports.tempfile                 1.0
backports.weakref                  1.0.post1
beautifulsoup4                     4.7.1
bitarray                           0.9.3
bkcharts                           0.2
bleach                             3.1.0
bokeh                              1.2.0
boto                               2.49.0
boto3                              1.14.61
botocore                           1.17.61
Bottleneck                         1.2.1
branca                             0.4.1
category-encoders                  2.2.2
certifi                            2019.6.16
cffi                               1.12.3
chardet                            3.0.4
Click                              7.0
click-spinner                      0.1.10
cloudpickle                        1.2.1
clyent                             1.2.2
colorama                           0.4.1
colorlog                           4.2.1
comtypes                           1.1.7
conda                              4.7.10
conda-build                        3.18.8
conda-package-handling             1.3.11
conda-verify                       3.4.2
configparser                       5.0.1
constantly                         15.1.0
contextlib2                        0.5.5
coverage                           5.3
coveralls                          2.1.2
cryptography                       2.7
cycler                             0.10.0
Cython                             0.29.14
cytoolz                            0.10.0
dacite                             1.5.1
dask                               2.1.0
decorator                          4.4.0
defusedxml                         0.6.0
distributed                        2.1.0
docopt                             0.6.2
docutils                           0.14
entrypoints                        0.3
et-xmlfile                         1.0.1
fastcache                          1.1.0
feature-engine                     0.6.1
filelock                           3.0.12
Flask                              1.1.1
flit-core                          2.3.0
folium                             0.11.0
furl                               2.1.0
future                             0.17.1
gensim                             3.8.3
genson                             1.2.2
gevent                             1.4.0
glob2                              0.7
greenlet                           0.4.15
h5py                               2.9.0
heapdict                           1.0.0
html5lib                           1.0.1
hyperlink                          20.0.1
hypothesis                         5.41.2
idna                               2.8
imageio                            2.5.0
imagesize                          1.1.0
imbalanced-learn                   0.7.0
importlib-metadata                 0.17
incremental                        17.5.0
ipykernel                          5.1.1
ipython                            7.6.1
ipython-genutils                   0.2.0
ipywidgets                         7.5.0
isort                              4.3.21
itsdangerous                       1.1.0
jdcal                              1.4.1
jedi                               0.13.3
Jinja2                             2.11.2
jmespath                           0.10.0
joblib                             0.13.2
json5                              0.8.4
jsonschema                         3.2.0
jupyter                            1.0.0
jupyter-client                     5.3.1
jupyter-console                    6.0.0
jupyter-core                       4.5.0
jupyterlab                         1.0.2
jupyterlab-server                  1.0.0
keyring                            18.0.0
kiwisolver                         1.1.0
lazy-object-proxy                  1.4.1
libarchive-c                       2.8
llvmlite                           0.29.0
locket                             0.2.0
lxml                               4.5.2
MarkupSafe                         1.1.1
matplotlib                         3.1.0
mccabe                             0.6.1
menuinst                           1.4.16
mistune                            0.8.4
mkl-fft                            1.0.12
mkl-random                         1.0.2
mkl-service                        2.0.2
mock                               3.0.5
more-itertools                     7.0.0
mpmath                             1.1.0
msgpack                            0.6.1
multipledispatch                   0.6.0
munch                              2.5.0
navigator-updater                  0.2.1
nbconvert                          5.6.1
nbformat                           4.4.0
networkx                           2.3
nibabel                            3.1.1
nltk                               3.4.4
nose                               1.3.7
notebook                           6.0.0
numba                              0.44.1
numexpr                            2.6.9
numpy                              1.19.2
numpydoc                           0.9.1
olefile                            0.46
openpyxl                           2.6.2
orderedmultidict                   1.0.1
packaging                          20.4
pandas                             1.1.3
pandocfilters                      1.4.2
parso                              0.5.0
partd                              1.0.0
path.py                            12.0.1
pathlib2                           2.3.4
patsy                              0.5.1
pep517                             0.8.2
pep8                               1.7.1
pickleshare                        0.7.5
Pillow                             6.1.0
pip                                20.2.4
pkginfo                            1.5.0.1
plotly                             4.9.0
pluggy                             0.12.0
ply                                3.11
pmdarima                           1.7.1
prometheus-client                  0.7.1
prompt-toolkit                     2.0.9
psutil                             5.6.3
psycopg2                           2.8.6
py                                 1.8.0
pycodestyle                        2.5.0
pycosat                            0.6.3
pycparser                          2.19
pycrypto                           2.6.1
pycurl                             7.43.0.3
pyflakes                           2.1.1
Pygments                           2.4.2
PyHamcrest                         2.0.2
pylint                             2.3.1
pyodbc                             4.0.26
pyOpenSSL                          19.0.0
pyparsing                          2.4.0
pyreadline                         2.1
pyrsistent                         0.14.11
PySocks                            1.7.0
pytest                             5.0.1
pytest-arraydiff                   0.3
pytest-astropy                     0.5.0
pytest-cov                         2.10.1
pytest-doctestplus                 0.3.0
pytest-openfiles                   0.3.2
pytest-remotedata                  0.3.1
python-dateutil                    2.8.0
pytoml                             0.1.21
pytz                               2019.1
PyWavelets                         1.0.3
pywin32                            223
pywinpty                           0.5.5
PyYAML                             5.3.1
pyzmq                              18.0.0
QtAwesome                          0.5.7
qtconsole                          4.5.1
QtPy                               1.8.0
requests                           2.22.0
requests-toolbelt                  0.9.1
retrying                           1.3.3
rope                               0.14.0
ruamel-yaml                        0.15.46
ruamel.yaml                        0.16.12
ruamel.yaml.clib                   0.2.2
s3transfer                         0.3.3
scikit-image                       0.15.0
scikit-learn                       0.23.2
scipy                              1.5.3
seaborn                            0.10.1
Send2Trash                         1.5.0
setuptools                         41.0.1
simplegeneric                      0.8.1
simplejson                         3.17.2
singledispatch                     3.4.0.3
six                                1.12.0
sklearn                            0.0
smart-open                         2.1.1
snowballstemmer                    1.9.0
sortedcollections                  1.1.2
sortedcontainers                   2.1.0
soupsieve                          1.8
Sphinx                             2.1.2
sphinxcontrib-applehelp            1.0.1
sphinxcontrib-devhelp              1.0.1
sphinxcontrib-htmlhelp             1.0.2
sphinxcontrib-jsmath               1.0.1
sphinxcontrib-qthelp               1.0.2
sphinxcontrib-serializinghtml      1.1.3
sphinxcontrib-websupport           1.1.2
spyder                             3.3.6
spyder-kernels                     0.5.1
SQLAlchemy                         1.3.5
statsmodels                        0.11.1
stumpy                             1.5.0
sympy                              1.4
tables                             3.5.2
tabpy                              2.3.1
tabulate                           0.8.7
tblib                              1.4.0
terminado                          0.8.2
testpath                           0.4.2
textblob                           0.15.3
threadpoolctl                      2.1.0
toml                               0.10.1
toolz                              0.10.0
tornado                            6.0.3
tqdm                               4.32.1
traitlets                          4.3.2
Twisted                            20.3.0
unicodecsv                         0.14.1
urllib3                            1.24.2
validators                         0.16.0
wcwidth                            0.1.7
webencodings                       0.5.1
Werkzeug                           0.15.4
wheel                              0.33.4
widgetsnbextension                 3.5.0
win-inet-pton                      1.1.0
win-unicode-console                0.5
wincertstore                       0.2
wrapt                              1.11.2
xlrd                               1.2.0
XlsxWriter                         1.1.8
xlwings                            0.15.8
xlwt                               1.3.0
zict                               1.0.0
zipp                               0.5.1
zope.interface                     5.2.0

如何将此结果存储到 Pandas Dataframe 中？只需复制数据并粘贴到两个列表中就可以完成这项工作，但是是否有任何基于代码的替代方案来实现这一目标？简而言之，如何将 pip 命令的结果存储到 Pandas Dataframe 中？

P.S：我提到了here，但没有任何帮助。

Answer 1

您可以使用subprocess 在内存中获取结果，而不是创建新文件。

import subprocess
from io import StringIO
command = "pip list"
process = subprocess.Popen(command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()
pd.read_csv(StringIO(output.decode("utf-8")),  sep=r"\s*", skiprows=[1])

Answer 2

我们可以使用os模块创建pip列表，然后我们使用pandas.read_csv和\s+作为分隔符将pip列表读入数据框：

import os
import pandas as pd

# create pip list txt
os.system('pip list > pip_list.txt')

# read content into pandas df
df = pd.read_csv('pip_list.txt', sep=r'\s+', skiprows=[1])

# clean up
os.remove('pip_list.txt')

             Package  Version
0            appnope    0.1.0
1        argon2-cffi   20.1.0
2            astroid    2.4.2
3    async-generator     1.10
4              attrs   20.2.0
..               ...      ...
100          urllib3  1.25.10
101          wcwidth    0.2.5
102     webencodings    0.5.1
103            wrapt   1.12.1
104             xlrd    1.2.0

[105 rows x 2 columns]