将多个 txt 文件读入 Dict 到 Pandas 数据帧中

Question

我正在尝试将多个 txt 文件加载到数据框中。我知道如何加载 url、csv 和 excel，但我找不到任何关于如何将多个 txt 文件加载到数据框中并与字典匹配或反之亦然的参考。

文本文件不是逗号或制表符分隔的，只是包含纯文本歌词的纯文本。

我检查了 pandas 文档，欢迎提供任何帮助。

https://pandas.pydata.org/pandas-docs/stable/reference/io.html

理想的数据框

我希望实现的数据框就像这个例子

                 |                                                        lyrics
    -------------+-----------------------------------------------------------------------------------------
    bonjovi      |    some text from the text files HiHello! WelcomeThank you Thank you for coming.
    -------------+---------------------------------------------------------------------------------------
    lukebryan    |    some other text from the text files.Hi.Hello WelcomeThank you Thank you for coming. 
    -------------+-----------------------------------------------------------------------------------------
    johnprine    |    yet some text from the text files. Hi.Hello WelcomeThank you Thank you for coming. 

基本示例 文件夹结构 /lyrics/

urls = 

    'lyrics/bonjovi.txt',
    'lyrics/lukebryan.txt',
    'lyrics/johnprine.txt',
    'lyrics/brunomars.txt',
    'lyrics/methodman.txt',
    'lyrics/bobmarley.txt',
    'lyrics/nickcannon.txt',
    'lyrics/weeknd.txt',
    'lyrics/dojacat.txt',
    'lyrics/ladygaga.txt',
    'lyrics/dualipa.txt',
    'lyrics/justinbieber.txt',]

音乐家名字

bands = ['bonjovi', 'lukebryan', 'johnprine', 'brunomars', 'methodman', 'bobmarley', 'nickcannon', 'weeknd', 'dojacat', 'ladygaga', 'dualipa', 'justinbieber']

打开文本文件 这些文件位于我运行 Jupyter 笔记本的目录 Lyrics/ 中。

for i, c in enumerate(bands):
     with open("lyrics/" + c + ".txt", "wb") as file:
         pickle.dump(lyrics[i], file)

仔细检查以确保数据已正确加载

data.keys()

希望得到这样的结果

dict_keys(['bonjovi', 'lukebryan', 'johnprine', 'brunomars', 'methodman', 'bobmarley', 'nickcannon', 'weeknd', 'dojacat', 'ladygaga', 'dualipa', '贾斯汀比伯'])

# Combine it!
data_combined = {key: [combine_text(value)] for (key, value) in data.items()}


# We are going to change this to key: artist, value: string format
def combine_text(list_of_text):
    '''Takes a list of text and combines them into one large chunk of text.'''
    combined_text = ' '.join(list_of_text)
    return combined_text

我们可以将其保存为字典格式，也可以将其放入 pandas 数据框

将熊猫导入为 pd

pd.set_option('max_colwidth',150)

data_df = pd.DataFrame.from_dict(data_combined).transpose()
data_df.columns = ['lyrics']
data_df = data_df.sort_index()
data_df

Answer 1

import os
import re
import pandas as pd

#get full path of txt file
filePath = []
for file in os.listdir("./lyrics"):
    filePath.append(os.path.join("./lyrics", file))

#pull file name from text file with regex, capturing the text before the .txt   
fileName = re.compile('\\\\(.*)\.txt')

#make empty dict Data with the key as the file name, and the value as the words in the file.
data = {}
for file in filePath:
    #capturing file name
    key = fileName.search(file)
    with open(file, "r") as readFile:
        # note that key[1] is the capture group from our search, and that the text is put into a list.
        data[key[1]] = [readFile.read()]

#make dataframe from dict, and rename columns.
df = pd.DataFrame(data).T.reset_index().rename(columns = {'index':'bands', 0:'lyrics'})

这就是我的做法。请注意，我概括了文件操作，因此我不必担心手动为键创建列表，并确保所有内容都匹配。