将电影文本数据解析为数据框

Question

我有一些来自电影脚本的 .txt 数据，看起来像这样。

                              JOHN
                   Hi man. How are you?

                              TOM
                   A little hungry but okay.

                              JOHN
                   Let's get breakfast then

我想解析出文本并创建一个包含 2 列的数据框。 I 代表人，例如 JOHN 和 TOM，第二列代表行（这是每个名称下方的文本块）。结果会像..

索引 |人 |线条

0 |约翰 | “嗨，伙计。你好吗？”

1 |汤姆 | “有点饿，但还好。”

2 |约翰 | “那我们去吃早餐吧”

Answer 1

如果每个文本都在一行中，则可以拆分为行

 lines = text.split('\n')

删除空格

 lines = [x.strip() for x in lines]

并使用 slice [start:end:step] 创建数据框

df = pd.DataFrame({
    'person': lines[0::3],
    'lines':  lines[1::3]
})

---

例子：

text = '''                            JOHN
                   Hi man. How are you?

                              TOM
                   A little hungry but okay.

                              JOHN
                   Let's get breakfast then'''

lines = text.split('\n')
lines = [x.strip() for x in lines]

import pandas as pd
df = pd.DataFrame({
    'person': lines[0::3],
    'lines': lines[1::3]
})

print(df)

结果：

  person                      lines
0   JOHN       Hi man. How are you?
1    TOM  A little hungry but okay.
2   JOHN   Let's get breakfast then

---

如果一个人可能有多行文本 - 即。

                        JOHN
                   Hi man.
                   How are you?

那么它需要更多的分割和条带化。

您可以在创建 DataFrame 之前进行。

text = '''                            JOHN
                   Hi man.
                   How are you?

                              TOM
                   A little hungry but okay.

                              JOHN
                   Let's get breakfast then'''

data = []

parts = text.split('\n\n')
for part in parts:
    person, lines = part.split('\n', 1)
    person = person.strip()
    lines = "\n".join(x.strip() for x in lines.split('\n'))
    data.append([person, lines])

import pandas as pd

df = pd.DataFrame(data)
df.columns = ['person', 'lines']

print(df)

或者你可以在创建DataFrame后尝试做

text = '''                            JOHN
                   Hi man.
                   How are you?

                              TOM
                   A little hungry but okay.

                              JOHN
                   Let's get breakfast then'''

lines = text.split('\n\n')
lines = [x.split('\n', 1) for x in lines]

import pandas as pd

df = pd.DataFrame(lines)
df.columns = ['person', 'lines']

df['person'] = df['person'].str.strip()
df['lines'] = df['lines'].apply(lambda txt: "\n".join(x.strip() for x in txt.split('\n')))

print(df)

结果：

  person                      lines
0   JOHN      Hi man.\nHow are you?
1    TOM  A little hungry but okay.
2   JOHN   Let's get breakfast then

Answer 2

我知道我参加这个聚会迟到了，但这会将整个脚本解析为角色名称字典，并将他们的对话作为值，那么你需要做的就是df = pd.DataFrame(final_dict.values(), columns = final_dict.keys())

*# Grouped regex pattern to capture char and dialouge in a tuple
char_dialogue = re.compile(r"(?m)^\s*\b([A-Z]+)\b\s*\n(.*(?:\n.+)*)")
extract_dialogue = char_dialogue.findall(script)

final_dict = {}

for element in extract_dialogue:
   # Seperating the character and dialogue from the tuple
   char = element[0]
   line = element[1]
   # If the char is already a key in the dictionary
   # and line is not empty append the dialogue to the value list
   if char in final_dict:
       if line != '':
           final_dict[char].append(line)
   else:
       # Else add the character name to the dictionary keys with their first line
       # Drop any lower case matches from group 0
       # Can adjust the len here if you have characters with fewer letters
       if char.isupper() and len(char) >2:
           final_dict[char] = [line]

        
# Some final cleaning to drop empty dalouge 

final_dict = {k: v for k, v in final_dict.items() if v  != ['']}

# More filtering to reutrn only main characters with more than 50 
# lines of dialogue 

final_dict = {k: v for k, v in final_dict.items() if len(v) > 50}*