【发布时间】:2022-11-21 11:50:32
【问题描述】:
我正在尝试在较大的文本格式中分隔句子(使用 spacy sentencizer),以便在转换器管道中处理它们。
不幸的是,这个流水线无法正确处理句子,因为句子在测试后还不是字符串格式。请参阅以下信息。
string = 'The Chromebook is exactly what it was advertised to be. It is super simple to use. The picture quality is great, stays connected to WIfi with no interruption. Quick, lightweight yet sturdy. I bought the Kindle Fire HD 3G and had so much trouble with battery life, disconnection problems etc. that I hate it and so I bought the Chromebook and absolutely love it. The battery life is good. Finally a product that lives up to its hype!'
#Added the sentencizer model to the classification package, so all the sentences in the summary texts of the reviews are being disconnected from each other
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp(string)
sentences = list(doc.sents)
sentences
这导致以下列表:
[The Chromebook is exactly what it was advertised to be.,
It is super simple to use.,
The picture quality is great, stays connected to WIfi with no interruption.,
Quick, lightweight yet sturdy.,
I bought the Kindle Fire HD 3G and had so much trouble with battery life, disconnection problems etc.,
that I hate it,
and so I bought the Chromebook and absolutely love it.,
The battery life is good.,
Finally a product that lives up to its hype!]
当我将此列表提供给以下管道时,出现此错误:ValueError: args[0]: Chromebook 正是它所宣传的那样。有错误的格式。应该是 str 类型或 list 类型
#Now in this line the list of reviews are being processed into triplets
from transformers import pipeline
triplet_extractor = pipeline('text2text-generation', model='Babelscape/rebel-large', tokenizer='Babelscape/rebel-large')
model_output = triplet_extractor(sentences, return_tensors=True, return_text=False)
extracted_text = triplet_extractor.tokenizer.batch_decode([x["generated_token_ids"] for x in model_output])
print("\n".join(extracted_text))
因此,有人可以指出我如何将“句子”列表中的所有句子转换为字符串格式吗?
期待回应。 :)
【问题讨论】:
标签: spacy huggingface-transformers