QA 模型从多个上下文返回同一问题的最佳答案

Question

我正在尝试使用预训练的 QA 模型构建 QA 系统。

我有一个文本列表，我基本上想使用每个文本提取同一问题的“最佳 x”答案：

例子：

contexts = ['the car is red', 'the car is yellow' , 'I love cats' , 'my car is green', 'the airplane is white'  ....]
question = 'what color is the car?'

到目前为止，我当时可以回答一个文本：

from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch

tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
qa_model = AutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-

word-masking-finetuned-squad")
inputs = tokenizer(question, contexts[0], add_special_tokens=True, return_tensors="pt")
input_ids = inputs["input_ids"].tolist()[0]

outputs = qa_model(**inputs)

answer_start_scores = outputs.start_logits
answer_end_scores = outputs.end_logits

answer_start= torch.argmax(answer_start_scores)
answer_end = torch.argmax(answer_end_scores) +1

answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end]))
answer

然后我可以对每个文本应用一个 for 循环并提取所有答案

但是：1）我不确定这是否是遍历每个上下文的最有效方法 2）我无法将答案从最佳答案到最差答案（即最有可能回答问题和答案的答案）排序这不太可能是对该问题的合理答案）

如何有效地浏览每个上下文以找到答案，并将答案从与问题最连贯的到最不连贯的顺序排列？

Answer 1

您可以尝试将上下文连接到一个字符串，并将它们中的每一个视为单独的句子。 然后从 QA 模型中获得 N 个最佳结果，查看分数并检查给定答案出现的上下文：

from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch
import numpy as np

contexts = ['the car is red', 'the car is yellow' , 'I love cats' , 'my car is green', 'the airplane is white' ]
question = 'what color is the car?'

#combine context and create list of indexes where in joined context next context starts
context = ('. ').join(contexts) + '.'
context_bins = np.cumsum([len(c)+1 for c in contexts])

tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
qa_model = AutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")

inputs = tokenizer(question, context, add_special_tokens=True, return_tensors="pt")
input_ids = inputs["input_ids"].tolist()[0]

outputs = qa_model(**inputs)

# convert scores to probabilities 
answer_start_scores = torch.nn.functional.softmax(outputs.start_logits)
answer_end_scores = torch.nn.functional.softmax(outputs.end_logits)
# or you can use logits
# answer_start_scores = outputs.start_logits
# answer_end_scores = outputs.end_logits

# Extract 5 greatest values fo start and end scores with indeces
answer_start_scores, answers_starts_idx = torch.topk(answer_start_scores, k=5)
answer_end_scores, answers_ends_idx = torch.topk(answer_end_scores, k=5)

print(f'Q: {question}')

for si, ei, ss, es in zip(
    answers_starts_idx[0], 
    answers_ends_idx[0],
    answer_start_scores[0],
    answer_end_scores[0]):

    score = ss*es

    context_idx = [i for i,p in enumerate(context_bins) if p > si][0]
    matching_context = contexts[ context_idx ]

    answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(input_ids[si:ei+1]))
    print(f'Score: {score:<7.7f} A: {answer:<30}; In context {context_idx} : {matching_context}')

输出：

Q: what color is the car?
Score: 0.8851697 A: red                           ; In context 0 : the car is red
Score: 0.0014527 A: yellow                        ; In context 1 : the car is yellow
Score: 0.0000762 A: the car is red.               ; In context 0 : the car is red
Score: 0.0000069 A: green                         ; In context 1 : the car is yellow
Score: 0.0000011 A: car is red. the car is yellow.; In context 0 : the car is red