如何预测训练数据集之外的数据答案

【问题标题】：How to predict data outside of the training data set如何预测训练数据集之外的数据
【发布时间】：2021-03-20 14:27:38
【问题描述】：

使用此模块从地址预测国家名称：

import re
import numpy as np
import pandas as pd
from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import accuracy_score
def normalize_text(s):
    s = s.lower()
    s = re.sub('\s\W',' ',s)
    s = re.sub('\W\s',' ',s)
    s = re.sub('\s+',' ',s)
    return(s)
df['TEXT'] = [normalize_text(s) for s in df['Full_Address']]

vectorizer = CountVectorizer()
x = vectorizer.fit_transform(df['TEXT'])

encoder = LabelEncoder()
y = encoder.fit_transform(df['CountryName'])

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)

nb = MultinomialNB()
nb.fit(x_train, y_train)
y_predicted = nb.predict(x_test)
accuracy_score(y_test, y_predicted)

我想使用我构建的模块来预测单个字符串地址，我该怎么做？我试过了：

nb.predict('1100 112th Ave NE #400, Bellevue, WA 98004, United States')

ValueError: Expected 2D array, got scalar array instead:
array=1100 112th Ave NE #400, Bellevue, WA 98004, United States.
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

更新：

如答案中所建议：

nb.predict([['1100 112th Ave NE #400, Bellevue, WA 98004, United States']])

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 82043 is different from 1)

【问题讨论】：

标签： python pandas scikit-learn naivebayes multinomial

【解决方案1】：

要预测您需要将数据传递给您为训练模型所做的所有预处理步骤：

single_address = '1100 112th Ave NE #400, Bellevue, WA 98004, United States'
normalized_address = normalize_text(single_address)
vectorized_address = vectorizer.transform([normalized_address])
#expected output
nb.predict(vectorized_address)

注意 2 种改进代码的方法：

normalize_text 步骤实际上并不是必需的，因为它所做的所有事情都会被 CountVectorizer 的标记器正则表达式 token_pattern='(?u)\\b\\w\\w+\\b' 和 lowercase=True 捕获
将所有预处理保存在 sklearn Pipeline 中。这样你的代码会更干净，更不容易出错（你肯定会避免像你遇到的错误）

一个有效的 [canonical?] 模板如何实现：

from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline

X = 30*['1100 112th Ave NE #400, Bellevue, WA 98004, United States']
y = 10*['US','France','Germany']

le = LabelEncoder()
y = le.fit_transform(y)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

vectorizer = CountVectorizer()
mnb = MultinomialNB()

ppl = Pipeline(steps=[('vectorizer',vectorizer),('mnb',mnb)])

ppl.fit(X_train, y_train)
single_address = '1100 112th Ave NE #400, Bellevue, WA 98004, United States'
ppl.predict([single_address])

拥有Pipeline 的额外好处是，您可以将其传递给GridSearchCV，以便通过交叉验证选择最佳参数。

【讨论】：

【解决方案2】：

用途：

nb.predict([['1100 112th Ave NE #400, Bellevue, WA 98004, United States']])

predict 方法需要一个数组。

【讨论】：