具有分类值的 KNN 无法正确预测

Question

我正在尝试建立一个模型，给定一个项目，预测它属于哪个商店。

我有一个大约 250 条记录的数据集，这些记录应该是不同在线商店中的商品。

每条记录由以下部分组成： Category,Sub Category,Price,Store Identifier(The y variable)

我已经尝试了几个邻居，尝试了曼哈顿距离但不幸的是无法获得更好的结果精度〜0.55。 随机森林产生的准确度约为 0.7。

我的直觉是模型应该能够预测这个问题。我错过了什么？

这是数据： https://pastebin.com/nUsSbkp4

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
from sklearn.neighbors import KNeighborsClassifier
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset = pd.read_csv('data.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, 3].values

labelencoder_X_0 = LabelEncoder()
X[:, 0] = labelencoder_X_0.fit_transform(X[:, 0])

labelencoder_X_1 = LabelEncoder()
X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])

onehotencoder_0 = OneHotEncoder(categorical_features = [0])
X = onehotencoder_0.fit_transform(X).toarray()

onehotencoder_1 = OneHotEncoder(categorical_features = [1])
X = onehotencoder_1.fit_transform(X).toarray()

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0)
# classifier = RandomForestClassifier(n_estimators=25, criterion='entropy', random_state = 0)
classifier = KNeighborsClassifier(n_neighbors=3, metric='minkowski', p=2)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)
cm = confusion_matrix(y_test, y_pred)

accuracy = classifier.score(X_test, y_test) 
print(accuracy)

Answer 1

KNN 可以使用分类预测器产生良好的预测。我之前已经成功了。但是有一些东西没有注意：

• 数字变量必须在相同的范围内，例如通过使用min-max scaling
• 可以尝试专门设计的错误指标，例如Gower distance

除此之外，你实际上在 one-hot-encoding 中有一个错误：

在调用第一个热编码器后，您有一个形状数组 (273, 21)：

onehotencoder_0 = OneHotEncoder(categorical_features = [0])
X = onehotencoder_0.fit_transform(X).toarray()
print(X.shape)
print(X[:5,:])

Out:
(275, 21)
[[ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   52.   33.99]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   52.   33.97]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   36.   27.97]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   37.   13.97]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   20.    9.97]]

然后你在第二列调用一个热编码，它只有两个值（零和一），因此结果：

onehotencoder_1 = OneHotEncoder(categorical_features = [1])
X = onehotencoder_1.fit_transform(X).toarray()
print(X.shape)
print(X[:5,:])

Out:
(275, 22)
[[ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   52.   33.99]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   52.   33.97]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   36.   27.97]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   37.   13.97]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   20.    9.97]]

所以，如果你能解决这个问题，或者简单地使用管道来避免这种情况，并像这样添加数值变量的缩放：

from sklearn.pipeline import Pipeline, FeatureUnion, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.neighbors import KNeighborsClassifier

class Columns(BaseEstimator, TransformerMixin):
    def __init__(self, names=None):
        self.names = names

    def fit(self, X, y=None, **fit_params):
        return self

    def transform(self, X):
        return X.loc[:,self.names]

dataset = pd.read_csv('data.csv', header=None)
dataset.columns = ["cat1", "cat2", "num1", "target"]

X = dataset.iloc[:, :-1]
y = dataset.iloc[:, 3]

labelencoder_X_0 = LabelEncoder()
X.iloc[:, 0] = labelencoder_X_0.fit_transform(X.iloc[:, 0])

labelencoder_X_1 = LabelEncoder()
X.iloc[:, 1] = labelencoder_X_1.fit_transform(X.iloc[:, 1])

numeric = ["num1"]
categorical = ["cat1", "cat2"]

pipe = Pipeline([
    ("features", FeatureUnion([
        ('numeric', make_pipeline(Columns(names=numeric),StandardScaler())),
        ('categorical', make_pipeline(Columns(names=categorical), OneHotEncoder(sparse=False)))
    ])),
])

X = pipe.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0)
# classifier = RandomForestClassifier(n_estimators=25, criterion='entropy', random_state = 0)
classifier = KNeighborsClassifier(n_neighbors=3, metric='minkowski', p=2)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)
cm = confusion_matrix(y_test, y_pred)

accuracy = classifier.score(X_test, y_test) 
print(accuracy)

Out: 
0.7101449275362319

正如您所见，这至少将准确性带入了 Random Forrest 的球场！

因此，您接下来可以尝试的是高尔距离。正在讨论将其添加到 sklearn here，因此可以查看 Ipython Notebook 中发布的代码并尝试。