时尚 MNIST 代码将包作为每个真实世界图像的输出

Question

我正在做一个关于时尚服装分类的项目。我想要一个多类分类问题的解决方案。给定真实世界的图像或视频中继，我需要将图像分为 3 类。 服装类型 - T 恤、裤子、套头衫、连衣裙、枕套等。 服装颜色 - 白色、红色、蓝色等 服装的质地/材料 - 棉、羊毛、亚麻、缎子等。

我必须训练自己的模型并找到自己的服装数据库。然后我找到了时尚 MNIST。我不担心找到布料的颜色，但不知道衣服的质地和类型。我必须训练自己的级联分类器。

当然，我在互联网上搜索可能的解决方案。我在 https://www.pyimagesearch.com/2019/02/11/fashion-mnist-with-keras-and-deep-learning/ 找到了 adrian 关于 pyimagesearch 的教程 我用他的代码来训练我的模型。但我将包作为真实世界图像的输出。

他的蒙太奇给出了正确的输出。

Fashion MNIST 的图像给出了正确的输出，但现实世界的图像偏向于包包。也许我需要图像分割和转换为灰度并调整到 28*28 以获得结果？

这是他的代码。

import matplotlib
matplotlib.use("Agg")
from pyimagesearch.minivggnet import MiniVGGNet
from sklearn.metrics import classification_report
from keras.optimizers import SGD
from keras.datasets import fashion_mnist
from keras.utils import np_utils
from keras import backend as K
from imutils import build_montages
import matplotlib.pyplot as plt
import numpy as np
import cv2

# initialize the number of epochs to train for, base learning rate,
# and batch size
NUM_EPOCHS = 25
INIT_LR = 1e-2
BS = 32

# grab the Fashion MNIST dataset (if this is your first time running
# this the dataset will be automatically downloaded)
print("[INFO] loading Fashion MNIST...")
((trainX, trainY), (testX, testY)) = fashion_mnist.load_data()

# if we are using "channels first" ordering, then reshape the design
# matrix such that the matrix is:
#   num_samples x depth x rows x columns
if K.image_data_format() == "channels_first":
    trainX = trainX.reshape((trainX.shape[0], 1, 28, 28))
    testX = testX.reshape((testX.shape[0], 1, 28, 28))

# otherwise, we are using "channels last" ordering, so the design
# matrix shape should be: num_samples x rows x columns x depth
else:
    trainX = trainX.reshape((trainX.shape[0], 28, 28, 1))
    testX = testX.reshape((testX.shape[0], 28, 28, 1))

# scale data to the range of [0, 1]
trainX = trainX.astype("float32") / 255.0
testX = testX.astype("float32") / 255.0

# one-hot encode the training and testing labels
trainY = np_utils.to_categorical(trainY, 10)
testY = np_utils.to_categorical(testY, 10)

# initialize the label names
labelNames = ["top", "trouser", "pullover", "dress", "coat",
    "sandal", "shirt", "sneaker", "bag", "ankle boot"]

# initialize the optimizer and model
print("[INFO] compiling model...")
opt = SGD(lr=INIT_LR, momentum=0.9, decay=INIT_LR / NUM_EPOCHS)
model = MiniVGGNet.build(width=28, height=28, depth=1, classes=10)
model.compile(loss="categorical_crossentropy", optimizer=opt,
    metrics=["accuracy"])

# train the network
print("[INFO] training model...")
H = model.fit(trainX, trainY,
    validation_data=(testX, testY),
    batch_size=BS, epochs=NUM_EPOCHS)

# make predictions on the test set
preds = model.predict(testX)

# show a nicely formatted classification report
print("[INFO] evaluating network...")
print(classification_report(testY.argmax(axis=1), preds.argmax(axis=1),
    target_names=labelNames))

# plot the training loss and accuracy
N = NUM_EPOCHS
plt.style.use("ggplot")
plt.figure()
plt.plot(np.arange(0, N), H.history["loss"], label="train_loss")
plt.plot(np.arange(0, N), H.history["val_loss"], label="val_loss")
plt.plot(np.arange(0, N), H.history["acc"], label="train_acc")
plt.plot(np.arange(0, N), H.history["val_acc"], label="val_acc")
plt.title("Training Loss and Accuracy on Dataset")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend(loc="lower left")
plt.savefig("plot.png")

# initialize our list of output images
images = []

# randomly select a few testing fashion items
for i in np.random.choice(np.arange(0, len(testY)), size=(16,)):
    # classify the clothing
    probs = model.predict(testX[np.newaxis, i])
    prediction = probs.argmax(axis=1)
    label = labelNames[prediction[0]]

    # extract the image from the testData if using "channels_first"
    # ordering
    if K.image_data_format() == "channels_first":
        image = (testX[i][0] * 255).astype("uint8")

    # otherwise we are using "channels_last" ordering
    else:
        image = (testX[i] * 255).astype("uint8")

    # initialize the text label color as green (correct)
    color = (0, 255, 0)

    # otherwise, the class label prediction is incorrect
    if prediction[0] != np.argmax(testY[i]):
        color = (0, 0, 255)

    # merge the channels into one image and resize the image from
    # 28x28 to 96x96 so we can better see it and then draw the
    # predicted label on the image
    image = cv2.merge([image] * 3)
    image = cv2.resize(image, (96, 96), interpolation=cv2.INTER_LINEAR)
    cv2.putText(image, label, (5, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.75,
        color, 2)

    # add the image to our list of output images
    images.append(image)

# construct the montage for the images
montage = build_montages(images, (96, 96), (4, 4))[0]

# show the output montage
cv2.imshow("Fashion MNIST", montage)
cv2.waitKey(0)

每次执行他的代码都需要很长时间。因此，我制作了 2 个文件。 1 保存模型以供其他文件使用。

这是我的代码第 1 部分

import matplotlib
matplotlib.use("Agg")

from pyimagesearch.minivggnet import MiniVGGNet
from sklearn.metrics import classification_report
from keras.optimizers import SGD
from keras.datasets import fashion_mnist
from keras.utils import np_utils
from keras import backend as K
from imutils import build_montages
import matplotlib.pyplot as plt
import numpy as np
import cv2


NUM_EPOCHS = 25
INIT_LR = 1e-2
BS = 32

print("[INFO] loading Fashion MNIST...")
((trainX, trainY), (testX, testY)) = fashion_mnist.load_data()


if K.image_data_format() == "channels_first":
    trainX = trainX.reshape((trainX.shape[0], 1, 28, 28))
    testX = testX.reshape((testX.shape[0], 1, 28, 28))

else:
    trainX = trainX.reshape((trainX.shape[0], 28, 28, 1))
    testX = testX.reshape((testX.shape[0], 28, 28, 1))


trainX = trainX.astype("float32") / 255.0
testX = testX.astype("float32") / 255.0

trainY = np_utils.to_categorical(trainY, 10)
testY = np_utils.to_categorical(testY, 10)

labelNames = ["top", "trouser", "pullover", "dress", "coat",
    "sandal", "shirt", "sneaker", "bag", "ankle boot"]

print("[INFO] compiling model...")
opt = SGD(lr=INIT_LR, momentum=0.9, decay=INIT_LR / NUM_EPOCHS)
model = MiniVGGNet.build(width=28, height=28, depth=1, classes=10)
model.compile(loss="categorical_crossentropy", optimizer=opt,
    metrics=["accuracy"])

print("[INFO] training model...")
H = model.fit(trainX, trainY,
    validation_data=(testX, testY),
    batch_size=BS, epochs=NUM_EPOCHS)

model.save('fashion_mnist_model.h5')

cv2.waitKey(0)
cv2.destroyAllWindows()

第二部分

from keras.models import load_model
from keras.preprocessing import image
import matplotlib.pyplot as plt
import numpy as np
import os
import cv2


if __name__ == "__main__":

    # load model
    model = load_model("fashion_mnist_model.h5")

    labelNames = ["top", "trouser", "pullover", "dress", "coat",
    "sandal", "shirt", "sneaker", "bag", "ankle boot"]

    # image path
    img_path = 'tshirt.jpg'

    # load a single image
    originalimg = cv2.imread(img_path,0)
    img = cv2.resize(originalimg,(28,28))
    img = img.reshape([-1, 28, 28, 1])

    # check prediction
    pred = model.predict(img)
    print(pred)
    prediction_result = np.argmax(pred[0])
    print(prediction_result)

    label = labelNames[prediction_result]
    print(label)

    color = (0, 255, 0)

    cv2.putText(originalimg, label, (5, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.75, color, 2)

    cv2.imshow('result',originalimg)

    cv2.waitKey(0)
    cv2.destroyAllWindows()

预计会输出 tshirt，但将其归类为包。如果需要，将 epoch 更改为 1 或 2 而不是 25，以便快速查看发生了什么。

我看到了另一个关于 tensorflow 的教程。 https://www.tensorflow.org/tutorials/keras/basic_classification 它还提供包作为现实世界图像的输出。也许它不适用于现实世界的图像？

这里是tensorflow的代码

from __future__ import absolute_import, division, print_function, unicode_literals

# TensorFlow and tf.keras
import tensorflow as tf
from tensorflow import keras

# Helper libraries
import numpy as np
import matplotlib.pyplot as plt

print(tf.__version__)

fashion_mnist = keras.datasets.fashion_mnist

(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

train_images.shape

len(train_labels)

train_labels

test_images.shape

len(test_labels)


train_images = train_images / 255.0

test_images = test_images / 255.0


model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.softmax)
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(train_images, train_labels, epochs=5)

import cv2

img = cv2.imread('pant.jpg',0)

img = cv2.resize(img,(28,28))

img = img / 255.0


cv2.imshow('result',img)

img = (np.expand_dims(img,0))

predictions_single = model.predict(img)
print(predictions_single)
prediction_result = np.argmax(predictions_single[0])
print(prediction_result)

cv2.waitKey(0)
cv2.destroyAllWindows()

再次，袋子作为输出而不是裤子。

Answer 1

可能为时已晚。但是，问题在于训练图像和外部图像的差异。在训练图像中，背景表示为0，而在您从网上收集的大多数外部图像中，背景将表示为1s。

Answer 2

您应该在predictions_single = model.predict(img) 之后添加[0]，如下所示：

predictions_single = model.predict(img)[0]

Answer 3

毗湿奴是对的。对于观看此内容的任何人，您只需：

img = 255 - img

为了让图片变成负片。