【问题标题】:error type :can't pickle _thread._local objects错误类型:无法腌制 _thread._local 对象
【发布时间】:2019-11-13 10:25:32
【问题描述】:

我已经使用 tensorflow (python) 实现了图像分类机器学习模型,该模型工作正常,现在我必须将模型置于生产阶段,因为我正在使用 sklearn joblib 库,并且还尝试了 pickle 库,但出现错误在这两种情况下

model = Models.Sequential()

model.add(Layers.Conv2D(200,kernel_size=(5,5),activation='relu',input_shape=(150,150,3)))
model.add(Layers.Conv2D(180,kernel_size=(5,5),activation='relu'))
model.add(Layers.MaxPool2D(5,5))

model.add(Layers.Conv2D(50,kernel_size=(5,5),activation='relu'))
model.add(Layers.MaxPool2D(5,5))
model.add(Layers.Flatten())
model.add(Layers.Dense(180,activation='relu'))
model.add(Layers.Dense(100,activation='relu'))
model.add(Layers.Dense(50,activation='relu'))
model.add(Layers.Dropout(rate=0.5))
model.add(Layers.Dense(6,activation='softmax'))

model.compile(optimizer=Optimizer.Adam(lr=0.0001),loss='sparse_categorical_crossentropy',metrics=['accuracy'])

model.summary()

trained = model.fit(Images,Labels,epochs=25,validation_split=0.20)




test_images,test_labels = get_images('C:/Users/shrey/Desktop/img_classification/New folder/seg_test/seg_test/')
test_images = np.array(test_images)
test_labels = np.array(test_labels)
test_images = test_images / 255.0
model.evaluate(test_images,test_labels, verbose=1)

test_images,test_labels = get_images('C:/Users/shrey/Desktop/img_classification/New folder/seg_test/seg_test/')
test_images = np.array(test_images)
test_labels = np.array(test_labels)
test_images = test_images / 255.0
model.evaluate(test_images,test_labels, verbose=1)

#Lets predict the images from the "pred" folder.
In [12]:





​
pred_images,no_labels = get_images('C:/Users/shrey/Desktop/img_classification/New folder/seg_pred/')
#pred_images = tf.image.decode_jpeg(pred_images)
#pred_images = tf.cast(pred_images, tf.float32)                                   
pred_images = np.array(pred_images)
pred_images.shape

from sklearn.externals import joblib

        with open('model_pickle','wb') as f:
             pickle.dump(model,f)

    ---------------------------------------------------------------------------

        Type Error                                 Trackback (most recent call last)
        <ipython-input-43-5da5ca65d688> in <module>
              1 with open('model_pickle','wb') as f:
        ----> 2      pickle.dump(model,f)

        Type Error: can't pickle _thread._local objects

【问题讨论】:

  • 我可以假设您的代码比您显示的要多吗?请分享足够的内容让我们重现该问题。例如,什么是“模型”?
  • 是的,我已经编辑了帖子,现在你可以看到它
  • 改用model.save('model.h5')
  • 是的,它创建了一个文件名“model.h5”,可以建议我如何使用这个“model.h5”文件进行预测

标签: python tensorflow machine-learning data-science pickle


【解决方案1】:

在第一个程序中,我们构建了模型,拟合模型,然后将模型以model.h5 保存到磁盘。在下一个程序中,我正在加载保存的模型model.h5 并使用加载的模型进行预测。您可以从here下载我们在程序中使用的数据集。

构建、拟合和保存模型 -

%tensorflow_version 2.x
print(tf.__version__)
# MLP for Pima Indians Dataset saved to single file
import numpy as np
from numpy import loadtxt
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# load pima indians dataset
dataset = np.loadtxt("/content/pima-indians-diabetes.csv", delimiter=",")

# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# define model
model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

# compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Model Summary
model.summary()

# Fit the model
model.fit(X, Y, epochs=150, batch_size=10, verbose=0)

# evaluate the model
scores = model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

# save model and architecture to single file
model.save("model.h5")
print("Saved model to disk")

输出 -

2.2.0
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 12)                108       
_________________________________________________________________
dense_1 (Dense)              (None, 8)                 104       
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 9         
=================================================================
Total params: 221
Trainable params: 221
Non-trainable params: 0
_________________________________________________________________
accuracy: 76.43%
Saved model to disk

加载模型并用于预测 -

# load and evaluate a saved model
import tensorflow as tf
from numpy import loadtxt
from tensorflow.keras.models import load_model

# load model
model = load_model('model.h5')

# summarize model
model.summary()

# LOAD THE NEW DATASET HERE
dataset = loadtxt("pima-indians-diabetes.csv", delimiter=",")

# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# PREDICT 
score = model.predict(X,verbose=0)
print(score.shape)

输出 -

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 12)                108       
_________________________________________________________________
dense_1 (Dense)              (None, 8)                 104       
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 9         
=================================================================
Total params: 221
Trainable params: 221
Non-trainable params: 0
_________________________________________________________________
(768, 1)

希望这能回答您的问题。快乐学习。

【讨论】:

  • @shrey sawhney - 希望我们已经回答了您的问题。如果您对答案感到满意,请您接受并投票。
猜你喜欢
  • 2022-01-11
  • 2020-04-07
  • 2019-12-14
  • 2020-01-20
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多