我如何将 .pkl.gz 文件从我的计算机导入 python 程序

Question

我有自己的数据集，我想在上面训练我的模型。我已经成功创建了 .pk.gz 文件，但我不知道如何将它们导入到我的模型中。

我正在使用带有 tensor-flow 和 tflearn 和 sublime text 3 的 windows 10、python 3.5.2 来编写代码。

我用来创建pickle文件的代码：

from numpy import genfromtxt

import gzip
import _pickle as cPickle

#data = sio.loadmat('C:/DeepLearning_lib/Theano/Data/test_x.mat')

train_set_x = genfromtxt('C:/Users/Jay/Desktop/MachineLearning/dataset/NSL-KDD Processed/Kdd_Train_41.csv', delimiter=',')

train_set_y = genfromtxt('C:/Users/Jay/Desktop/MachineLearning/dataset/NSL-KDD Processed/NSL_TrainLabels_mat4.csv', delimiter=',')

valid_set_x = genfromtxt('C:/Users/Jay/Desktop/MachineLearning/dataset/NSL-KDD Processed/Kdd_Valid_41.csv', delimiter=',')

valid_set_y = genfromtxt('C:/Users/Jay/Desktop/MachineLearning/dataset/NSL-KDD Processed/NSL_ValidLabels_int2.csv', delimiter=',')

test_set_x = genfromtxt('C:/Users/Jay/Desktop/MachineLearning/dataset/NSL-KDD Processed/Kdd_Test_41.csv', delimiter=',')

test_set_y = genfromtxt('C:/Users/Jay/Desktop/MachineLearning/dataset/NSL-KDD Processed/NSL_TestLabels_mat5.csv', delimiter=',')



train_set = test_set_x
train_set_labels= test_set_y

valid_set = valid_set_x
valid_set_labels= valid_set_y

test_set = train_set_x
test_set_labels= train_set_y


f = gzip.open('C:/Users/Jay/Desktop/Data/train_set.pkl.gz','wb')
cPickle.dump(train_set, f, protocol=2)

f.close()

f = gzip.open('C:/Users/Jay/Desktop/Data/train_set_labels.pkl.gz','wb')
cPickle.dump(train_set_labels, f, protocol=2)

f.close()

f = gzip.open('C:/Users/Jay/Desktop/Data/valid_set_labels.pkl.gz','wb')
cPickle.dump(valid_set_labels, f, protocol=2)

f.close()
f = gzip.open('C:/Users/Jay/Desktop/Data/test_set_labels.pkl.gz','wb')
cPickle.dump(test_set_labels, f, protocol=2)

f.close()
f = gzip.open('C:/Users/Jay/Desktop/Data/valid_set.pkl.gz','wb')
cPickle.dump(valid_set, f, protocol=2)

f.close()
f = gzip.open('C:/Users/Jay/Desktop/Data/test_set.pkl.gz','wb')
cPickle.dump(test_set, f, protocol=2)

f.close()

错误：当使用'rb'时

'OSError: [Errno 9] peek() on write-only GzipFile object'

Answer 1

下面的代码应该重构你的train_set：

with gzip.open('C:/Users/Jay/Desktop/Data/train_set.pkl.gz', 'rb') as f:
  train_set = cPickle.load(f)