方法“export_savedmodel”的参数“serving_input_fn”是什么？答案

【问题标题】：What is the argument `serving_input_fn` of method `export_savedmodel`?方法“export_savedmodel”的参数“serving_input_fn”是什么？
【发布时间】：2019-12-02 15:23:14
【问题描述】：

我正在尝试训练一个 Char RNN 并在训练后导出/保存模型，以便我可以在推理时使用它。这是模型：

def char_rnn_model(features, target):
    """Character level recurrent neural network model to predict classes."""
    target = tf.one_hot(target, 15, 1, 0)
    #byte_list = tf.one_hot(features, 256, 1, 0)
    byte_list = tf.cast(tf.one_hot(features, 256, 1, 0), dtype=tf.float32)
    byte_list = tf.unstack(byte_list, axis=1)

    cell = tf.contrib.rnn.GRUCell(HIDDEN_SIZE)
    _, encoding = tf.contrib.rnn.static_rnn(cell, byte_list, dtype=tf.float32)


    logits = tf.contrib.layers.fully_connected(encoding, 15, activation_fn=None)
    #loss = tf.contrib.losses.softmax_cross_entropy(logits, target)
    loss = tf.contrib.losses.softmax_cross_entropy(logits=logits, onehot_labels=target)

    train_op = tf.contrib.layers.optimize_loss(
      loss,
      tf.contrib.framework.get_global_step(),
      optimizer='Adam',
      learning_rate=0.001)

    return ({
      'class': tf.argmax(logits, 1),
      'prob': tf.nn.softmax(logits)
    }, loss, train_op)

和训练部分：

# train
model_dir = "model"
classifier = learn.Estimator(model_fn=char_rnn_model,model_dir=model_dir)
count=0
n_epoch = 20
while count<n_epoch:
        print("\nEPOCH " + str(count))
        classifier.fit(x_train, y_train, steps=1000,batch_size=10)
        y_predicted = [
              p['class'] for p in classifier.predict(
              x_test, as_iterable=True,batch_size=10)
        ]
        score = metrics.accuracy_score(y_test, y_predicted)
        print('Accuracy: {0:f}'.format(score))
        count+=1

(x_train 是一个形状为 (16639, 100) 的 uint8 数组)

Tensorflow 文档讲述了 export_savedmodel 方法似乎可以满足我的要求。但我不明白第二个论点serving_input_fn。应该是什么？ classifier.export_savedmodel(output_dir, ???)

我正在使用 Tensorflow 1.8.0 和 python 2.7.14。

这与this thread有关。

============ 编辑 ============

我尝试了in this thread建议的两种解决方案：

使用参数model_dir 重新定义一个估计器，该参数与训练期间使用的值相同，希望模型能够自动恢复优化权重：

with tf.Session() as sess: new_saver = tf.train.import_meta_graph(meta_file) new_saver.restore(sess, tf.train.latest_checkpoint(model_dir)) classifier = learn.Estimator(model_fn=char_rnn_model,model_dir=model_dir) new_input = ['Some sequence of characters'] char_processor = learn.preprocessing.ByteProcessor(100) new_input_processed = np.array(list(char_processor.transform(new_input))) print('new_input_processed: ', new_input_processed) p = classifier.predict(new_input_processed, as_iterable=True) predicted_class = p['class'] print('predicted_class: ', predicted_class)

但我收到以下错误

INFO:tensorflow:Restoring parameters from section_header_rnn/section_header_text_only_char_rnn_max_depth=3/model.ckpt-512000
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_keep_checkpoint_max': 5, '_task_type': None, '_train_distribute': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f6794bc3050>, '_model_dir': 'section_header_rnn/section_header_text_only_char_rnn_max_depth=3', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_session_config': None, '_tf_random_seed': None, '_save_summary_steps': 100, '_environment': 'local', '_num_worker_replicas': 0, '_task_id': 0, '_log_step_count_steps': 100, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1.0
}
, '_evaluation_master': '', '_master': ''}
('new_input_processed: ', array([[ 83, 111, 109, 101,  32, 115, 101, 113, 117, 101, 110,  99, 101,
         32, 111, 102,  32,  99, 104,  97, 114,  97,  99, 116, 101, 114,
        115,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
          0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=uint8))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-e069212a21a0> in <module>()
     11     new_input_processed = np.array(list(char_processor.transform(new_input)))
     12     print('new_input_processed: ', new_input_processed)
---> 13     p = classifier.predict(new_input_processed, as_iterable=True)
     14     predicted_class = p['class']
     15     print('predicted_class: ', predicted_class)

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/util/deprecation.pyc in new_func(*args, **kwargs)
    430                 'in a future version' if date is None else ('after %s' % date),
    431                 instructions)
--> 432       return func(*args, **kwargs)
    433     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    434                                        _add_deprecated_arg_notice_to_docstring(

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.pyc in predict(self, x, input_fn, batch_size, outputs, as_iterable, iterate_batches)
    668         outputs=outputs,
    669         as_iterable=as_iterable,
--> 670         iterate_batches=iterate_batches)
    671 
    672   def get_variable_value(self, name):

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.pyc in _infer_model(self, input_fn, feed_fn, outputs, as_iterable, iterate_batches)
    966       training_util.create_global_step(g)
    967       features = self._get_features_from_input_fn(input_fn)
--> 968       infer_ops = self._get_predict_ops(features)
    969       predictions = self._filter_predictions(infer_ops.predictions, outputs)
    970       mon_sess = monitored_session.MonitoredSession(

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.pyc in _get_predict_ops(self, features)
   1312     labels = tensor_signature.create_placeholders_from_signatures(
   1313         self._labels_info)
-> 1314     return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.INFER)
   1315 
   1316   def export_savedmodel(self,

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.pyc in _call_model_fn(self, features, labels, mode, metrics, config)
   1225     if 'model_dir' in model_fn_args:
   1226       kwargs['model_dir'] = self.model_dir
-> 1227     model_fn_results = self._model_fn(features, labels, **kwargs)
   1228 
   1229     if isinstance(model_fn_results, model_fn_lib.ModelFnOps):

<ipython-input-10-55c752822baa> in char_rnn_model(features, target)
      1 def char_rnn_model(features, target):
      2     """Character level recurrent neural network model to predict classes."""
----> 3     target = tf.one_hot(target, 15, 1, 0)
      4     #byte_list = tf.one_hot(features, 256, 1, 0)
      5     byte_list = tf.cast(tf.one_hot(features, 256, 1, 0), dtype=tf.float32)

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.pyc in one_hot(indices, depth, on_value, off_value, axis, dtype, name)
   2500 
   2501     return gen_array_ops.one_hot(indices, depth, on_value, off_value, axis,
-> 2502                                  name)
   2503 
   2504 

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.pyc in one_hot(indices, depth, on_value, off_value, axis, name)
   4364     _, _, _op = _op_def_lib._apply_op_helper(
   4365         "OneHot", indices=indices, depth=depth, on_value=on_value,
-> 4366         off_value=off_value, axis=axis, name=name)
   4367     _result = _op.outputs[:]
   4368     _inputs_flat = _op.inputs

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.pyc in _apply_op_helper(self, op_type_name, name, **keywords)
    526               raise ValueError(
    527                   "Tried to convert '%s' to a tensor and failed. Error: %s" %
--> 528                   (input_name, err))
    529             prefix = ("Input '%s' of '%s' Op has type %s that does not match" %
    530                       (input_name, op_type_name, observed))

ValueError: Tried to convert 'indices' to a tensor and failed. Error: None values not supported.

第二种解决方案是使用the thread的答案中定义的函数serving_input_fn：

def serving_input_fn():

inputs = {'features': tf.placeholder(tf.uint8)}
return tf.estimator.export.ServingInputReceiver(inputs, inputs)

def train_test(x_train, y_train, x_test, y_test, max_depth):

# Build model
model_dir = "model_dir"

classifier = learn.Estimator(model_fn=char_rnn_model,model_dir=model_dir)

count=0
while count<1 #n_epoch:
    print("\nEPOCH " + str(count))
    classifier.fit(x_train, y_train, steps=1000,batch_size=10)
    y_predicted = [
          p['class'] for p in classifier.predict(
          x_test, as_iterable=True,batch_size=10)
    ]
    score = metrics.accuracy_score(y_test, y_predicted)
    print('Accuracy: {0:f}'.format(score))
    count+=1

print("\n More details:")
predicted = [
      p['class'] for p in classifier.predict(
      x_test, as_iterable=True,batch_size=10)
]
print(metrics.classification_report(y_test, predicted))

export_dir = model_dir + "_EXPORT"
classifier.export_savedmodel(export_dir, serving_input_fn)

return f1_score

但我又得到一个错误：

++++++++++
RUN 0
++++++++++
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_keep_checkpoint_max': 5, '_task_type': None, '_train_distribute': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f7af77a40d0>, '_model_dir': 'section_header_text_only_char_rnn_max_depth=3', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_session_config': None, '_tf_random_seed': None, '_save_summary_steps': 100, '_environment': 'local', '_num_worker_replicas': 0, '_task_id': 0, '_log_step_count_steps': 100, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1.0
}
, '_evaluation_master': '', '_master': ''}

EPOCH 0
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from section_header_text_only_char_rnn_max_depth=3/model.ckpt-516000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 516001 into section_header_text_only_char_rnn_max_depth=3/model.ckpt.
INFO:tensorflow:loss = 4.84674e-05, step = 516001
INFO:tensorflow:global_step/sec: 16.343
INFO:tensorflow:loss = 1.6498e-05, step = 516101 (6.118 sec)
INFO:tensorflow:global_step/sec: 50.2856
INFO:tensorflow:loss = 9.97774e-06, step = 516201 (1.989 sec)
INFO:tensorflow:global_step/sec: 51.1929
INFO:tensorflow:loss = 7.67226e-05, step = 516301 (1.953 sec)
INFO:tensorflow:global_step/sec: 52.1178
INFO:tensorflow:loss = 4.43674e-05, step = 516401 (1.919 sec)
INFO:tensorflow:global_step/sec: 51.4657
INFO:tensorflow:loss = 1.10982e-05, step = 516501 (1.943 sec)
INFO:tensorflow:global_step/sec: 51.517
INFO:tensorflow:loss = 2.57473e-05, step = 516601 (1.941 sec)
INFO:tensorflow:global_step/sec: 50.8899
INFO:tensorflow:loss = 3.01228e-05, step = 516701 (1.965 sec)
INFO:tensorflow:global_step/sec: 50.3907
INFO:tensorflow:loss = 3.85045e-06, step = 516801 (1.985 sec)
INFO:tensorflow:global_step/sec: 51.3651
INFO:tensorflow:loss = 1.09075e-05, step = 516901 (1.947 sec)
INFO:tensorflow:Saving checkpoints for 517000 into section_header_text_only_char_rnn_max_depth=3/model.ckpt.
INFO:tensorflow:Loss for final step: 1.77617e-05.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from section_header_text_only_char_rnn_max_depth=3/model.ckpt-517000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
Accuracy: 0.980756

 More details:
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from section_header_text_only_char_rnn_max_depth=3/model.ckpt-517000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
             precision    recall  f1-score   support

          0       0.99      0.99      0.99      6139
          1       0.94      0.95      0.94      1292

avg / total       0.98      0.98      0.98      7431

Confusion Matrix
[[6066   73]
 [  70 1222]]
Done

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-90-d7817a1be627> in <module>()
     65     print("RUN " + str(i_run))
     66     print("++++++++++")
---> 67     f1 = train_test(x_train, y_train, x_test, y_test, max_depth)
     68     f1s.append(f1)
     69 print("\n\n")

<ipython-input-90-d7817a1be627> in train_test(x_train, y_train, x_test, y_test, max_depth)
     53     # export inference graph
     54     export_dir = model_dir + "_EXPORT"
---> 55     classifier.export_savedmodel(export_dir, serving_input_fn)
     56 
     57     return f1_score

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.pyc in export_savedmodel(self, export_dir_base, serving_input_fn, default_output_alternative_key, assets_extra, as_text, checkpoint_path, graph_rewrite_specs, strip_default_attrs)
   1386       input_ops = serving_input_fn()
   1387       input_alternatives, features = (
-> 1388           saved_model_export_utils.get_input_alternatives(input_ops))
   1389 
   1390       # TODO(b/34388557) This is a stopgap, pending recording model provenance.

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/python/util/deprecation.pyc in new_func(*args, **kwargs)
    248               'in a future version' if date is None else ('after %s' % date),
    249               instructions)
--> 250       return func(*args, **kwargs)
    251     return tf_decorator.make_decorator(
    252         func, new_func, 'deprecated',

/home/user/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/utils/saved_model_export_utils.pyc in get_input_alternatives(input_ops)
    171     input_alternatives[DEFAULT_INPUT_ALTERNATIVE_KEY] = default_inputs
    172   else:
--> 173     features, unused_labels = input_ops
    174 
    175   if not features:

ValueError: too many values to unpack

【问题讨论】：

标签： python tensorflow machine-learning inference

【解决方案1】：

尝试查看是否有 TensorFlow 文档中所说的 model.save 方法。否则，您可以只保存权重，然后将它们重新加载到默认模型中。你可以写：

# Save the weights
model.save_weights('./checkpoints/my_checkpoint')

# Create a new model instance
model = create_model()

# Restore the weights
model.load_weights('./checkpoints/my_checkpoint')

编辑

为了快速创建模型，您可以使用 Sequential 类来保存模型或权重（见这里：https://www.tensorflow.org/api_docs/python/tf/keras/Sequential#save）。

Sequential 对象基本上是您使用add(...) 填充的容器。操作将通过您指定的层顺序完成。参见 Keras 文档（tensorflow 的包装器，它使用基本相同的对象），或直接 tnesorflow。

这里我向你展示一个简单的文本处理模型，它由以下组成：

嵌入层（只是一个查找表）
双向 LSTM 层
丢弃层
密集层（维度 = 100）
密集层（维度 = 类数）

请注意，您不必为运行不同的 epoch 指定循环，只需将参数放在 fit 方法中即可。

#Create empty sequential
model = Sequential()

# Add the different layers
model.add(Embedding(VOCAB_SIZE, EMBEDDING_DIM, input_length=MAX_SEQUENCE_LENGTH, weights=[embedding_matrix]))
model.add(Bidirectional(LSTM(512, return_sequences=False)))
model.add(Dropout(0.2))
model.add(Dense(100, activation='sigmoid'))
model.add(Dense(num_classes, activation='softmax'))

# Compile configures the model for training
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit train data
history = model.fit(train_we, np.array(train_lab), validation_split=0.2, epochs=num_epochs, batch_size=batch_size)

# Custom function to plot the loss graph
utils.plot_history(history)

# Make predictions: it return a vector of probability on the given classes
list_prediction_proba = model.predict(test_we)

# Find the most likely class
prediction = [np.where(probabilities == probabilities.max())[0].min() for probabilities in list_prediction_proba]

如果您需要更多信息，请尝试以下操作： https://machinelearningmastery.com/save-load-keras-deep-learning-models/

【讨论】：

感谢您的回答。根据文档（在我的帖子中链接），tf.contrib.learn.Estimator 类中没有save 方法。关于您的建议，create_model() 应该是什么？我从来没有直接用tensorflow创建模型对不起^^。假设我已经管理了模型，tf.contrib.learn.Estimator 类中没有 save_weights()load_weights 方法。
我编辑了答案，希望有用。我恳请您，如果答案是正确的，请将其标记为正确，这样我可以获得一点声誉=）
感谢您的编辑。但是，我想使用与我的帖子中定义的模型完全相同的模型（参见函数char_rnn_model）。我在您的帖子中添加了 +1，这一定会提高您的声誉
好的，谢谢，你的目标是什么？你想实现什么？
目标是能够将经过训练的模型保存在文件中并将它们加载回内存中，以便对新输入进行预测。现在，我看到在我的 EDIT 之后添加了两个解决方案： 1/加载元文件，恢复一个 tensorflow 会话，实例化一个新的 tf.contrib.learn.Estimator 模型，其 model_dir 与用于训练和预测的模型相同；使用此解决方案时，我有一个ValueError（我的帖子中出现完整错误）2/使用export_savedmodel 方法；问题是我不知道如何定义export_savedmodel 的参数serving_input_fn