Keras Lambda 层和变量：“TypeError: can't pickle _thread.lock objects”答案

【问题标题】：Keras Lambda layer and variables : "TypeError: can't pickle _thread.lock objects"Keras Lambda 层和变量：“TypeError: can't pickle _thread.lock objects”
【发布时间】：2018-08-31 16:16:33
【问题描述】：

使用 lambda 层和共享变量时，我无法保存 keras 模型。这是给出此错误的最小代码：

# General imports.
import numpy as np

# Keras for deep learning.
from keras.layers.core import Dense,Lambda
from keras.layers import Input
from keras.models import Model
import keras.backend as K

n_inputs = 20
n_instances = 100

def preprocess(X,minimum,span):
    output = (X - minimum)/span
    return output

inputs = Input(shape=(n_inputs,),name='input_tensor')
maximum = K.max(inputs)
minimum = K.min(inputs)
span = maximum - minimum

x = Lambda(preprocess,arguments={'minimum':minimum,'span':span})(inputs)
x = Dense(units=100,activation='elu')(x)
outputs = Dense(units=n_inputs,activation='elu')(x)

model = Model(inputs=inputs,outputs=outputs)
model.compile(optimizer='adam', loss='mse')

x = np.array([np.random.randn(20) for i in range(n_instances)])
y = np.array([np.random.randn(20) for i in range(n_instances)])

model.fit(x,y,epochs=10)
model.save('test.h5')       # This line doesn't work.

这是我得到的完整错误：

Traceback (most recent call last):
  File "C:\Dropbox\HELMo_Gramme\M2\Stage\Programmation\Filter\sanstitre0.py", line 35, in <module>
    model.save('test.h5')       # This line doesn't work.
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2580, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\models.py", line 111, in save_model
    'config': model.get_config()
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2421, in get_config
    return copy.deepcopy(config)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle _thread.lock objects

模型可以训练，可以用来预测，这个问题只在保存的时候出现。对于使用 lambda 层的人，我已经看到类似的错误（例如this link），但我认为我的问题略有不同（大多数时候它使用的是我不使用的 seq2seq.py）。但是，它似乎仍然与深拷贝有关。

如果我删除 lambda 层或外部变量，它会起作用。我可能正在做一些我不应该对变量做的事情，但我不知道如何正确地做到这一点。我需要它们超出预处理函数的范围，因为我在后处理函数中使用了相同的变量。

我了解模型内部的预处理不是最有效的，但我有理由这样做，并且此数据集的性能不是问题。

更新

我忘了说明我希望能够在另一个 Lambda 层中重用 maximum、minimum 和 span，这就是为什么它们被定义在 preprocess 范围之外的原因。

更新 2

maxim 的解决方案确实有帮助，但在我的实际代码中仍然不起作用。不同之处在于，我实际上是在函数中创建模型并返回它，但不知何故，这会返回相同类型的错误。

示例代码：

# General imports.
import numpy as np

# Keras for deep learning.
from keras.layers.core import Dense,Lambda
from keras.layers import Input
from keras.models import Model
import keras.backend as K

n_inputs = 101
n_instances = 100

def create_model(n_inputs):

    def preprocess(X):
        maximum = K.max(inputs)
        minimum = K.min(inputs)
        span = maximum - minimum
        output = (X - minimum)/span
        return output

    def postprocess(X):
        maximum = K.max(inputs)
        minimum = K.min(inputs)
        span = maximum - minimum
        output = X*span + minimum
        return output

    inputs = Input(shape=(n_inputs,),name='input_tensor')

    x = Lambda(preprocess)(inputs)
    x = Dense(units=100,activation='elu')(x)
    outputs = Dense(units=n_inputs,activation='elu')(x)
    outputs = Lambda(postprocess)(outputs)

    model = Model(inputs=inputs,outputs=outputs)
    model.compile(optimizer='adam', loss='mse')
    return model

x = np.array([np.random.randn(n_inputs) for i in range(n_instances)])
y = np.array([np.random.randn(n_inputs) for i in range(n_instances)])

model = create_model(n_inputs)
model.fit(x,y,epochs=10)
model.save('test.h5')       # This line doesn't work.

错误：

Traceback (most recent call last):
  File "C:\Dropbox\HELMo_Gramme\M2\Stage\Programmation\Filter\sanstitre0.py", line 46, in <module>
    model.save('test.h5')       # This line doesn't work.
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2580, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\models.py", line 111, in save_model
    'config': model.get_config()
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\topology.py", line 2421, in get_config
    return copy.deepcopy(config)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\ProgramData\Anaconda3\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle _thread.lock objects

我需要在函数中创建我的模型，因为我正在优化超参数，因此我使用不同的参数集迭代不同的模型（我可能可以在循环中完成，但它不是不错）。

【问题讨论】：

标签： python tensorflow machine-learning lambda keras

【解决方案1】：

问题在于 lambda 参数：minimum 和 span。它们是从输入推导出来的，但是当您像这样定义 lambda 层时：

x = Lambda(preprocess,arguments={'minimum':minimum,'span':span})(inputs)

...它们被视为需要序列化的独立参数（作为 lambda 的上下文）。这会导致错误，因为它们都是 tensorflow 张量，而不是静态值或 numpy 数组。

将您的代码更改为：

# `preprocess` encapsulates all intermediate values in itself.
def preprocess(X):
  maximum = K.max(X)
  minimum = K.min(X)
  span = maximum - minimum
  output = (X - minimum) / span
  return output

inputs = Input(shape=(n_inputs,), name='input_tensor')
x = Lambda(preprocess)(inputs)

【讨论】：

它确实有效，但我应该澄清一下，我希望能够在第二个 lambda 函数中重用 maximum、minimum 和 span，而无需重新计算它们（这就是为什么我在预处理范围之外定义它们）。有没有办法做到这一点？
@Flabou 在这种情况下，它们的计算成本相当低，所以我不会打扰。
同意，但知道什么是好的做法仍然很有趣。
Lambdas 被设计成独立的，所以如果我需要更复杂的东西，我会使用自定义层 - keras.io/layers/writing-your-own-keras-layers
我在实际代码中遇到了第二个错误，因为模型是在函数内部创建的（我已经针对这个问题更新了我的问题）。如果您有任何想法，我将不胜感激！

【解决方案2】：

我遇到了同样的问题（Lambda 层 + keras@2.1.5 上带有 tensorflow@1.11.0 后端的多个参数）和 model.save_weights(...) 而不是 model.save(...) 有效 - 如果您只是想稍后加载训练过的权重并且你不需要存储架构。

【讨论】：

【解决方案3】：

在similar question here on Stackoverflow 上查看我的回答。

针对您的特定情况，更改以下几行：

(...)
x = Lambda(preprocess)(inputs)
(...)
outputs = Lambda(postprocess)(outputs)
(...)

这些行：

(...)
x = Lambda(lambda t: preprocess(t))(inputs)
(...)
outputs = Lambda(lambda t: postprocess(t))(outputs)
(...)

【讨论】：