将正则化添加到嵌入层时出现警告

Question

当我为我的分类数据添加 l2 正则化到我的嵌入时：

    emb_layer = []
    cat_dim = len(cat_ix)

    X = Input(shape=(cat_dim,))
    X_split = Lambda(lambda x: tf.split(x, cat_dim, 1))(X)
    
    for i in range(len(cat_ix)):
        cardinality = int(df[cat_ix[i]].nunique())
        embed_dim = int(min(np.ceil(cardinality/2),10))

        embedding = Embedding(cardinality + 1, embed_dim, name=cat_ix[i],embeddings_regularizer = l2(1e-4))(X_split[i])
        
        emb_layer.append(embedding)
    
    #Finalizing 
    emb_layer = Concatenate(axis=2)(emb_layer)
    emb_layer = Flatten(name = 'embedding')(emb_layer)

我收到以下警告：

UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory
 "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "

Answer 1

通过像这样添加正则化来修复它：

embedding = Embedding(cardinality + 1, embed_dim, name=cat_ix[i])(X_split[i])
embedding.embeddings_regularizer = l2(1e-4)