无法初始化我的神经网络 PyTorch 的权重

Question

我无法使用函数 MyNet.apply(init_weights) 初始化权重。

这些是我的功能：

def init_weights(net):
    if type(net) == torch.nn.Module:
        torch.nn.init.kaiming_uniform_(net.weight)
        net.bias.data.fill_(0.01)  # tots els bias a 0.01

我的神经网络如下：

class NeuralNet(torch.nn.Module):
    def __init__(self):
        super().__init__() # Necessary for torch to detect this class as trainable
        # Here define network architecture
        self.layer1 = torch.nn.Linear(28**2, 32).to(device) # Linear layer with 32 neurons
        self.layer2 = torch.nn.Linear(32, 64).to(device) # Linear layer with 64 neurons
        self.layer3 = torch.nn.Linear(64, 128).to(device)  # Linear layer with 128 neurons
        self.output = torch.nn.Linear(128, 1).to(device) # Linear layer with 1 output neuron (binary output)




    def forward(self, x):
        # Here define architecture behavior
        x = torch.sigmoid(self.layer1(x)).to(device) # x = torch.nn.functional.relu(self.layer1(x))
        x = torch.sigmoid(self.layer2(x)).to(device)  
        x = torch.sigmoid(self.layer3(x)).to(device)

        return torch.sigmoid(self.output(x)).to(device) # Binary output

type(net) 打印为线性，因此它永远不会进入 if 语句，如果我删除它会产生以下错误：

AttributeError: 'NeuralNet' 对象没有属性 'weight'

Answer 1

你应该只初始化线性层的权重：

def init_weights(net):
    if type(net) == torch.nn.Linear:
        torch.nn.init.kaiming_uniform_(net.weight)
        net.bias.data.fill_(0.01)  # tots els bias a 0.01