使用 PyTorch 初始化权重和偏差 - 如何更正尺寸？答案

【问题标题】：Initialising weights and bias with PyTorch - how to correct dimensions?使用 PyTorch 初始化权重和偏差 - 如何更正尺寸？
【发布时间】：2018-12-31 06:14:12
【问题描述】：

使用这个模型，我试图用我预定义的权重和偏差来初始化我的网络：

dimensions_input = 10
hidden_layer_nodes = 5
output_dimension = 10

class Model(torch.nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.linear = torch.nn.Linear(dimensions_input,hidden_layer_nodes)
        self.linear2 = torch.nn.Linear(hidden_layer_nodes,output_dimension)

        self.linear.weight = torch.nn.Parameter(torch.zeros(dimensions_input,hidden_layer_nodes))
        self.linear.bias = torch.nn.Parameter(torch.ones(hidden_layer_nodes))

        self.linear2.weight = torch.nn.Parameter(torch.zeros(dimensions_input,hidden_layer_nodes))
        self.linear2.bias = torch.nn.Parameter(torch.ones(hidden_layer_nodes))

    def forward(self, x):
        l_out1 = self.linear(x)
        y_pred = self.linear2(l_out1)
        return y_pred

model = Model()

criterion = torch.nn.MSELoss(size_average = False)
optim = torch.optim.SGD(model.parameters(), lr = 0.00001)

def train_model():
    y_data = x_data.clone()
    for i in range(10000):
        y_pred = model(x_data)
        loss = criterion(y_pred, y_data)

        if i % 5000 == 0:
            print(loss)
        optim.zero_grad()

        loss.backward()
        optim.step()

运行时错误：

张量 (10) 的扩展大小必须与现有大小 (5) 匹配在非单一维度 1

我的尺寸看起来正确，因为它们与相应的线性图层匹配？

【问题讨论】：

linear2 权重与指定尺寸不匹配

标签： neural-network pytorch

【解决方案1】：

由于x_data 未定义，提供的代码无法运行，所以我不能确定这是问题所在，但让我印象深刻的一件事是您应该替换

self.linear2.weight = torch.nn.Parameter(torch.zeros(dimensions_input,hidden_layer_nodes))
self.linear2.bias = torch.nn.Parameter(torch.ones(hidden_layer_nodes))

与

self.linear2.weight = torch.nn.Parameter(torch.zeros(hidden_layer_nodes, output_dimension))
self.linear2.bias = torch.nn.Parameter(torch.ones(output_dimension))

【讨论】：