专家混合 - fminunc 优化过早停止

Question

我有一个混合专家代码，它适用于少量参数。如下：

global x_au;
global x_vi;
global x_alpha;
global y;
global parameter;

options = optimoptions(@fminunc,'GradObj', 'on', 'Algorithm','quasi-newton','MaxIter', 10000,'Display','iter-detailed'); % change number of iterations
optTheta=[];

x_au=x_au_train;
x_vi=x_vi_train;
x_alpha=x_alpha_train;
y=y_train;
parameter=zeros(8969,1);

%expectation step
fprintf('opt1 begins');
opt_1;
fprintf('opt1 complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm1=total_error(parameter(1:4483));
k=1;
count = 1;
while(1)
opt_1; 
fprintf('expectation complete');
%maximaization step
[x] = fminunc(@costfunction,parameter(1:4483),options);
parameter(1:4483)=x;
resnorm2=total_error(parameter(1:4483));
fprintf('resnorm1-resnorm2 - %f, resnorm2 - %f, k - %f',resnorm1-resnorm2,0.000001*resnorm2,k);
if((resnorm1-resnorm2)< .000001*resnorm2 & k~=1)  %% to decrease training time 
    break;
end

但是现在，当我必须在具有大量参数的问题上使用它时，我会得到以下日志。

                                                     First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.
                                                   First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 1.000000 
                                                    First-order 
 Iteration  Func-count       f(x)        Step-size       optimality
     0           1      5.31444e+10                      4.75e+14

Optimization stopped because the objective function cannot be decreased in the 
current search direction. Either the predicted change in the objective function,
or the line search interval is less than eps.

    resnorm1-resnorm2 - 0.000000, resnorm2 - 53144.356560, k - 2.000000
>> 

然后这个过程以非常糟糕的结果结束。正如，可以看出 fminunc 无法正确优化。有人能帮我一下吗？

Answer 1

看起来你需要降低训练速度系数，或者标准化你的参数向量。

Answer 2

我将参数初始化从零更改为 rand，再加上标准化，我让它工作了。