RNN LSTM 估计正弦波频率和相位

Question

为了进一步了解 RNN 和 LSTM，我尝试实现一个简单的 LSTM 来估计正弦波的频率和相位。事实证明，这令人惊讶地难以收敛。 MSE 相当高（以千计） 唯一似乎有点工作的事情是，如果我生成的正弦波都具有相同的相位（同时开始）并且训练样本作为向量传递，而不是像在循环神经网络。同时，这里是不收敛的代码。在这段代码中，我删除了每个频率的不同相位 关于这里出了什么问题的任何想法

我查看了这个Keras : How should I prepare input data for RNN? 并尝试修改我的输入，但没有运气。

from keras.models import Sequential  
from keras.layers.core import Activation, Dropout ,Dense 
from keras.layers.recurrent import GRU, LSTM
import numpy as np
from sklearn.cross_validation import train_test_split

np.random.seed(0)  # For reproducability
TrainingNums = 12000 #Number of Trials
numSampleInEach = 200 #Length of each sinewave
numPhaseExamples = 1 #for each freq, so many different phases

X = np.zeros((TrainingNums,numSampleInEach))
Y = np.zeros((TrainingNums,2))

#create sinewaves here
for iii in range(0, TrainingNums//numPhaseExamples):
    freq = np.round(np.random.randn()*100)
    for kkk in range(0,numPhaseExamples):
    #set timeOffset below to 0, if you want the same phase every run
        timeOffset = 0# 0 for now else np.random.randint(0,90)
        X[iii*numPhaseExamples+kkk,:] = np.sin(2*3.142*freq*np.linspace(0+timeOffset,numSampleInEach-1+timeOffset,numSampleInEach)/10000)
        Y[iii*numPhaseExamples+kkk,0] = freq
        Y[iii*numPhaseExamples+kkk,1] = timeOffset

X = np.reshape(X,(TrainingNums, numSampleInEach,1))
#This below works when there is no phase variation
#X = np.reshape(X,(TrainingNums, numSampleInEach,1))

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.33)

#Now create the RNN
model = Sequential()  
#batch_input_shape = [batch_size,timeStep,dataDimension]
model.add(LSTM(128,input_shape=   (numSampleInEach,1),return_sequences=True))

#For things to work for freq estimation only the following change helps
#model.add(LSTM(128,input_shape=(1,numSampleInEach),return_sequences=True))
model.add(Dropout(0.2))
model.add(Activation("relu")) 

#second layer of RNN
model.add(LSTM(128,return_sequences=False))
model.add(Dropout(0.2))
model.add(Activation("relu")) 

model.add(Dense(2,activation="linear"))
model.compile(loss="mean_squared_error", optimizer="Nadam") 
print model.summary()

print "Model compiled."
model.fit(X_train, y_train, batch_size=16, nb_epoch=150, 
      validation_split=0.1)
result = model.evaluate(X_test, y_test, verbose=0)
print 'mse: ', result

所以问题是：

1. 期望 RNN 估计频率和相位是否正确？
2. 我尝试了几种架构（多层 LSTM、具有更多节点的单层等）。我也尝试过不同的架构。

Answer 1

在LSTM之后去掉激活才是正确的答案