【发布时间】:2020-06-17 18:49:17
【问题描述】:
在处理英语到印地语时出现错误“IndexError: index 22 is out of bounds for axis 1 with size 22”.LSTM 网络
def generate_batch(X = X_train, y = y_train, batch_size = 128):
''' Generate a batch of data '''
while True:
for j in range(1, len(X), batch_size):
encoder_input_data = np.zeros((batch_size, max_length_src),dtype='float32')
decoder_input_data = np.zeros((batch_size, max_length_tar),dtype='float32')
decoder_target_data = np.zeros((batch_size, max_length_tar, num_decoder_tokens),dtype='float32')
for i, (input_text, target_text) in enumerate(zip(X[j:j+batch_size], y[j:j+batch_size])):
for t, word in enumerate(input_text.split()):
encoder_input_data[i, t] = input_token_index[word] # encoder input seq
for t, word in enumerate(target_text.split()):
if t<len(target_text.split())-1:
decoder_input_data[i, t] = target_token_index[word] # decoder input seq #erro point
if t>0:
# decoder target sequence (one hot encoded)
# does not include the START_ token
# Offset by one timestep
decoder_target_data[i, t - 1, target_token_index[word]] = 1.
yield([encoder_input_data, decoder_input_data], decoder_target_data)
【问题讨论】:
-
最后一个可寻址索引是 21。
标签: python keras encoding nlp decoding