【发布时间】:2021-06-10 14:50:33
【问题描述】:
我编写了一些 python 代码来使用词频列表对来自文本语料库的数据进行矢量化。我的这行代码出现 IndexError: list index out of range 错误;
if words[i+1] in vector:
for c in range(len(vector)):
我正在创建一个从词频列表向量化数据语料库的代码,完整代码如下:
# Process the original data, use a sliding window, convert the original data into vector form, and generate training samples.
def loadData():
# Processing raw data
data1 = open(r"ChinaCorpus.txt", 'r').read()
data1 = data1.replace('[', '')
data1 = data1.replace(']', '')
words = data1.split()
i = 0
while i < len(words): # Remove the previous date factor and reduce its impact on parameter adjustment
if "1998" in words[i]:
del words[i]
i = i - 1
i += 1
lables = []
print(len(words))
for i in range(len(words)):
t = words[i].find('/')
lables.append(words[i][t:])
words[i] = words[i][0:t]
data2 = open(r"WordFreq.txt", "r",encoding= 'UTF-8').read()
vector = data2.split()
with open("VectorData.txt", "a", encoding='utf-8') as f:
for i in range(1,len(words)):
flag = 0
s = ""
if words[i-1] in vector:
for a in range(len(vector)):
if words[i-1] == vector[a]:
s += str(a)
s += ' '
break
else:
s += '0 '
if words[i] in vector:
for b in range(len(vector)):
if words[i] == vector[b]:
s += str(b)
s += ' '
if lables[i] == "/ns":
flag = 1
break
else:
s += '0 '
if words[i+1] in vector:
for c in range(len(vector)):
if words[i+1] == vector[c]:
s += str(c)
s += ' '
break
else:
s += '0 '
if flag == 1:
s += '1'
else:
s += '0'
s += '\n'
print(i)
f.write(s)
f.close()
if __name__ == '__main__':
loadData()
错误截图:
【问题讨论】:
-
words[i+1]在出现words[len(words) + 1]时会报错。尝试改用range(1, len(words) -1) -
就是这样,谢谢!
标签: python