【发布时间】:2020-05-13 07:29:50
【问题描述】:
对于当前的一个研究项目,我计划在 Python/Pandas 的基础上,在预先定义的时间范围内读取 JSON 对象“Main_Text”。但是,在计算唯一词时,代码会为行 if word in d: 产生错误 TypeError: unhashable type: 'list' indices must be integers。
我已经通过故障排除线程,除其他外,尝试将事物设置为元组(如某些线程所建议的那样),这已经克服了错误但导致输出为空。是否有任何有用的调整来完成这项工作?
JSON 文件具有以下结构:
[
{"No":"121","Stock Symbol":"A","Date":"05/11/2017","Text Main":"Sample text"}
]
相关的代码摘录如下:
import string
import json
import csv
import pandas as pd
import datetime
import numpy as np
# Loading and reading dataset
file = open("Glassdoor_A.json", "r")
data = json.load(file)
df = pd.json_normalize(data)
df['Date'] = pd.to_datetime(df['Date'])
# Create an empty dictionary
d = dict()
# Filtering by date
start_date = "01/01/2009"
end_date = "01/01/2015"
after_start_date = df["Date"] >= start_date
before_end_date = df["Date"] <= end_date
between_two_dates = after_start_date & before_end_date
filtered_dates = df.loc[between_two_dates]
print(filtered_dates)
# Processing
for row in filtered_dates:
line = list(filtered_dates['Text Main'])
# Remove the leading spaces and newline character
line = [val.strip() for val in line]
# Convert the characters in line to
# lowercase to avoid case mismatch
line = [val.lower() for val in line]
# Remove the punctuation marks from the line
line = [val.translate(val.maketrans("", "", string.punctuation)) for val in line]
# Split the line into words
words = [val.split(" ") for val in line]
# Iterate over each word in line
for word in words:
# Check if the word is already in dictionary
if word in d:
# Increment count of word by 1
d[word] = d[word] + 1
else:
# Add the word to dictionary with count 1
d[word] = 1
【问题讨论】:
标签: python pandas dataframe nlp