尝试通过选择特定数据在 Python 中创建 3D 矩阵

Question

您好，我正在尝试在这里制作一个 3D 矩阵。这是 MovieLens 数据 (https://grouplens.org/datasets/movielens/100k/)，我将 u1.base 和 u1.test 对作为训练集和测试集（分别）。下面是您将在代码中发现的变量training_set 的数据格式的图像。

我尝试创建的 3D 矩阵的格式为 (User, Movie, Timestamp)，每个单元格中的数据是用户 1 在时间 1 对电影 1 给出的评分。

如果有帮助，下面是创建二维矩阵的代码，行中的用户和列中的所有电影。

import numpy as np
import pandas as pd

training_set = pd.read_csv('ml-100k/u1.base', delimiter = '\t')
training_set = np.array(training_set, dtype='int')
test_set = pd.read_csv('ml-100k/u1.test', delimiter = '\t')
test_set = np.array(test_set, dtype = 'int64')

nb_users = int(max(max(training_set[:, 0]), max(test_set[:, 0])))
nb_movies = int(max(max(training_set[:, 1]), max(test_set[:, 1]))) 

def convert(data):
    new_data = [] #final list that we will return
    for id_users in range(1, nb_users+1):
        id_movies = data[:, 1][data[:, 0] == id_users] #contains the IDs of the movies rated by the id_user
        id_ratings = data[:, 2][data[:, 0] == id_users] #all movie ratings given by specific user
        ratings = np.zeros(nb_movies)
        ratings[id_movies-1] = id_ratings #these two lines are just so that the movies that are not rated by user have null (0) values
        new_data.append(list(ratings))
    return (new_data)
training_set = convert(training_set)
test_set = convert(test_set)

以下是我尝试过的代码，它给出了许多错误，以至于我无法滚动到它抛出的第一个错误。

import numpy as np
import pandas as pd
training_set = pd.read_csv('ml-100k/u1.base', delimiter = '\t')
training_set = np.array(training_set, dtype='int')
test_set = pd.read_csv('ml-100k/u1.test', delimiter = '\t')
test_set = np.array(test_set, dtype = 'int64')

nb_users = int(max(max(training_set[:, 0]), max(test_set[:, 0])))
nb_movies = int(max(max(training_set[:, 1]), max(test_set[:, 1])))

#The changes I made start here --

nb_timestamps = int(max(len(training_set[:, 3]), len(test_set[:, 3])))

ts_min = int(min(min(training_set[:, 3]), min(test_set[:, 3])))
ts_max = int(max(max(training_set[:, 3]), max(test_set[:, 3])))


def convert(data):
    new_data = [] #final list that we will return
    for timestamp in range(ts_min, ts_max+1):
        for id_users in range(1, nb_users+1):
            id_movies = data[:, 1][data[:, 0] == id_users][data[:, 3] == timestamp]
            #contains the IDs of the movies rated by the id_user
            id_ratings = data[:, 2][data[:, 0] == id_users][data[:, 3] == timestamp]
            ratings = np.zeros(nb_movies)
            ratings[id_movies-1] = id_ratings
        new_data.append(list(ratings))
    return (new_data)
training_set = convert(training_set)
test_set = convert(test_set)

Answer 1

备注：请不要将此作为答案（暂时）。

您的代码有几处需要改进的地方：

• 当您阅读 csv 时，您将第一行作为标题，这意味着您没有考虑所有数据
• 如果在这种情况下（应该如此）只有一个用户只能对一部电影进行一次评分，您可以使用 pd.pivot_table 来获取您的 2D 矩阵。

import pandas as pd
import numpy as np
training_set = pd.read_csv('ml-100k/u1.base',
                           delimiter='\t',
                           header=None, # First row is not header
                           names=["user", "movie",
                                  "rating", "timestamp"]) # rename headers

# with pd.pivot_table you get a df where user are in rows 
# and movies in columns. The value is the rating for movie (i,j)
ratings = pd.pivot_table(training_set,
                         index=["user"],
                         columns=["movie"],
                         values="rating")

如果你想要0s 而不是NaN，你可以使用ratings.fillna(0)。但我不会这样做。你应该小心，因为这会弄乱你想要提取的最终统计数据。

如果您需要二维矩阵，您可以使用ratings.values。

更新

为了获得您的 3D 矩阵，我们可以使用时间戳进行相同的旋转

timestamps = pd.pivot_table(training_set,
                         index=["user"],
                         columns=["movie"],
                         values="timestamp")

# get matrix
mat_ratings = ratings.values
mat_timestamps = timestamps.values

# stack matrix
mat3d = np.dstack((mat_ratings, mat_timestamps))

您现在可以从 2 个形状为 (943, 1650) 的矩阵中检查我们得到一个形状为 (943, 1650, 2) 的矩阵。注意要得到矩阵mat 的形状只需运行mat.shape。