最近在youtube 上学习CS231n的课程，并尝试完成Assgnments,收获很多，这里记录下过程和结果以及过程中遇到的问题，我并不是只是完成需要补充的代码段，对于自己不熟悉的没用过的库函数查了官方文档，并做了注释， 每个Assignments 完成后，我会将代码会放到我的GitHub。Assignments 1 is here
在本地完成，需要从这里下载数据，将cifar-10-batches-py文件夹解压到cs231n/datasets 下
Q1-1 k-Nearest Neighbor (kNN) exercise
1.import Lib
2.load data and print data info and show some data
这里有一个enumerate() 函数 ## 用于将一个可遍历的数据对象(如列表、元组或字符串)组合为一个索引序列，同时列出数据和数据下标。我以前用过unique()函数来返回value and index,对这个函数忘记了
3.process data :Reshape the image data into rows
4. realize the KNN and use KNN method to classify
create KNN classifier object
remember that we do not need to anthing for the KNN,,the Classifier simply remembers the data and does no further processing
We would now like to classify the test data with the kNN classifier. Recall that we can break down this process into two steps:
- First we must compute the distances between all test examples and all train examples.
- Given these distances, for each test example we find the k nearest examples and have them vote for the label
here we will use three methods to compute the distance between X_train data and X_test data
- method 1. compute_distance_two_loop
Compute the distance between each test point in X and each training point in self.X_train using a nested loop over both the training data and the test data.
def compute_distances_two_loops(self, X):
"""
Compute the distance between each test point in X and each training point
in self.X_train using a nested loop over both the training data and the
test data.
Inputs:
- X: A numpy array of shape (num_test, D) containing test data.
Returns:
- dists: A numpy array of shape (num_test, num_train) where dists[i, j]
is the Euclidean distance between the ith test point and the jth training
point.
"""
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train))
for i in xrange(num_test):
for j in xrange(num_train):
#####################################################################
# TODO: #
# Compute the l2 distance between the ith test point and the jth #
# training point, and store the result in dists[i, j]. You should #
# not use a loop over dimension. #
#####################################################################
#pass
dists[i,j] = np.sqrt(np.sum(np.square(self.X_train[j,:]-X[i,:])))
#####################################################################
# END OF YOUR CODE #
#####################################################################
return dists
we can visualize the distance matrix: each row is a single test example and its distances to training examples
then we compete the predict_label
def predict_labels(self, dists, k=1):
"""
Given a matrix of distances between test points and training points,
predict a label for each test point.
Inputs:
- dists: A numpy array of shape (num_test, num_train) where dists[i, j]
gives the distance betwen the ith test point and the jth training point.
Returns:
- y: A numpy array of shape (num_test,) containing predicted labels for the
test data, where y[i] is the predicted label for the test point X[i].
"""
num_test = dists.shape[0]
y_pred = np.zeros(num_test)
for i in xrange(num_test):
# A list of length k storing the labels of the k nearest neighbors to
# the ith test point.
closest_y = []
#########################################################################
# TODO: #
# Use the distance matrix to find the k nearest neighbors of the ith #
# testing point, and use self.y_train to find the labels of these #
# neighbors. Store these labels in closest_y. #
# Hint: Look up the function numpy.argsort. #
#########################################################################
#pass
closest_y = self.y_train[np.argsort(dists[i][:k])]
#########################################################################
# TODO: #
# Now that you have found the labels of the k nearest neighbors, you #
# need to find the most common label in the list closest_y of labels. #
# Store this label in y_pred[i]. Break ties by choosing the smaller #
# label. #
#########################################################################
#pass
y_pred[i] = np.argmax(np.bincount(closest_y))
#########################################################################
# END OF YOUR CODE #
#########################################################################
return y_pred
here given a hint: use the numpy.argsort(),I search the document :numpy.argsort(a, axis=-1, kind='quicksort', order=None)[source] Returns the indices that would sort an array.
our first trest setting K =1 and its accuracy is Got 54 / 500 correct => accuracy: 0.108000 ,I do not know why is much lower than the offical answer, I am sure the method is right
the set K =5,it is a little higher than K=1, Got 56 / 500 correct => accuracy: 0.112000
- method 2.compute_distances_one_loop
1 def compute_distances_one_loop(self, X):
2 """
3 Compute the distance between each test point in X and each training point
4 in self.X_train using a single loop over the test data.
5
6 Input / Output: Same as compute_distances_two_loops
7 """
8 num_test = X.shape[0]
9 num_train = self.X_train.shape[0]
10 dists = np.zeros((num_test, num_train))
11 for i in xrange(num_test):
12 #######################################################################
13 # TODO: #
14 # Compute the l2 distance between the ith test point and all training #
15 # points, and store the result in dists[i, :]. #
16 #######################################################################
17 #pass
18 dists[i,:] = np.sqrt(np.sum(np.square(self.X_train-X[i,:]),axis =1))
19 #######################################################################
20 # END OF YOUR CODE #
21 #######################################################################
22 return dists
# To ensure that our vectorized implementation is correct, we make sure that it agrees with the naive implementation. There are many ways to decide whether two matrices are similar; one of the simplest is the Frobenius norm ( if forget the F-norm,reference the Matrix Analisis)
the result is :same Difference was: 0.000000 Good! The distance matrices are the same
- method 3 compute_distances_no_loops (implement the fully vectorized version inside compute_distances_no_loops)
here give a hint : Try to formulate the l2 distance using matrix multiplication and two broadcast sums.
give the code first
1 def compute_distances_no_loops(self, X): 2 """ 3 Compute the distance between each test point in X and each training point 4 in self.X_train using no explicit loops. 5 6 Input / Output: Same as compute_distances_two_loops 7 """ 8 num_test = X.shape[0] 9 num_train = self.X_train.shape[0] 10 dists = np.zeros((num_test, num_train)) 11 ######################################################################### 12 # TODO: # 13 # Compute the l2 distance between all test points and all training # 14 # points without using any explicit loops, and store the result in # 15 # dists. # 16 # # 17 # You should implement this function using only basic array operations; # 18 # in particular you should not use functions from scipy. # 19 # # 20 # HINT: Try to formulate the l2 distance using matrix multiplication # 21 # and two broadcast sums. # 22 ######################################################################### 23 #pass 24 dists = np.multiply(np.dot(X,self.X_train.T),-2) 25 sq1 = np.sum(np.square(X),axis=1,keepdims = True) 26 sq2 = np.sum(np.square(self.X_train),axis =1) 27 dists = np.add(dists,sq1) 28 dists = np.add(dists,sq2) 29 dists = np.sqrt(dists) 30 ######################################################################### 31 # END OF YOUR CODE # 32 ######################################################################### 33 return dists