使用表单上传 csv 文件 django

Question

我正在处理一个学校管理项目，我想上传一个 CSV 文件并通过覆盖当前数据或在数据库中更新它来保存它。但它没有被添加到数据库中。另外，我喜欢添加更多字段，因此如果可能的话，使用动态（即通过循环），这样我以后就不必更改它了。

models.py

class Student(models.Model):
  registration_number = models.CharField(max_length=200, unique=True)
  firstname = models.CharField(max_length=200)
  surname = models.CharField(max_length=200)
  date_of_birth = models.DateField(default=timezone.now)
  current_class = models.ForeignKey(StudentClass, on_delete=models.SET_NULL, blank=True, null=True)
  date_of_admission = models.DateField(default=timezone.now)
  parent_mobile_number = models.CharField(max_length=15)
  address = models.TextField()

class StudentBulkUpload(models.Model):
  date_uploaded = models.DateTimeField(auto_now=True)
  csv_file = models.FileField(upload_to='students/bulkupload/')

forms.py

class StudentBulkUploadForm(forms.ModelForm):
  class Meta:
    model = StudentBulkUpload
    fields = ("csv_file",)

views.py

def uploadcsv(request):
  if request.method == 'GET':
    form = StudentBulkUploadForm()
    return render(request, 'students/students_upload.html', {'form':form})

  # If not GET method then proceed
  try:
    form = StudentBulkUploadForm(data=request.POST, files=request.FILES)
    if form.is_valid():
      csv_file = form.cleaned_data['csv_file']
    if not csv_file.name.endswith('.csv'):
      messages.error(request, 'File is not CSV type')
      return redirect('students:student-upload')
    # If file is too large
    if csv_file.multiple_chunks():
      messages.error(request, 'Uploaded file is too big (%.2f MB)' %(csv_file.size(1000*1000),))
      return redirect('students:student-upload')
    
    file_data = csv_file.read().decode('utf-8')
    lines = file_data.split('\n')

    # loop over the lines and save them in db. If error, store as string and then display
    for line in lines:
      fields = line.split(',')
      data_dict = {}
      print(data_dict)
      try:
        form = StudentBulkUploadForm(data_dict)
        if form.is_valid():
          form.save()
        else:
          logging.getLogger('error_logger').error(form.errors.as_json())
      except Exception as e:
        logging.getLogger('error_logger').error(form.errors.as_json())
        pass
  except Exception as e:
    logging.getLogger('error_logger').error('Unable to upload file. ' + repr(e))
    messages.error(request, 'Unable to upload file. ' + repr(e))
  return redirect('students:student-upload')

student_upload.html

<form method="POST" enctype="multipart/form-data">
    {% csrf_token %}
    {{ form.as_p }}

    <input type="submit" class="btn btn-success" value="Submit">
</form>

Answer 1

是否有必要通过表单传递数据并将其保存到db。否则，您可以简单地创建模型对象并将字典传递给它并保存。

我认为你应该（简而言之）：

• 首先使用StudentBulkUpload() 上传并保存文件
• 获取文件路径并读取内容
• 如果模型字段和 csv 列的名称相同会​​更好
• 遍历每一行并创建一个字典，其中仅包含迭代中的学生详细信息
• 创建Student() 的实例并将字典传递给它并保存
• 对于外键，使用 get() 从 StudentClass() 获取对象，并相应地使用存储在 csv 中的值

---

我认为您可以通过两种方式保存学生详细信息：

• 创建一个模型对象并通过以正常方式读取每一行来赋值

new_student = Student()
new_student.registration_number = fields[0]
new_student.firstname = fields[1]
# like so for other fields

new_student.save()

• 创建一个模型对象，创建一个键值字典，其中键对应于模型的字段名称。

# create a dictionary `new_student_details` containing values of a student

new_student = Student()
new_student.__dict__.update(new_student_details)
new_student.save()

---

创建一个函数来读取 csv 文件并保存学生详细信息

import csv
def save_new_students_from_csv(file_path):
    # do try catch accordingly
    # open csv file, read lines
    with open(file_path, 'r') as fp:
        students = csv.reader(fp, delimiter=',')
        row = 0
        for student in students:
            if row==0:
                headers = student
                row = row + 1
            else:
                # create a dictionary of student details
                new_student_details = {}
                for i in range(len(headers)):
                    new_student_details[headers[i]] = student[i]

                # for the foreign key field current_class in Student you should get the object first and reassign the value to the key
                new_student_details['current_class'] = StudentClass.objects.get() # get the record according to value which is stored in db and csv file

                # create an instance of Student model
                new_student = Student()
                new_student.__dict__.update(new_student_details)
                new_student.save()
                row = row + 1
        fp.close()

---

您的代码应该如下所示：

def uploadcsv(request):
    if request.method == 'GET':
        form = StudentBulkUploadForm()
        return render(request, 'students/students_upload.html', {'form':form})

    # If not GET method then proceed
    try:
        form = StudentBulkUploadForm(data=request.POST, files=request.FILES)
        if form.is_valid():
            csv_file = form.cleaned_data['csv_file']
            if not csv_file.name.endswith('.csv'):
                messages.error(request, 'File is not CSV type')
                return redirect('students:student-upload')
            # If file is too large
            if csv_file.multiple_chunks():
                messages.error(request, 'Uploaded file is too big (%.2f MB)' %(csv_file.size(1000*1000),))
                return redirect('students:student-upload')

            # save and upload file 
            form.save()

            # get the path of the file saved in the server
            file_path = os.path.join(BASE_DIR, form.csv_file.url)

            # a function to read the file contents and save the student details
            save_new_students_from_csv(file_path)
            # do try catch if necessary
                
    except Exception as e:
        logging.getLogger('error_logger').error('Unable to upload file. ' + repr(e))
        messages.error(request, 'Unable to upload file. ' + repr(e))
    return redirect('students:student-upload')

注意： csv 文件需要遵循正确的格式。你可以随便保存Student()。无论如何，文件需要上传和阅读。必须逐行保存Student()。这只是一个结构。由于我已经删除了您的大部分代码，因此请随意进行必要的更改

Answer 2

在您的代码中，您正在对 CSV 文件进行所有解析和解码，而不是使用已经编写的代码。我建议使用 Django 的CSV import module 来解决这个问题。这样您就可以创建一个自动接收 CSV 数据并将其转换为模型的模型：

from model import CsvModel

class MyCSvModel(CsvModel):
    student_name = CharField()
    foo = IntegerField()

然后，您将像往常一样创建一个可以指向此模型的 django 表单，这样您就可以处理用户上传。

但是，如果这不是一个选项，因为您已经有某个模型想要保持不变（或任何原因），请查看this StackOverflow 问题，该问题解释了如何使用模块 Pandas 读取 csv 并将其转换为数据帧。

希望这有帮助！