用于 ajax Django 上传的 AWS S3 后端答案

【问题标题】：AWS S3 backend for ajax Django upload用于 ajax Django 上传的 AWS S3 后端
【发布时间】：2014-06-02 09:33:18
【问题描述】：

我正在尝试使用 (django-ajax-uploader) 在我的 DJANGO 应用中实现 ajax 上传。一切似乎都正常，但是当我上传文件时，由于来自 AWS S3 的错误响应导致上传失败并出现 500 错误：

S3ResponseError at /ajax-upload↵S3ResponseError: 400 Bad Request↵<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate...

这是我的后端类：

from ajaxuploader.backends.base import AbstractUploadBackend
class S3Upload(AbstractUploadBackend):
    NUM_PARALLEL_PROCESSES = 4

    def upload_chunk(self, chunk, *args, **kwargs):
        self._counter += 1
        buffer = StringIO()
        buffer.write(chunk)
        self._pool.apply_async(
            self._mp.upload_part_from_file(buffer, self._counter))
        buffer.close()

    def setup(self, filename, *args, **kwargs):
        self._bucket = boto.connect_s3(
            settings.AWS_ACCESS_KEY_ID,
            settings.AWS_SECRET_ACCESS_KEY
        ).lookup(settings.AWS_BUCKET_NAME)

        self._mp = self._bucket.initiate_multipart_upload(filename)
        self._pool = Pool(processes=self.NUM_PARALLEL_PROCESSES)
        self._counter = 0

    def upload_complete(self, request, filename, *args, **kwargs):
        # Tie up loose ends, and finish the upload
        self._pool.close()
        self._pool.join()
        self._mp.complete_upload()

模板（javascript）：

<script src="{% static "ajaxuploader/js/fileuploader.js" %}"></script>

<script>
    $(function(){
    var uploader = new qq.FileUploader({
        action: "{% url "my_ajax_upload" %}",
        element: $('#file-uploader')[0],
        multiple: true,
        onComplete: function(id, fileName, responseJSON) {
            if(responseJSON.success) {
                alert("success!");
            } else {
                alert("upload failed!");
            }
        },
        onAllComplete: function(uploads) {
            // uploads is an array of maps
            // the maps look like this: {file: FileObject, response: JSONServerResponse}
            alert("All complete!");
        },
        params: {
            'csrf_token': '{{ csrf_token }}',
            'csrf_name': 'csrfmiddlewaretoken',
            'csrf_xname': 'X-CSRFToken',
        },
    });
    });
</script>

【问题讨论】：

标签： python ajax django amazon-s3

【解决方案1】：

我使用自定义 s3 后端解决了这个问题，该后端覆盖了上传功能并使用 django-storages 而不是 boto 来保存文件。试试这个：

from ajaxuploader.backends.base import AbstractUploadBackend
from django.core.files.storage import default_storage

class S3CustomUpload(AbstractUploadBackend):
    NUM_PARALLEL_PROCESSES = 4

    def upload_chunk(self, chunk):
        #save file to s3
        self._fd.write(chunk)
        self._fd.close()

    def setup(self, filename):
        self._fd = default_storage.open('%s/%s' % ('uploads/materials/', str(filename)), 'wb')

    def upload(self, uploaded, filename, raw_data, *args, **kwargs):
        try:
            if raw_data:
                # File was uploaded via ajax, and is streaming in.
                chunk = uploaded.read(self.BUFFER_SIZE)
                while len(chunk) > 0:
                    self.upload_chunk(chunk, *args, **kwargs)
                    chunk = uploaded.read(self.BUFFER_SIZE)
            else:
                # File was uploaded via a POST, and is here.
                for chunk in uploaded.chunks():
                    self.upload_chunk(chunk, *args, **kwargs)
            return True
        except:
            # things went badly.
            return False

    def upload_complete(self, request, filename, *args, **kwargs):
        upload = Upload()
        upload.upload = settings.S3_URL + "uploads/materials/"+ filename
        upload.name = filename
        upload.save()

        return {'pk': upload.pk}

【讨论】：

【解决方案2】：

这对我有用：

def upload_chunk(self, chunk, *args, **kwargs):
    self._counter += 1
    buffer = BytesIO(chunk)
    self._pool.apply_async(
        self._mp.upload_part_from_file(buffer, self._counter))
    buffer.close()

【讨论】：