MIDIutil 通过 Flask 将 MIDI 文件写入 boto3 API 服务器

Question

努力尝试在连接到 s3 服务器的 Flask 应用程序中编写 MIDIUtil 文件。

在本地实例中，这并不费力：

LOCAL_UPLOAD_FOLDER = './_static/uploads/MIDI_files/'
file_name = "NAME.mid"
file_path = f'{LOCAL_UPLOAD_FOLDER}{file_name}'
MyMIDI = MIDIFile(1)

with open(file_path, "wb") as output_file:
     MyMIDI.writeFile(output_file)

但是，我不确定如何将其应用于 s3 资源，这是我的实例化...

def get_upload_folder(UPLOAD_FOLDER=None, UPLOAD_FOLDER_KEY=None,
                      client_resource=None, client=None):
    """ Determines How to Upload / Send File for Download """
    # Flask Cloud Settings - Upload Folder
    if os.getenv('CONTEXT') == 'CLOUD':

        # Client Side
        UPLOAD_FOLDER_TYPE = 'CLOUD'
        session = boto3.session.Session()
        client = session.client(
            's3', endpoint_url=os.getenv('ENDPOINT_URL'),
            config=botocore.config.Config(s3={'addressing_style': 'virtual'}),
            region_name=os.getenv('REGION_NAME'), aws_access_key_id=os.getenv('SECRET_ID'),
            aws_secret_access_key=os.getenv('SECRET_KEY')
        )

        # Resource Side
        client_resource = boto3.resource(
            's3', endpoint_url='https://nyc3.digitaloceanspaces.com',
            config=botocore.config.Config(s3={'addressing_style': 'virtual'}),
            region_name='nyc3', aws_access_key_id=os.getenv('SECRET_ID'),
            aws_secret_access_key=os.getenv('SECRET_KEY')
        )

    UPLOAD_FOLDER, UPLOAD_FOLDER_KEY = 'MY_BUCKET', 'uploads/MIDI_files/'

   return UPLOAD_FOLDER_TYPE, UPLOAD_FOLDER, UPLOAD_FOLDER_KEY, client_resource, client

到目前为止，我已经尝试过：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)
    client.download_fileobj(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

以及大量其他 .put_object 与 client 和 client_resource boto3 对象的组合......

我在想我的问题在于：

• MIDIUtil.Midifile 的writeFile(filehandler)

也许这个函数在我将put_object 转换为 s3 BODY= 之前关闭了 MIDI 二进制流数据？也许我需要通过字节（IO）/流对象来解析二进制数据..？

或者

• 尝试使用我的 s3 对象实现可写目录。

也许我可以更好地分配 s3 UPLOAD_FOLDER...我只是不确定如何在 FLASK 中建立这种连接...

app.config['UPLOAD_FOLDER'] = client.Object(
    Bucket=UPLOAD_FOLDER, Key=UPLOAD_FOLDER_KEY,
    ACL='private'
)

任何帮助表示赞赏！感觉我可能已经用这种方法更接近了...... 它确实写入了 s3 Bucket，所以我可能会担心获取可用的 URL，但是 MIDI 文件已损坏并且空白 =(

file_path = f'{UPLOAD_FOLDER_KEY}{file_name}'
            response = client.generate_presigned_post(UPLOAD_FOLDER,
                                                      file_name,
                                                      ExpiresIn=3600)
            post_url = response['url']
            data = response['fields']
            key = data['key']
            with open(file_name, 'wb') as f:
                http_response = requests.post(url=post_url, data=data,
                                              files={file_name: MyMIDI.writeFile(f)})

print(response) 产生：

{'url': 'ENDPOINT_URL', 'fields': {'key': 'files(from above)', 'x-amz-algorithm': 'STUFF', 'x-amz-credential': 'STUFF', 'x-amz-date': 'STUFF', 'policy': 'STUFF', 'x-amz-signature': 'STUFF'}}```

如果我可以从中提取一个 URL 以重定向到... 试图解散这个 Article on S3 File Uploads 以获得答案。

Answer 1

此代码将无法按预期工作：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)
    client.upload_file(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

此代码的问题在于您正在使用上下文管理器 (with)，并且上下文管理器在您退出上下文管理器之前不会关闭输出文件。因此，在您尝试将文件上传到 S3 时，文件内容不会刷新到磁盘。

需要这样写：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)

client.upload_file(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

Answer 2

我的初步解决方案：我不能完全解释为什么；但我认为这个问题伴随着MyMIDI.writefile(filehandler) 和boto3 函数之间的交互而出现。我认为它在 .write() 和 .close() 内与 MIDIUtil 包的 .writefile() 嵌套有关，同时必须同时为 s3's Body 参数生成字节数据。所以这是我的解决方法......

# Working Version on S3 Deployment
# Assign generate_presigned_post to variable 
response = client.generate_presigned_post(UPLOAD_FOLDER,
                                          file_name,
                                          ExpiresIn=3600)

# Have MIDIUtil write / close the file within writefile       
with open(file_name, 'wb') as file:
    MIDI_FILE = MyMIDI.writeFile(file)

# Read the written binary contents for the s3 Body; assign to a variable (content)
f = open(file_name, 'rb')  
content = f.read()
       
# Stage the object with its file_name and s3 Bucket(UPLOAD_FOLDER)
MIDI_Object = client_resource.Object(UPLOAD_FOLDER, file_name)

# Write to the s3 Bucket using put
MIDI_Object.put(Body=content)

瞧！它在我的 S3 存储桶中不再是空白的，可以下载！！ :D

更好的解决方案：（感谢@jarmod）更少的代码行和做同样的事情:)

with open(MIDI_file_name, 'wb') as file:
    MIDI_FILE = MIDI_DATA.writeFile(file)

object_name = os.path.basename(file_name)
client.upload_file(MIDI_file_name, UPLOAD_FOLDER, object_name)