你如何让亚马逊 SQS 与 Django celery 一起工作

Question

我正在尝试让问题与 Amazon SQS 一起使用，但任务未运行。但是，我的任务用 rabbitmq 开箱即用

这是我尝试过的链接（以及其他）

Celery with Amazon SQS

https://www.caktusgroup.com/blog/2011/12/19/using-django-and-celery-amazon-sqs/

它们似乎已经过时了。根据我的提示，当我登录亚马逊控制台时，我没有收到任何消息，但我看到 celery 日志有活动显示它们正在与亚马逊通信

    DEBUG b'<?xml version="1.0"?><ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><ReceiveMessageResult/><ResponseMetadata><RequestId>f8d10c14-99f7-520b-a58d-5d2cc203ad8b</RequestId></ResponseMetadata></ReceiveMessageResponse>'
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Method: GET
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Path: /967610578225/transfer_files
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Data: 
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Headers: {}
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Host: eu-west-1.queue.amazonaws.com
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Port: 443
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Params: {'Version': '2012-11-05', 'Action': 'ReceiveMessage', 'WaitTimeSeconds': 0, 'MaxNumberOfMessages': 4}
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG Token: None
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log DEBUG CanonicalRequest:
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log  GET
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log  /967610578225/transfer_files
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log  Action=ReceiveMessage&MaxNumberOfMessages=4&Version=2012-11-05&WaitTimeSeconds=0
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log  host:eu-west-1.queue.amazonaws.com
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log  x-amz-date:20170307T065652Z
    Mar 7 08:56:52 ip-172-31-41-253 e-celery-worker.log
, 'Authorization': 'AWS4-HMAC-SHA256 Credential=xxxxxx/20170307/eu-west-1/sqs/aws4_request,SignedHeaders=host;x-amz-date,Signature=xxxxxx', 'User-Agent': 'Boto/2.45.0 Python/3.4.3 Linux/4.1.17-22.30.amzn1.x86_64', 'X-Amz-Date': '20170307T065652Z'}
    Mar 7 08:56:52 ip-172-31-41-253 eright-celery-worker.log DEBUG Response headers: [('Server', 'Server'), ('Date', 'Tue, 07 Mar 2017 06:56:52 GMT'), ('Content-Type', 'text/xml'), ('Content-Length', '240'), ('Connection', 'keep-alive'), ('x-amzn-RequestId', 'bbad9a79-f65e-568d-aaeb-cb41adaa390d')]

我的设置是

BROKER_TRANSPORT = 'sqs'
BROKER_TRANSPORT_OPTIONS = {
    'region': 'eu-west-1',
    'polling_interval': 2,
    'visibility_timeout': 3600,
}
BROKER_USER = 'xxx'
BROKER_PASSWORD = 'xxxxx/xxxx'

CELERY_IMPORTS = (
    'apps.files.tasks',
)

CELERY_QUEUES = {
    'default': {
        'binding_key': 'default'
    },

    # Files
    'files_copy_paste': {
        'binding_key': 'files.copy.paste'
    },
}

任何帮助将不胜感激

Answer 1

我让 Django + Celery 使用以下设置与 SQS 一起工作：

import urllib.parse

# Defined in environment settings
AWS_ACCESS_KEY_ID = os.environ["AWS_ACCESS_KEY_ID"] 
AWS_SECRET_ACCESS_KEY = os.environ["AWS_SECRET_ACCESS_KEY"]

CELERY_BROKER_URL = 'sqs://{0}:{1}@'.format(
    urllib.parse.quote(AWS_ACCESS_KEY_ID, safe=''),
    urllib.parse.quote(AWS_SECRET_ACCESS_KEY, safe='')
)

CELERY_BROKER_TRANSPORT_OPTIONS = {
    'region': 'sqs.eu-west-1',
    'queue_name_prefix': 'celery-'
}

您是否指定了您的AWS_ACCESS_KEY_ID 和AWS_SECRET_ACCESS_KEY？

我还发现我必须在我的地区名称前加上sqs.。