【问题标题】:Cannot parse SQS Json response无法解析 SQS Json 响应
【发布时间】:2022-08-15 02:24:12
【问题描述】:

我创建了一个消费者来收集 SQS 队列中的退回和投诉。它适用于我的大部分消息,但有一些消息会引发异常。

有效的 JSON 输出:

{\"notificationType\":\"Complaint\",\"complaint\":{\"feedbackId\":\"0100018217177664-7d73c160-0be8-49aa-b9c4-bd1d1aebd362-000000\",\"complaintSubType\":null,\"complainedRecipients\":[{\"emailAddress\":\"some-email-addres@yahoo.com\"}],\"timestamp\":\"2022-07-19T15:33:09.000Z\",\"userAgent\":\"Yahoo!-Mail-Feedback/2.0\",\"complaintFeedbackType\":\"abuse\",\"arrivalDate\":\"2022-07-08T12:58:35.000Z\"},\"mail\":{\"timestamp\":\"2022-07-08T12:58:34.633Z\",\"source\":\"noreply@example.com\",\"sourceArn\":\"arn:aws:ses:us-east-1:669689702539:identity/example.com\",\"sourceIp\":\"12.34.56.789\",\"callerIdentity\":\"ses-smtp-user.20190509-104648\",\"sendingAccountId\":\"1234567890\",\"messageId\":\"01000181dde3fb09-77fa66a7-6425-4a17-974c-5b49c8ab930d-000000\",\"destination\":[\"some-email@yahoo.com\"],\"headersTruncated\":false,\"headers\":[{\"name\":\"Received\",\"value\":\"from mail-aws-va-1 ([12.34.56.789]) by email-smtp.amazonaws.com with SMTP (SimpleEmailService-d-9BN0NGUJI) id MfVVhu8nS86Hc4osVk6w for some-email@yahoo.com; Fri, 08 Jul 2022 12:58:34 +0000 (UTC)\"},{\"name\":\"Received\",\"value\":\"by mail-aws-va-1 (Postfix, from userid 111) id E1DA561FD2; Fri,  8 Jul 2022 07:57:50 -0500 (CDT)\"},{\"name\":\"Received\",\"value\":\"from localhost.localdomain (some-server [192.168.20.2]) by mail-aws-va-1 (Postfix) with ESMTP id E5776621E0 for <some-email@yahoo.com>; Fri,  8 Jul 2022 07:57:16 -0500 (CDT)\"},{\"name\":\"Date\",\"value\":\"Fri, 8 Jul 2022 07:57:16 -0500\"},{\"name\":\"To\",\"value\":\"some-email@yahoo.com\"},{\"name\":\"From\",\"value\":\"\\\"Some Gift Shop Inc.\\\" <noreply@example.com>\"},{\"name\":\"Reply-to\",\"value\":\"\\\"Some Gift Shop Inc.\\\" <blahblahblah@some-email.com>\"},{\"name\":\"Subject\",\"value\":\"Some Subject\"},{\"name\":\"Message-ID\",\"value\":\"<9d6ba9128d45561e70b162efc7b6b1b2@localhost.localdomain>\"},{\"name\":\"X-Priority\",\"value\":\"3\"},{\"name\":\"X-Mailer\",\"value\":\"PHPMailer (phpmailer.sourceforge.net) [version 2.0.0 rc1]\"},{\"name\":\"X-Sender\",\"value\":\"ecamp@example.com\"},{\"name\":\"List-Unsubscribe\",\"value\":\"<mailto:abuse+592848.6036692998@flowershopnetwork.com>, <https://www.bedfordflorist.net/ecamp/unsubscribe/5789/592848>\"},{\"name\":\"MIME-Version\",\"value\":\"1.0\"},{\"name\":\"Content-Transfer-Encoding\",\"value\":\"8bit\"},{\"name\":\"Content-Type\",\"value\":\"text/html; charset=\\\"iso-8859-1\\\"\"}],\"commonHeaders\":{\"from\":[\"\\\"Some Gift Shop Inc.\\\" <noreply@example.com>\"],\"replyTo\":[\"\\\"Some Gift Shop Inc.\\\" <blahblahblah@some-email.com>\"],\"date\":\"Fri, 8 Jul 2022 07:57:16 -0500\",\"to\":[\"some-email@yahoo.com\"],\"messageId\":\"<9d6ba9128d45561e70b162efc7b6b1b2@localhost.localdomain>\",\"subject\":\"Some Subject\"}}}

我可以通过以下方式获取我需要的所有对象:

for message in queue.receive_messages(WaitTimeSeconds=10, MaxNumberOfMessages=10):
        try:
            # Grab the json objects
            body = json.loads(message.body)
            headers = body[\'mail\'][\'headers\'][13][\'value\']
            email = body[\'complaint\'][\'complainedRecipients\'][0][\'emailAddress\']
            timestamp = body[\'complaint\'][\'timestamp\']
            formatted_time = datetime.strptime(timestamp, \'%Y-%m-%dT%H:%M:%S.%fZ\').strftime(\"%Y-%m-%d %H:%M:%S\")
            subject = body[\'mail\'][\'commonHeaders\'][\'subject\']
            region = body[\'mail\'][\'sourceArn\'].split(\':\')[3]
            ecamp_id = headers.split(\'|\')[0]
            client_id = headers.split(\'|\')[1]
            ecamp_source = headers.split(\'|\')[2]
        except KeyError as e:
            print(\"Key Error: \" + str(e))
        except IndexError as i:
            print(\"Index Error: \" + str(i))

但是,这个其他 JSON 给我带来了问题:

{
  \"Type\" : \"Notification\",
  \"MessageId\" : \"99acffbc-585e-5f26-a9d0-a4da29d03df7\",
  \"TopicArn\" : \"arn:aws:sns:us-west-2:669689702539:sns-oregon-complaint-topic\",
  \"Message\" : \"{\\\"notificationType\\\":\\\"Complaint\\\",\\\"complaint\\\":{\\\"feedbackId\\\":\\\"010101010101010101010-blahblah\\\",\\\"complaintSubType\\\":null,\\\"complainedRecipients\\\":[{\\\"emailAddress\\\":\\\"PDKLEIN2012@YAHOO.COM\\\"}],\\\"timestamp\\\":\\\"2022-08-05T01:15:58.000Z\\\",\\\"userAgent\\\":\\\"Yahoo!-Mail-Feedback/2.0\\\",\\\"complaintFeedbackType\\\":\\\"abuse\\\",\\\"arrivalDate\\\":\\\"2022-08-05T00:51:34.000Z\\\"},\\\"mail\\\":{\\\"timestamp\\\":\\\"2022-08-05T00:51:32.988Z\\\",\\\"source\\\":\\\"noreply@example.com\\\",\\\"sourceArn\\\":\\\"arn:aws:ses:us-west-2:1234567890:identity/example.com\\\",\\\"sourceIp\\\":\\\"12.34.56.789\\\",\\\"callerIdentity\\\":\\\"mail-aws-or-1\\\",\\\"sendingAccountId\\\":\\\"123456789\\\",\\\"messageId\\\":\\\"010101826b7c6dfc-95c44e97-db49-467b-a65c-e27344874d74-000000\\\",\\\"destination\\\":[\\\"SOME-EMAIL@YAHOO.COM\\\"],\\\"headersTruncated\\\":false,\\\"headers\\\":[{\\\"name\\\":\\\"Received\\\",\\\"value\\\":\\\"from mail-aws-or-1 ([12.34.56.789]) by email-smtp.amazonaws.com with SMTP (SimpleEmailService-d-SPTLPQQAI) id 1uYbytwalUOhARFznQm0 for SOME_EMAIL@YAHOO.COM; Fri, 05 Aug 2022 00:51:32 +0000 (UTC)\\\"},{\\\"name\\\":\\\"Received\\\",\\\"value\\\":\\\"by mail-aws-or-1 (Postfix, from userid 111) id 48D951A294F; Thu,  4 Aug 2022 19:51:06 -0500 (CDT)\\\"},{\\\"name\\\":\\\"Received\\\",\\\"value\\\":\\\"from localhost.localdomain (server_name [192.168.20.2]) by mail-aws-or-1 (Postfix) with ESMTP id AEA191A2932 for <SOME_EMAIL@YAHOO.COM>; Thu,  4 Aug 2022 19:51:04 -0500 (CDT)\\\"},{\\\"name\\\":\\\"Date\\\",\\\"value\\\":\\\"Thu, 4 Aug 2022 19:51:04 -0500\\\"},{\\\"name\\\":\\\"To\\\",\\\"value\\\":\\\"SOME_EMAIL@YAHOO.COM\\\"},{\\\"name\\\":\\\"From\\\",\\\"value\\\":\\\"Some Business Name <noreply@example.com>\\\"},{\\\"name\\\":\\\"Reply-to\\\",\\\"value\\\":\\\"Some Business Name <somebusinessname@somebusinessname.com>\\\"},{\\\"name\\\":\\\"Subject\\\",\\\"value\\\":\\\"Some Subject\'!\\\"},{\\\"name\\\":\\\"Message-ID\\\",\\\"value\\\":\\\"<193c811f09ad915e7857235253a06da0@localhost.localdomain>\\\"},{\\\"name\\\":\\\"X-Priority\\\",\\\"value\\\":\\\"3\\\"},{\\\"name\\\":\\\"X-Mailer\\\",\\\"value\\\":\\\"PHPMailer (phpmailer.sourceforge.net) [version 2.0.0 rc1]\\\"},{\\\"name\\\":\\\"X-Sender\\\",\\\"value\\\":\\\"ecamp@example.com\\\"},{\\\"name\\\":\\\"List-Unsubscribe\\\",\\\"value\\\":\\\"<mailto:abuse+7388957.7173542430@example.com>, <https://www.somebusinessname.com/ecamp/unsubscribe/7331/7388957>\\\"},{\\\"name\\\":\\\"Content-Description\\\",\\\"value\\\":\\\"7331|347977|SomeClassName.class\\\"},{\\\"name\\\":\\\"MIME-Version\\\",\\\"value\\\":\\\"1.0\\\"},{\\\"name\\\":\\\"Content-Transfer-Encoding\\\",\\\"value\\\":\\\"8bit\\\"},{\\\"name\\\":\\\"Content-Type\\\",\\\"value\\\":\\\"text/html; charset=\\\\\\\"iso-8859-1\\\\\\\"\\\"}],\\\"commonHeaders\\\":{\\\"from\\\":[\\\"Petal Perfect Flower Shop <noreply@example.com>\\\"],\\\"replyTo\\\":[\\\"Some Business Name <somebusinessname@somebusinessname.com>\\\"],\\\"date\\\":\\\"Thu, 4 Aug 2022 19:51:04 -0500\\\",\\\"to\\\":[\\\"SOME_EMAIL@YAHOO.COM\\\"],\\\"messageId\\\":\\\"<193c811f09ad915e7857235253a06da0@localhost.localdomain>\\\",\\\"subject\\\":\\\"Some Subject\'!\\\"}}}\",
  \"Timestamp\" : \"2022-08-05T01:15:58.917Z\",
  \"SignatureVersion\" : \"1\",
  \"Signature\" : \"xxsdflkj0983244r098ujsadflkjlkjawe0rjoisajdf09ui234rijlksadf\",
  \"SigningCertURL\" : \"https://sns.us-west-2.amazonaws.com/SimpleNotificationService-213847068461846841354.pem\",
  \"UnsubscribeURL\" : \"https://sns.us-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-west-2:669689702539:sns-oregon-complaint-topic:8c681b37-39c8-40b6-8b8e-16e58ef374c2\"
}

我收到Key Error: \'mail\'

当我使用print(type(body[\'Message\']) 检查类型时,我得到&lt;class \'str\'&gt;。那么,为了正确解析这些其他类型的消息,我需要求助于正则表达式吗?有没有办法将此字符串更改为字典,以便我仍然可以使用 body[\'Message\'] 访问元素?

    标签: python json boto3 amazon-sqs


    【解决方案1】:

    原因是您的第二个响应包含一个包含转义 JSON 的字符串。

    您应该检查可选密钥的可用性。这可以通过多种方式完成,这是一种可能的解决方案:

    body = json.loads(message.body)
    #...
    if 'mail' in body :
        mail = body['mail']
    #...
    

    另请参阅this question mor 详细信息和其他选项。

    要解析转义字符串,您需要再次将此字符串加载为 JSON:

    message = json.loads(body['Message'])
    

    【讨论】:

    • “邮件”确实存在于所有消息中。只是结构不同而已。无论我尝试什么,我都会遇到异常,即TypeError: string indices must be integersKeyError 'mail'
    • 明白了,更新我的答案。
    • 嘿!那行得通!我再次重新加载了body['Message'],然后像以前一样访问了嵌套项目。太感谢了!
    猜你喜欢
    • 2018-06-19
    • 2021-03-12
    • 1970-01-01
    • 2016-01-16
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2022-01-12
    相关资源
    最近更新 更多