在 AWS Lambda 中解析字典响应答案

【问题标题】：Parsing Dictionary Response in AWS Lambda在 AWS Lambda 中解析字典响应
【发布时间】：2018-12-23 07:42:39
【问题描述】：

我正在尝试创建一个通过 S3 触发器使用 CloudTrail 事件的 AWS Lambda 函数。此功能会在删除 CloudWatch 日志时发出警报。事件：

'eventSource'：'logs.amazonaws.com'

和

'eventName': 'DeleteLogStream'

需要作为同一个事件一起被发现。我的活动中有数据，但我无法捕获和打印它。

import boto3
import gzip
import json

SNS_TOPIC = "<SNS TOPIC ARN>"
SNS_SUBJECT = "<SUBJECT>"


s3_client = boto3.client('s3')
sns_client = boto3.client('sns')


def handler(event, context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key = record['s3']['object']['key']

    
    # Fetch logs from S3
    s3_object = s3_client.get_object(
        Bucket=bucket,
        Key=key,
    )

    # Extract file and metadata from gzipped S3 object
    with gzip.open(s3_object['Body'], 'rb') as binaryObj:
        binaryContent = binaryObj.read()
    
    # Convert from binary data to text
    raw_logs = binaryContent.decode()
    
    # Change text into a dictionary
    dict_logs = json.loads(raw_logs)
    

    # Make sure json_logs key 'Records' exists
    if 'Records' in dict_logs.keys():
    
        print("Printing Dictionary Content: {} \n\n".format(dict_logs))
        
	if dict_logs['Records'][0]['eventSource'] == 'logs.amazonaws.com' and dict_logs['Records'][0]['eventName'] == 'DeleteLogStream':
			print("Found DeleteLogStream event from logs.amazonaws.com!")
		
        # Print Key-Value pair for each item found
        for key, value in dict_logs['Records'][0].items():
            # Account for values that are also dictionaries
            if isinstance(value, dict):
                print("Parent Key: {}".format(key))
                for k, v in value.items():
                    print("Subdict Key: {}".format(k))
                    print("Subdict Value: {}".format(v))
                continue
            else:
                print("Key: {}".format(key))
                print("Value: {}".format(value))

        
        alert_message = "The following log was found: <extracted log contents here>"
        
        # Publish message to SNS topic
        sns_response = sns_client.publish(
            TopicArn=SNS_TOPIC,
            Message=alert_message,
            Subject=SNS_SUBJECT,
            MessageStructure='string',
        )

    else:
        print("Records key not found")

这是我得到的结果： Result from Code

我的代码出于调试目的打印键/值。任何想法为什么“DeleteLogStream”和“logs.amazonaws.com”值没有解析出来？

下面的示例 json 事件： https://raw.githubusercontent.com/danielkowalski1/general-scripts/master/sampleevent

【问题讨论】：

代码的哪一部分试图获取DeleteLogStream和logs.amazonaws.com？
您在事件中收到大量数据。事件中的第一条记录将 eventSource 设为 s3.amazonaws.com，这就是 if dict_logs['Records'][0]['eventSource'] == 'logs.amazonaws.com' 条件失败的原因。如果您使用一些 JSON 解析器解析 JSON，您会注意到 eventSource 从第 5 条记录开始有多个 logs.amazonaws.com。所以你需要做if dict_logs['Records'][4]['eventSource'] == 'logs.amazonaws.com'。
您在 LambdaFunction 中的事件中接收到的数据量非常大。您确定为 Lambda 配置了正确的触发器吗？看起来您正在从您账户的所有 AWS 服务中获取 cloudTrail 事件。
使用jsonpath可以大大简化您的生活。
@9000 刚刚检查过了。看起来棒极了:)

标签： python python-3.x amazon-web-services aws-lambda amazon-cloudtrail

【解决方案1】：

好的，问题已解决。这会遍历整个 Records 列表，然后筛选每个列表值的字典，从而找到所有出现的“DeleteLogStream”。

EVENT_SOURCE = "logs.amazonaws.com"
EVENT_NAME = "DeleteLogStream"     

# Make sure 'Records'key exists
    if 'Records' in dict_logs.keys():
        for item in dict_logs['Records']:

            # Trigger only if a log
            if ('eventSource' in item):
                if (item['eventSource'] == EVENT_SOURCE):
                    if (item['eventName'] == EVENT_NAME):
                        # Grab other useful details for investigation
                        if item['sourceIPAddress']:
                            src_ip = item['sourceIPAddress']
                        if item['userIdentity']['arn']:
                            src_user = item['userIdentity']['arn']

【讨论】：