Boto3 和 DynamoDB - 如何模拟聚合

Question

我在 DynamoDB 中有一个表格，格式如下：

DeviceId (PK)	SensorDataType	SensorValue	CurrentTime (SK)
BSMD002	HeartRate	86	2021-03-13 14:50:17.292663
BSMD002	HeartRate	106	2021-03-13 14:50:17.564644
BSMD002	HeartRate	97	2021-03-13 14:50:17.854391

我正在使用 boto3 从该表中提取数据，并希望创建一个基于用户输入的新表（DeviceId、日期范围）。该表将包含传感器类型数据 - 按分钟分组的最大值、最小值和平均值。

我知道 DynamoDB 不支持聚合，使用 Streams + Lambda 是更有效的方法。但是想了解在boto3中是否有办法做到这一点。到目前为止，已经按照以下代码提取数据。

import boto3
import time
from datetime import datetime, timedelta
from boto3.dynamodb.conditions import Key, Attr

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('BSMDataTable')

devicetag = input(" Enter the Device ID to find: ").upper()
datefrom = input("Enter Starting Date in YYYY-MM-DD format: ")
dateto = input("Enter Ending Date in YYYY-MM-DD format: ")

fe = Key('CurrentTime').between(datefrom,dateto) & Key('DeviceId').eq(devicetag);
response = table.query(
                KeyConditionExpression=fe
            )
for i in response['Items']:
    print(i)

Answer 1

其实你们很亲近。缺少的只是响应中项目的聚合。

这是一个例子。 我们首先按分钟对项目进行分组，然后计算每分钟的统计数据。

import statistics
import itertools

# Sample data
response = {
    "Items": [
        {"DeviceId": "BSMD002", "SensorDataType": "HeartRate", "SensorValue": 86, "CurrentTime": "2021-03-13 14:50:17.123"},
        {"DeviceId": "BSMD002", "SensorDataType": "HeartRate", "SensorValue": 100, "CurrentTime": "2021-03-13 14:50:18.123"},
        {"DeviceId": "BSMD002", "SensorDataType": "HeartRate", "SensorValue": 19, "CurrentTime": "2021-03-13 14:51:17.123"},
    ]
}

# Group the response by items per minute
items_by_minute = itertools.groupby(
    response["Items"], 
    key=lambda x: x["CurrentTime"][:16]  #  The first 16 characters including the minute
)

# Calculate the statistics for each minute
for minute, items in items_by_minute:
    values_per_minute = [item["SensorValue"] for item in items]
    
    avg = statistics.mean(values_per_minute)
    min_value = min(values_per_minute)
    max_value = max(values_per_minute)
    
    print(f"Minute: {minute} / Average {avg} / Min {min_value} / Max {max_value}")

输出

Minute: 2021-03-13 14:50 / Average 93 / Min 86 / Max 100
Minute: 2021-03-13 14:51 / Average 19 / Min 19 / Max 19