如何确定 Azure EventGrid 消息传递失败的原因？答案

【问题标题】：How to determine cause of Azure EventGrid message delivery failure?如何确定 Azure EventGrid 消息传递失败的原因？
【发布时间】：2021-08-15 01:32:46
【问题描述】：

如何确定 EventGrid 消息传递失败的原因？

目前尝试过：

在 EventGrid 系统主题上设置Diagnostic Setting
打开 Log Analytics 并使用以下命令查询 AEGDeliveryFailureLogs 表

//==================================================//
// Assign variables
//==================================================//
let varStart = ago(5h);
let varEnd = now();
//==================================================//
// Query table
//==================================================//
AegDeliveryFailureLogs
| where TimeGenerated between (varStart .. varEnd)
| project split(Message, ',')

在我的案例中产生了 1700 多条记录
示例结果：

[
"outcome=Aborted",
" latencyInMs=180004",
" id=<redacted>",
" outputEventSystemId=<redacted>",
" state=Filtered",
" deliveryTime=5/26/2021 3:22:01 PM",
" deliveryCount=3",
" probationCount=1",
" deliverySchema=EventGridEvent",
" eventSubscriptionDeliverySchema=EventGridEvent",
" outputEventFields=InputEvent| EventSubscriptionId| DeliveryTime| DeliveryCount| State| Id| ProbationCount| LastDeliveryOutcome| DeliverySchema| LastDeliveryAttemptTime| SystemId| UseMappedResourceArmIdForBilling",
" outputEventFieldCount=12",
" requestExpiration=5/26/2021 3:23:01 PM",
" delivered=False id=<redacted>",
" inputEventSystemId=<redacted> publishTime=5/26/2021 3:14:42 PM",
" eventTime=5/26/2021 3:14:42 PM",
" eventType=Microsoft.Storage.BlobCreated",
" deliveryTime=1/1/0001 12:00:00 AM",
" filteringState=FilteringPending",
" inputSchema=EventGridEvent",
" publisher=MICROSOFT-STORAGE-STORAGEACCOUNTS.<redacted>.EVENTGRID.AZURE.NET",
" size=1162",
" subject=/blobServices/default/containers/<redacted>/blobs/2021-05-25/<redacted>/<redacted>.csv",
" inputEventFields=Id| PublishTime| SerializedBody| EventType| Topic| Subject| FilteringHashCode| SystemId| Publisher| FilteringTopic| TopicCategory| DataVersion| MetadataVersion| InputSchema| EventTime| FilteringPolicy",
" inputEventFieldCount=16",
" type=AzureFunction",
" subType=NotApplicable",
" supportsBatching=False",
" aadIntegration=False",
" useSystemIdentity=False",
"armId=/subscriptions/<redacted>/resourceGroups/<redacted>/providers/Microsoft.Web/sites/<redacted>/functions/<redacted>",
" deliveryResponse=",
" httpStatusCode=InternalServerError",
" errorType=UnexpectedError",
" errorMessage=An unexpected error has occurred. Please report the x-ms-request-id header value to our forums for assistance or raise a support ticket.",
" errorMessage=The operation timed out after: 00:03:00"
]

有一个明确的outcome=Aborted 和一个httpStatusCode/errorType/errorMessage 表明存在问题，但这就是调查停止的地方。

是什么导致了这些 EventGrid 消息问题，我该如何确定原因？

【问题讨论】：

看起来，AEG 在 3 分钟内没有收到来自投递端点（azure 函数）的消息投递状态。
您能否描述一下 1.) 您是如何从上述消息中看出这一点的，以及 2.) 在正常工作的 Azure 函数中，由于什么原因可能会在 5 小时内发生 1700 倍？
我推荐阅读docs.microsoft.com/en-us/azure/event-grid/…
哦，我已阅读所有文档。 InternalServerError 的 httpStatusCode 不在该文档中。
您的示例显示，事件消息已尝试传递 3 次，并且根据重试策略，传递过程已中止。在启用死信的情况下，消息将存储（5 分钟后）在死信存储中，并附有传递失败原因的详细信息。您的订阅者在 azure function 高级计划中运行，所以看看它是如何横向扩展的等，另一种选择是将事件消息传递到队列事件处理程序资源，然后通过 azure 函数将其拉出触发。

标签： azure azure-log-analytics kql azure-eventgrid

【解决方案1】：

似乎确定为什么事件网格消息未能传递的唯一方法是打开死信。

这是一个示例死信消息。

它清楚地显示了deadLetterReason 和一些其他有价值的调试信息：

[
    {
        "id": "redacted",
        "eventTime": "2021-06-07T22:52:42.7156553Z",
        "eventType": "Microsoft.Storage.BlobCreated",
        "dataVersion": "",
        "metadataVersion": "1",
        "topic": "/subscriptions/redacted/resourceGroups/redacted/providers/Microsoft.Storage/storageAccounts/redacted",
        "subject": "/blobServices/default/containers/redacted/blobs/20210525/redacted.csv",
        "deadLetterReason": "TimeToLiveExceeded",
        "deliveryAttempts": 12,
        "lastDeliveryOutcome": "TimedOut",
        "publishTime": "2021-06-07T22:52:43.1497044Z",
        "lastDeliveryAttemptTime": "2021-06-08T21:50:11.3677712Z",
        "data": {
            "api": "PutBlockList",
            "clientRequestId": "redacted",
            "requestId": "redacted",
            "eTag": "0x8D92A06F5883BA6",
            "contentType": "application/octet-stream",
            "contentLength": 4767672448,
            "blobType": "BlockBlob",
            "blobUrl": "https://redacted.blob.core.windows.net/redacted/20210525/redacted.csv",
            "url": "https://redacted.blob.core.windows.net/redacted/20210525/redacted.csv",
            "sequencer": "000000000000000000000000000093100000000000001286",
            "storageDiagnostics": {
                "batchId": "redacted"
            }
        }
    }
]

【讨论】：