【问题标题】:Suggestions for Implementing Session Block for Service Bus为服务总线实现会话块的建议
【发布时间】:2020-10-15 08:19:31
【问题描述】:

目前,我们有一个带有后台服务的 dotnet 核心应用程序,该应用程序从启用会话的服务总线接收消息,其中sessionId 是userId,并且该消息包含对用户信息的更新。现在我们想实现一个功能,通过阻止特定的userId/sessionId 来暂时暂停对特定用户的更新,但在解除阻止时仍按顺序处理消息。解决这个问题的最佳方法是什么?

我已尝试查看服务总线文档和示例。主要是message deferral、message session state和session state sample

而且我找到了一些关于SessionState和Message deferral的信息,我想知道它们是否可以用来实现这个功能并且仍然保证处理顺序(FIFO不管消息是否被延迟)。我正在考虑尝试将序列号存储在会话状态中并继续通过该编号接收延迟消息并增加它以接收下一条消息,直到我用完消息?

目前,我们的代码如下所示:

            this.queue.RegisterSessionHandler(
                this.SessionHandler,
                new SessionHandlerOptions(this.ExceptionHandler)
                {
                    AutoComplete = false,
                    MessageWaitTimeout = TimeSpan.FromMinutes(1),
                });

其中this.SessionHandler 是一个处理消息然后通过调用session.CompleteAsync 和session.CloseAsync 完成并关闭会话的函数。但是,我在概念化如何将延迟逻辑添加到我们的代码时遇到了麻烦。因为目前,RegisterSessionHandler 已经处理了会话锁并使用sessionId(我假设)对消息进行负载平衡,这很棒。但是RegisterSessionHandler 也不允许您指定要处理的特定sessionId。

假设我有几条消息推迟到userId/sessionId: A。当我想取消阻止该用户的处理时,我不能简单地将延迟消息插入到队列中。因为发件人仍然会不断地为用户 A 发送消息到队列中,它会打乱顺序。

我上面提到的会话状态示例有一个关于使用会话状态和处理延迟消息的好例子。但是,它只使用了一个sessionId,并没有使用RegisterSessionHandler。我的问题是:如果我们想实现延迟消息处理逻辑(保留顺序),我们是否必须实现自己的RegisterSessionHandler 并处理sessionId 负载平衡?

提前致谢!

【问题讨论】:

  • 从问题来看,您发布的代码面临什么问题还不是很清楚?另外,当您说“但我无法从文档中真正理解它们是如何工作的。”时,请提供文档链接,究竟什么不清楚?情景方面,您的方向是正确的。
  • @krishg 抱歉,我之前不是很清楚。我已根据您的建议更新了我的问题。如果您发现其他不清楚的地方,请告诉我。谢谢!

标签: azure .net-core messaging servicebus azure-servicebus-queues


【解决方案1】:

您应该在 QueueClient 中使用 SessionClient 而不是使用 RegisterSessionHandler 以更好地处理延迟场景并保持顺序。您可以在消息正文中维护一些步骤/序列号。当您实际处理消息时,还要添加 LastProcessedStep/Seqence。 Session state 允许跟踪处理程序与会话相关的处理状态,以便客户端可以在会话处理期间在处理节点(包括故障转移)之间保持敏捷。 sample 通过维护延迟消息(步骤)来处理延迟消息。它结合了延迟和会话功能,以便使用会话状态工具来跟踪工作流的处理状态,其中各个步骤的输入未按预期顺序到达。请注意发送方代码,它还演示了通过以不可预知的顺序发送消息,但凭借会话状态,接收方检测到顺序。

//   
//   Copyright © Microsoft Corporation, All Rights Reserved
// 
//   Licensed under the Apache License, Version 2.0 (the "License"); 
//   you may not use this file except in compliance with the License. 
//   You may obtain a copy of the License at
// 
//   http://www.apache.org/licenses/LICENSE-2.0 
// 
//   THIS CODE IS PROVIDED *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS
//   OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION
//   ANY IMPLIED WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A
//   PARTICULAR PURPOSE, MERCHANTABILITY OR NON-INFRINGEMENT.
// 
//   See the Apache License, Version 2.0 for the specific language
//   governing permissions and limitations under the License. 

namespace SessionState
{
    using Microsoft.Azure.ServiceBus;
    using Microsoft.Azure.ServiceBus.Core;
    using Newtonsoft.Json;
    using System;
    using System.Collections.Generic;
    using System.Text;
    using System.Threading.Tasks;

    public class Program : MessagingSamples.Sample
    {
        public async Task Run(string connectionString)
        {
            Console.WriteLine("Press any key to exit the scenario");

            var sendTask = this.SendMessagesAsync(Guid.NewGuid().ToString(), connectionString, SessionQueueName);
            var sendTask2 = this.SendMessagesAsync(Guid.NewGuid().ToString(), connectionString, SessionQueueName);
            var receiveTask = this.ReceiveMessagesAsync(connectionString, SessionQueueName);

            await Task.WhenAll(sendTask, sendTask2, receiveTask);
        }

        async Task SendMessagesAsync(string session, string connectionString, string queueName)
        {
            var sender = new MessageSender(connectionString, queueName);


            Console.WriteLine("Sending messages to Queue...");

            ProcessingState[] data = new[]
            {
                new ProcessingState {Step = 1, Title = "Buy"},
                new ProcessingState {Step = 2, Title = "Unpack"},
                new ProcessingState {Step = 3, Title = "Prepare"},
                new ProcessingState {Step = 4, Title = "Cook"},
                new ProcessingState {Step = 5, Title = "Eat"},
            };

            var rnd = new Random();
            var tasks = new List<Task>();
            for (int i = 0; i < data.Length; i++)
            {
                var message = new Message(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(data[i])))
                {
                    SessionId = session,
                    ContentType = "application/json",
                    Label = "RecipeStep",
                    MessageId = i.ToString(),
                    TimeToLive = TimeSpan.FromMinutes(2)
                };

                tasks.Add(Task.Delay(rnd.Next(30)).ContinueWith(
                      async (t) =>
                      {
                          await sender.SendAsync(message);
                          lock (Console.Out)
                          {
                              Console.ForegroundColor = ConsoleColor.Yellow;
                              Console.WriteLine("Message sent: Id = {0}", message.MessageId);
                              Console.ResetColor();
                          }
                      }));
            }
            await Task.WhenAll(tasks);
        }

        async Task ReceiveMessagesAsync(string connectionString, string queueName)
        {
            var client = new SessionClient(connectionString, queueName, ReceiveMode.PeekLock);

            while (true)
            {
                var session = await client.AcceptMessageSessionAsync();
                await Task.Run(
                    async () =>
                    {
                        ProcessingState processingState;

                        var stateData = await session.GetStateAsync();
                        if (stateData != null)
                        {
                            processingState = JsonConvert.DeserializeObject<ProcessingState>(Encoding.UTF8.GetString(stateData));
                        }
                        else
                        {
                            processingState = new ProcessingState
                            {
                                LastProcessedRecipeStep = 0,
                                DeferredSteps = new Dictionary<int, long>()
                            };
                        }

                        while (true)
                        {
                            try
                            {
                                //receive messages from Queue
                                var message = await session.ReceiveAsync(TimeSpan.FromSeconds(5));
                                if (message != null)
                                {
                                    if (message.Label != null &&
                                        message.ContentType != null &&
                                        message.Label.Equals("RecipeStep", StringComparison.InvariantCultureIgnoreCase) &&
                                        message.ContentType.Equals("application/json", StringComparison.InvariantCultureIgnoreCase))
                                    {
                                        var body = message.Body;

                                        ProcessingState recipeStep = JsonConvert.DeserializeObject<ProcessingState>(Encoding.UTF8.GetString(body));
                                        if (recipeStep.Step == processingState.LastProcessedRecipeStep + 1)
                                        {
                                            lock (Console.Out)
                                            {
                                                Console.ForegroundColor = ConsoleColor.Cyan;
                                                Console.WriteLine(
                                                    "\t\t\t\tMessage received: \n\t\t\t\t\t\tMessageId = {0}, \n\t\t\t\t\t\tSequenceNumber = {1}, \n\t\t\t\t\t\tEnqueuedTimeUtc = {2}," +
                                                    "\n\t\t\t\t\t\tExpiresAtUtc = {5}, \n\t\t\t\t\t\tContentType = \"{3}\", \n\t\t\t\t\t\tSize = {4},  \n\t\t\t\t\t\tContent: [ step = {6}, title = {7} ]",
                                                    message.MessageId,
                                                    message.SystemProperties.SequenceNumber,
                                                    message.SystemProperties.EnqueuedTimeUtc,
                                                    message.ContentType,
                                                    message.Size,
                                                    message.ExpiresAtUtc,
                                                    recipeStep.Step,
                                                    recipeStep.Title);
                                                Console.ResetColor();
                                            }
                                            await session.CompleteAsync(message.SystemProperties.LockToken);
                                            processingState.LastProcessedRecipeStep = recipeStep.Step;
                                            await
                                                session.SetStateAsync(
                                                    Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(processingState)));
                                        }
                                        else
                                        {
// in your case, if customer update is blocked, you can defer
                                            processingState.DeferredSteps.Add((int)recipeStep.Step, (long)message.SystemProperties.SequenceNumber);
                                            await session.DeferAsync(message.SystemProperties.LockToken);
                                            await session.SetStateAsync(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(processingState)));
                                        }
                                    }
                                    else
                                    {
                                        await session.DeadLetterAsync(message.SystemProperties.LockToken);//, "ProcessingError", "Don't know what to do with this message");
                                    }
                                }
                                else
                                {
                                    while (processingState.DeferredSteps.Count > 0)
                                    {
                                        long step;

                                        if (processingState.DeferredSteps.TryGetValue(processingState.LastProcessedRecipeStep + 1, out step))
                                        {
                                            var deferredMessage = await session.ReceiveDeferredMessageAsync(step);
                                            var body = deferredMessage.Body;
                                            ProcessingState recipeStep = JsonConvert.DeserializeObject<ProcessingState>(Encoding.UTF8.GetString(body));
                                            lock (Console.Out)
                                            {
                                                Console.ForegroundColor = ConsoleColor.Cyan;
                                                Console.WriteLine(
                                                    "\t\t\t\tdeferredMessage received: \n\t\t\t\t\t\tMessageId = {0}, \n\t\t\t\t\t\tSequenceNumber = {1}, \n\t\t\t\t\t\tEnqueuedTimeUtc = {2}," +
                                                    "\n\t\t\t\t\t\tExpiresAtUtc = {5}, \n\t\t\t\t\t\tContentType = \"{3}\", \n\t\t\t\t\t\tSize = {4},  \n\t\t\t\t\t\tContent: [ step = {6}, title = {7} ]",
                                                    deferredMessage.MessageId,
                                                    deferredMessage.SystemProperties.SequenceNumber,
                                                    deferredMessage.SystemProperties.EnqueuedTimeUtc,
                                                    deferredMessage.ContentType,
                                                    deferredMessage.Size,
                                                    deferredMessage.ExpiresAtUtc,
                                                    recipeStep.Step,
                                                    recipeStep.Title);
                                                Console.ResetColor();
                                            }
                                            await session.CompleteAsync(deferredMessage.SystemProperties.LockToken);
                                            processingState.LastProcessedRecipeStep = processingState.LastProcessedRecipeStep + 1;
                                            processingState.DeferredSteps.Remove(processingState.LastProcessedRecipeStep);
                                            await session.SetStateAsync(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(processingState)));
                                        }
                                    }
                                    break;
                                }
                            }
                            catch (ServiceBusException e)
                            {
                                if (!e.IsTransient)
                                {
                                    Console.WriteLine(e.Message);
                                    throw;
                                }
                            }
                        }
                        await session.CloseAsync();
                    });
            }
        }

       public static int Main(string[] args)
        {
            try
            {
                var app = new Program();
                app.RunSample(args, app.Run);
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
                return 1;
            }
            return 0;
        }

        class ProcessingState
        {
            [JsonProperty]
            public int LastProcessedRecipeStep { get; set; }
            [JsonProperty]
            public Dictionary<int, long> DeferredSteps { get; set; }
            [JsonProperty]
            public int Step { get; internal set; }
            [JsonProperty]
            public string Title { get; internal set; }
        }
    }
}

您也可以关注Ordering Messages in Azure Service Bus,它非常简要地解释了这个概念。但是那里提供的样本与上面的略有不同。

警告:使用消息会话还意味着对于会话(对于您的情况是用户 ID),该会话中的消息将始终由单个接收者接收和处理。因此,在设置 Session 和 SessionId 时要注意。如果创建一个非常大的会话,这将强制 Azure 服务总线将大部分消息发送到一个订阅者,从而减少多线程的好处。如果您将会话设置得太细,那么它就会失去预期的好处,而且您只会增加不必要的开销。

【讨论】:

  • 感谢您的回答!但是,此处的示例仅演示了使用单个 sessionId 处理消息。我看到了使用SessionClient(connectionString, queueName, ReceiveMode.PeekLock); 和client.AcceptMessageSessionAsync(); 的代码如果我想让后台任务在启用会话锁定的同时同时处理多个会话(因此不会同时处理来自同一会话的两条消息),我会有多个 sessionclients 并让它们都在 for 循环中调用 AcceptMessageSessionAsync 吗?谢谢。
  • 它也适用于多个 sessionId。您可以通过更新发送方逻辑进行测试。接收器是一个连续循环,它等待会话AcceptMessageSessionAsync.,并且对于每个会话,它启动一个任务,该任务依次运行循环。你在跑步的过程中遇到过什么问题吗?你在哪里看到 sessionId 传递给AcceptMessageSessionAsync 方法?
  • 对不起,如果我不清楚。我的意思是,如果我们想同时处理多条消息(具有不同的 sessionId),我们是否也应该生成多个会话客户端?谢谢!
  • 不,你没有。为了您的方便,我刚刚更新了上面的 Run 方法以发送另一组具有不同 sessionId 的消息。但接收器保持不变。它已经为每个会话启动了任务(在外部 `while(true)' 块中)。我建议运行它来调试以便更好地理解。它只是一个简单的控制台应用程序,您可以在本地机器上运行 :) github.com/Azure/azure-service-bus/tree/master/samples/DotNet/…
  • 您也可以关注Ordering Messages in Azure Service Bus,它非常简要地解释了这个概念。但是那里提供的样本与上面的略有不同。
猜你喜欢
  • 1970-01-01
  • 2010-12-05
  • 1970-01-01
  • 2013-02-17
  • 2017-04-25
  • 2012-08-22
  • 1970-01-01
  • 2020-10-23
  • 2018-05-07
相关资源
最近更新 更多