【问题标题】:Cancelling callbacks in Boost ASIO在 Boost ASIO 中取消回调
【发布时间】:2015-05-02 06:16:22
【问题描述】:

我一直在尝试将我的代码从每个网络连接使用一个 io_service 切换到使用共享的,并且我看到服务器套接字上的一些非常奇怪的行为(客户端套接字似乎工作正常)。

为了弄清楚发生了什么,我重新开始构建一个简单的示例,让我可以检查我对所有应该发生的事情的假设。我遇到的第一个问题是io_service::run 在没有处理程序时不会退出,据我所知,处理程序没有从工作队列中删除。

我有一个线程执行async_accept,后跟async_read。有一个单独的客户端线程(它有自己的io_service)。客户端线程的io_service 从未运行,而服务器的io_service 在另一个线程中运行。

我正在使用条件变量在服务器线程中等待读取完成(这永远不会发生,因为客户端从不写入)。这超时就好了,然后我打电话给socket.cancel()。我希望这会删除读取处理程序并运行以退出,因为工作队列现在是空的。

我确实看到读取处理程序被调用(带有取消错误),但运行永远不会退出。当我将套接字生命周期与处理程序生命周期联系起来时(通过 lambda 将 shared_ptr 捕获到套接字),内存也不会被释放。

服务器是这样设置的:

std::mutex mutex;
std::unique_lock<std::mutex> lock(mutex);
std::condition_variable signal;

boost::asio::io_service server_service;
boost::asio::ip::tcp::acceptor listener(server_service);
std::mutex read_mutex;
std::unique_lock<std::mutex> read_lock(read_mutex);
std::condition_variable read_done;
std::thread server([&]() {
    std::unique_lock<std::mutex> lock(mutex);
    listener.open(boost::asio::ip::tcp::v4());
    listener.set_option(boost::asio::socket_base::enable_connection_aborted(true));
    listener.bind(boost::asio::ip::tcp::endpoint(boost::asio::ip::tcp::v4(), 4567));
    listener.listen();

    std::shared_ptr<connection> server_cnx(new connection(server_service));
    listener.async_accept(server_cnx->socket,
        [&, server_cnx](const boost::system::error_code& error) {
            log_thread() << "Server got a connection " << error << std::endl;
            boost::asio::async_read_until(server_cnx->socket, server_cnx->buffer, '\n',
                [&, server_cnx](const boost::system::error_code& error, std::size_t bytes) {
                    log_thread() << "Got " << bytes << ", " << error << std::endl;
                    std::unique_lock<std::mutex> lock(read_mutex);
                    lock.unlock();
                    read_done.notify_one();
                });
        });
    lock.unlock();
    signal.notify_one();
    if ( read_done.wait_for(read_lock, std::chrono::seconds(1)) == std::cv_status::timeout ) {
        log_thread() << "Server read timed out -- cancelling socket jobs" << std::endl;
        server_cnx->socket.cancel();
        server_cnx->socket.close();
    } else {
        log_thread() << "Server data read" << std::endl;
    }
    log_thread() << "Exiting server thread" << std::endl;
});
signal.wait(lock);
log_thread() << "Server set up" << std::endl;

io_service 线程是这样设置的:

std::thread server_io([&]() {
    log_thread() << "About to service server IO requests" << std::endl;
    try {
        server_service.run();
    } catch ( ... ) {
        log_thread() << "Exception caught" << std::endl;
    }
    log_thread() << "**** Service jobs all run" << std::endl;
    signal.notify_one();
});

输出如下:

10.0002 139992957945728 Server set up
10.0005 139992957945728 Client set up
10.0006 139992848398080 About to service server IO requests
10.0006 139992848398080 Server got a connection system:0
11.0003 139992934819584 Server read timed out -- cancelling socket jobs
11.0004 139992934819584 Exiting server thread
11.0004 139992848398080 Got 0, system:125
20.0006 139992957945728 IO thread timed out servicing requests -- stopping it
^^^ This should not happen because the server service should have run out of work
20.0006 139992957945728 Waiting for things to close....
22.0008 139992957945728 Wait over, exiting

(列是时间+10s、线程ID、日志消息)

在 11 秒标记处,您可以看到调用了 async_read_until。这是服务器 io_service 中的最后一个处理程序,但 run 没有退出。

即使在等待run 退出火灾并且等待线程执行io_service::stop() 超时之后,run 仍然没有退出(那里还有 2 秒的等待时间)。

完整代码在github

【问题讨论】:

  • 很奇怪的使用模式..
  • 嗯,目前我主要关心出现问题时会发生什么。

标签: multithreading sockets c++11 boost boost-asio


【解决方案1】:

当服务器线程尝试解锁不属于它的read_lock 时,程序正在调用未定义的行为。

int main()
{
  ...
  std::mutex read_mutex;
  std::unique_lock<std::mutex> read_lock(read_mutex); // Acquired by main.
  std::condition_variable read_done;
  std::thread server([&]() { // Capture lock reference.
    std::unique_lock<std::mutex> lock(mutex);
    ...
    // The next line invokes undefined behavior as this thread does did
    // not acquire read_lock.mutex().
    if (read_done.wait_for(read_lock, ...)
    //                     ^^^^^^^^^ caller does not own.
    {
      ...
    }
  });
  signal.wait(lock);
  ...
}

特别是在调用condition_variable::wait_for(lock) 时,标准要求lock.owns_lock() 为真并且lock.mutex() 被调用线程锁定。


混合同步和异步流通常会增加复杂性。在这种特殊情况下,同步调用在每一层中交织在一起,使用较低级别的结构进行事件/信号通知而没有持久状态,我认为它增加了不必要的复杂性并使流程过于复杂。此外,广泛的变量范围会增加复杂性。如果 read_lock 从未被 lambda 捕获,则会发生编译器错误。

在尝试观察两个事件时考虑空间分离:

// I will eventually be interested when the server starts
// accepting connections, so start setting up now.
std::mutex server_mutex;
std::unique_lock<std::mutex> server_lock(server_mutex);
std::condition_variable server_started;
std::thread server([&]()
  {
    // I will eventually be interested when the server reads
    // data, so start setting up now.
    std::mutex read_mutex;
    std::unique_lock<std::mutex> read_lock(read_mutex);
    std::condition_variable read_done;
    listener.async_accept(..., 
      [&](...)
      {
        // Got connection.
        async_read_until(...,
          [&](...)
          {
            // Someone may be interested that data has been read,
            // so use the correct mutex and condition_variable
            // pair.
            std::unique_lock<std::mutex> read_lock(read_mutex);
            read_lock.unlock();
            read_done.notify_one();
          });
      }); // async_accept
    // Someone may be interested that I am accepting connections,
    // so use the correct mutex and condition_variable pair.
    std::unique_lock<std::mutex> server_lock(server_mutex);
    server_lock.unlock();
    server_done.notify_one();

    // I am now interested in if data has been read.
    read_done.wait_for(read_lock);
  }); // server thread
// I am now interested in if the server has started.
server_started.wait(server_lock);

调用者必须准备处理一个事件,开始一个操作,然后等待事件,并且操作必须知道调用者感兴趣的事件。为了使情况更糟,现在必须考虑锁排序以防止死锁.请注意在上面的示例中,服务器线程如何获取read_mutex,然后获取server_mutex。另一个线程不能以不同的顺序获取互斥锁,而不会引入死锁的机会。就复杂性而言,这种方法在事件数量上的扩展性很差。

可能值得考虑重新检查程序的流程和控制结构。如果它可以写成主要是异步的,那么回调链、延续或信号槽系统 (Boost.Signals) 可能会使解决方案变得简单。如果一个人更喜欢像同步读取异步代码一样,那么 Boost.Asio 对coroutines 的支持可以提供一个干净的解决方案。最后,如果需要同步等待异步操作的结果或超时,请考虑使用 Boost.Asio 的support for std::future 或直接使用它们。

// Use an asynchronous operation so that it can be cancelled on timeout.
std::future<std::size_t> on_read = boost::asio::async_read_until(
    socket, buffer, '\n',boost::asio::use_future);

// If timeout occurs, then cancel the operation.
if (on_read.wait_for(std::chrono::seconds(1)) == std::future_status::timeout)
{
  socket.cancel();
}
// Otherwise, the operation completed (with success or error).
else
{
  // If the operation failed, then on_read.get() will throw a
  // boost::system::system_error.
  auto bytes_transferred = on_read.get();
}

虽然我强烈主张重新检查整体控制结构并减少变量范围,但以下示例与上述示例大致相同,但使用std::future 可能更易于维护:

// I will eventually be interested when the server starts
// accepting connections, so start setting up now.
std::promise<void> server_started_promise;
auto server_started = server_started_promise.get_future();
std::thread server([&]()
  {
    // I will eventually be interested when the server reads
    // data, so start setting up now.
    std::promise<void> read_done_promise;
    auto read_done = read_done_promise.get_future();
    listener.async_accept(..., 
      [&](...)
      {
        // Got connection.
        async_read_until(...,
          [&](...)
          {
            // Someone may be interested that data has been read.
            read_done_promise.set_value();
          });
      }); // async_accept
    // Someone may be interested that I am accepting connections.
    server_started_promise.set_value();

    // I am now interested in if data has been read.
    read_done.wait_for(...);
  }); // server thread
// I am now interested in if the server has started.
server_started.wait();

下面是基于demonstrates使用std::future以同步方式控制流和超时异步操作的原始代码的完整示例:

#include <future>
#include <iostream>
#include <thread>
#include <boost/asio.hpp>
#include <boost/asio/use_future.hpp>
#include <boost/optional.hpp>
#include <boost/utility/in_place_factory.hpp>

int main()
{
  using boost::asio::ip::tcp;

  // Setup server thread.
  boost::asio::io_service server_io_service;
  std::promise<tcp::endpoint> server_promise;
  auto server_future = server_promise.get_future();

  // Start server thread.
  std::thread server_thread(
    [&server_io_service, &server_promise]
    {
      tcp::acceptor acceptor(server_io_service);
      acceptor.open(tcp::v4());
      acceptor.set_option(
        boost::asio::socket_base::enable_connection_aborted(true));
      acceptor.bind(tcp::endpoint(tcp::v4(), 0));
      acceptor.listen();

      // Handlers will not chain work, so control the io_service with a work
      // object.
      boost::optional<boost::asio::io_service::work> work(
        boost::in_place(std::ref(server_io_service)));

      // Accept a connection.
      tcp::socket server_socket(server_io_service);
      auto on_accept = acceptor.async_accept(server_socket,
                                             boost::asio::use_future);

      // Server has started, so notify caller.
      server_promise.set_value(acceptor.local_endpoint());

      // Wait for connection or error.
      boost::system::system_error error =
        make_error_code(boost::system::errc::success);
      try
      {
        on_accept.get();
      }
      catch (const boost::system::system_error& e)
      {
        error = e;
      }
      std::cout << "Server got a connection " << error.code() << std::endl;

      // Read from connection.
      boost::asio::streambuf buffer;
      auto on_read = boost::asio::async_read_until(
          server_socket, buffer, '\n', boost::asio::use_future);

      // The async_read operation is work, so destroy the work object allowing
      // run() to exit.
      work = boost::none;

      // Timeout the async read operation.
      if (on_read.wait_for(std::chrono::seconds(1)) ==
            std::future_status::timeout)
      {
        std::cout << "Server read timed out -- cancelling socket jobs"
                  << std::endl;
        server_socket.close();
      }
      else
      {
        error = make_error_code(boost::system::errc::success);
        std::size_t bytes_transferred = 0;
        try
        {
          bytes_transferred = on_read.get();
        }
        catch (const boost::system::system_error& e)
        {
          error = e;
        }
        std::cout << "Got " << bytes_transferred << ", " 
                  << error.code() << std::endl;
      }
      std::cout << "Exiting server thread" << std::endl;
    });

  // Wait for server to start accepting connections.
  auto server_endpoint = server_future.get();
  std::cout << "Server set up" << std::endl;

  // Client thread.
  std::promise<void> promise;
  auto future = promise.get_future();
  std::thread client_thread(
    [&server_endpoint, &promise]
    {
      boost::asio::io_service io_service;
      tcp::socket client_socket(io_service);
      boost::system::error_code error;
      client_socket.connect(server_endpoint, error);
      std::cout << "Connected " << error << std::endl;
      promise.set_value();
      // Keep client socket alive, allowing server to timeout.
      std::this_thread::sleep_for(std::chrono::seconds(2));
      std::cout << "Exiting client thread" << std::endl;
    });
  // Wait for client to connect.
  future.get();
  std::cout << "Client set up" << std::endl;

  // Reset generic promise and future.
  promise = std::promise<void>();
  future = promise.get_future();

  // Run server's io_service.
  std::thread server_io_thread(
    [&server_io_service, &promise]
    {
      std::cout << "About to service server IO requests" << std::endl;
      try
      {
        server_io_service.run();
      }
      catch (const std::exception& e)
      {
        std::cout << "Exception caught: " << e.what() << std::endl;
      }
      std::cout << "Service jobs all run" << std::endl;
      promise.set_value();
    });

  if (future.wait_for(std::chrono::seconds(3)) ==
        std::future_status::timeout)
  {
    std::cout << "IO thread timed out servicing requests -- stopping it" 
              << std::endl;
    server_io_service.stop();
  }

  // Join all threads.
  server_io_thread.join();
  server_thread.join();
  client_thread.join();
}

【讨论】:

  • 这个错误正是在主线程而不是服务器线程中获取了 read_lock。这当然是一个人为的示例,旨在测试事物的行为方式。 Boost ASIO 设计的问题是,如果你想要超时,你必须使用异步 API,即使你真的在使用同步 API。出于兼容性原因,我需要保留旧的 API,但我将过渡到异步执行所有操作。我喜欢未来的想法。
  • @KayEss 对于同步超时需要取消的操作,boost::asio::use_future 提供了一个相当干净的解决方案。使用它时,需要考虑处理程序不再链接工作,因此可能需要显式使用和控制work 对象的生命周期,以防止run() 退出。我已经在原版的基础上用一个完整的例子更新了上面的代码。
  • 谢谢。我现在基本上已经完成了这项工作。我正在使用工作实例来控制客户端 IO 服务,但服务器不需要它,因为始终至少有一个 async_accept 处理程序处于活动状态。我在所有事情上都使用了带有超时的条件变量——这已经比我之前使用截止时间计时器的要干净得多。上述练习至少达到了它的目的,我认为我的 io_service 心智模型现在与实际发生的情况相符。
【解决方案2】:

当然,这种多线程是一项棘手的工作。事实证明,在这种情况下,读取锁是在错误的位置获取的,因此处理程序被等待它完成的线程阻塞。

我想这里的教训是永远不要在没有某种超时的情况下处理线程锁。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2014-11-22
    • 1970-01-01
    • 2013-08-07
    • 1970-01-01
    相关资源
    最近更新 更多