【发布时间】:2018-04-21 22:01:54
【问题描述】:
我有一个疯狂的想法,即在我们大多数人通常想要并且会使用互斥同步的某些情况下,可以省略互斥同步。
好吧,假设你有这种情况:
Buffer *buffer = new Buffer(); // Initialized by main thread;
...
// The call to buffer's `accumulateSomeData` method is thread-safe
// and is heavily executed by many workers from different threads simultaneously.
buffer->accumulateSomeData(data); // While the code inside is equivalent to vector->push_back()
...
// All lines of code below are executed by a totally separate timer
// thread that executes once per second until the program is finished.
auto bufferPrev = buffer; // A temporary pointer to previous instance
// Switch buffers, put old one offline
buffer = new Buffer();
// As of this line of code all the threads will switch to new instance
// of buffer. Which yields that calls to `accumulateSomeData`
// are executed over new buffer instance. Which also means that old
// instance is kinda taken offline and can be safely operated from a
// timer thread.
bufferPrev->flushToDisk(); // Ok, so we can safely flush
delete bufferPrev;
虽然很明显,在buffer = new Buffer(); 期间仍然可能存在未完成的操作,这些操作会在前一个实例上添加数据。但是由于磁盘操作很慢,我们会遇到自然的障碍。
那么您如何估计在没有互斥同步的情况下运行此类代码的风险?
编辑
如今,在 SO 中提出问题而不被几个愤怒的家伙无缘无故抢劫是非常困难的。
这是我的correct 代码:
#include <cassert>
#include "leveldb/db.h"
#include "leveldb/filter_policy.h"
#include <iostream>
#include <boost/asio.hpp>
#include <boost/chrono.hpp>
#include <boost/thread.hpp>
#include <boost/filesystem.hpp>
#include <boost/lockfree/stack.hpp>
#include <boost/lockfree/queue.hpp>
#include <boost/uuid/uuid.hpp> // uuid class
#include <boost/uuid/uuid_io.hpp> // streaming operators etc.
#include <boost/uuid/uuid_generators.hpp> // generators
#include <CommonCrypto/CommonDigest.h>
using namespace std;
using namespace boost::filesystem;
using boost::mutex;
using boost::thread;
enum FileSystemItemType : char {
Unknown = 1,
File = 0,
Directory = 4,
FileLink = 2,
DirectoryLink = 6
};
// Structure packing optimizations are used in the code below
// http://www.catb.org/esr/structure-packing/
class FileSystemScanner {
private:
leveldb::DB *database;
boost::asio::thread_pool pool;
leveldb::WriteBatch *batch;
std::atomic<int> queue_size;
std::atomic<int> workers_online;
std::atomic<int> entries_processed;
std::atomic<int> directories_processed;
std::atomic<uintmax_t> filesystem_usage;
boost::lockfree::stack<boost::filesystem::path*, boost::lockfree::fixed_sized<false>> directories_pending;
void work() {
workers_online++;
boost::filesystem::path *item;
if (directories_pending.pop(item) && item != NULL)
{
queue_size--;
try {
boost::filesystem::directory_iterator completed;
boost::filesystem::directory_iterator iterator(*item);
while (iterator != completed)
{
bool isFailed = false, isSymLink, isDirectory;
boost::filesystem::path path = iterator->path();
try {
isSymLink = boost::filesystem::is_symlink(path);
isDirectory = boost::filesystem::is_directory(path);
} catch (const boost::filesystem::filesystem_error& e) {
isFailed = true;
isSymLink = false;
isDirectory = false;
}
if (!isFailed)
{
if (!isSymLink) {
if (isDirectory) {
directories_pending.push(new boost::filesystem::path(path));
directories_processed++;
boost::asio::post(this->pool, [this]() { this->work(); });
queue_size++;
} else {
filesystem_usage += boost::filesystem::file_size(iterator->path());
}
}
}
int result = ++entries_processed;
if (result % 10000 == 0) {
cout << entries_processed.load() << ", " << directories_processed.load() << ", " << queue_size.load() << ", " << workers_online.load() << endl;
}
++iterator;
}
delete item;
} catch (boost::filesystem::filesystem_error &e) {
}
}
workers_online--;
}
public:
FileSystemScanner(int threads, leveldb::DB* database):
pool(threads), queue_size(), workers_online(), entries_processed(), directories_processed(), directories_pending(0), database(database)
{
}
void scan(string path) {
queue_size++;
directories_pending.push(new boost::filesystem::path(path));
boost::asio::post(this->pool, [this]() { this->work(); });
}
void join() {
pool.join();
}
};
int main(int argc, char* argv[])
{
leveldb::Options opts;
opts.create_if_missing = true;
opts.compression = leveldb::CompressionType::kSnappyCompression;
opts.filter_policy = leveldb::NewBloomFilterPolicy(10);
leveldb::DB* db;
leveldb::DB::Open(opts, "/temporary/projx", &db);
FileSystemScanner scanner(std::thread::hardware_concurrency(), db);
scanner.scan("/");
scanner.join();
return 0;
}
我的问题是:我可以省略我尚未使用的batch 的同步吗?因为它是线程安全的,并且在实际将任何结果提交到磁盘之前切换缓冲区就足够了?
【问题讨论】:
-
访问
buffer的“正在运行的线程”如何通知他们正在使用的实例发生了变化?这是一个非常典型的双缓冲区示例,但是当您知道互斥锁可以使代码正确时,为什么还要避免使用互斥锁呢?你会说“性能”,我会问你是否做好了。 -
所有线程都运行在一个共同的父类成员函数
work上。他们都在每次请求时访问这个父成员变量。如果它改变了,他们开始看到改变的成员变量。至于您关于使用互斥锁的问题,我有一个与您相反的惯用问题@Chad,如果我的程序在没有互斥锁的情况下仍然正确,为什么我应该使用互斥锁? :) 我无法翻译orofiled,你这是什么意思? -
你想让
std::atomic<Buffer *>避免UB和互斥体吗? -
“如果我的程序在没有互斥锁的情况下仍然正确,为什么还要使用互斥锁?”。你的程序是不正确的,即使它看起来是正确的。
-
@Jarod42 那是伪代码,它不可能或不可能在常规意义上是正确的,因为预期执行此伪代码的机器是阅读它并提供答案的人问题。它可能不正确的唯一方法
[for you]是如果您没有得到问题或以其他方式定义术语correct program:)
标签: c++ multithreading mutex