【问题标题】:Writing file from different threads in C++在 C++ 中从不同线程写入文件
【发布时间】:2018-09-13 13:40:11
【问题描述】:

我正在编写一个从不同线程写入二进制文件的程序。每个线程将写入文件的不同位置。我不使用同步,我的程序正常工作。我想问你我是否应该使用一些同步以及如何或是否足以希望操作系统同步无论如何都会做到这一点。我在 Linux 上,使用 gcc 编译器,但在某些时候它也可能在其他平台上运行。我使用以下函数从不同线程写入文件。

void writeBytesFrom(std::string fileName, uint64_t fromPosition, uint8_t* buffer, int numBytes)
{
    if(CHAR_BIT != 8)
    {
        std::stringstream errMsg;
        errMsg << "Can not use this platform since CHAR_BIT size is not 8, namely it is "
               << CHAR_BIT << ".";
        LOGE << errMsg.str();
        throw std::runtime_error(errMsg.str());
    }
    std::ofstream file(fileName,
                       std::ios::binary | std::ios::out
                           | std::ios::in); // Open binary, for output, for input
    if(!file.is_open()) // cannot open file
    {
        std::stringstream errMsg;
        errMsg << "Can not open file " << fileName << ".";
        LOGE << errMsg.str();
        throw std::runtime_error(errMsg.str());
    }
    file.seekp(fromPosition); // put pointer
    std::streampos cur = file.tellp();
    file.write((char*)buffer, numBytes);
    auto pos = file.tellp();
    auto num = pos - cur;
    file.close();
    if(num != numBytes)
    {
        std::stringstream errMsg;
        errMsg << num << " bytes written from number of bytess that should be written "
               << numBytes << " to " << fileName << ".";
        LOGE << errMsg.str();
        throw std::runtime_error(errMsg.str());
    }
}

如果您对改进我的代码有任何进一步的建议,我愿意接受。我使用 uint8_t 缓冲区的原因是,我更自然地理解缓冲区表示 8 位字节,而不是使用 unsigned char。我知道有些纯粹主义者可能会反对。

谢谢, 沃伊塔。

【问题讨论】:

  • 我认为你的代码是靠运气工作的。 std::basic_filebuf 没有声明通过两个 filebuf 并发访问同一个文件。
  • 事实上,你的线程一次只能写入文件,所以最好有一个“写入顺序”队列。您的线程将其数据/位置放入队列,特定线程将负责将数据写入磁盘。
  • 只要每个线程打开自己的文件描述符(std::ifstream 对象)并写入文件的不同部分应该没有问题。

标签: c++ synchronization


【解决方案1】:

如果您对改进我的代码有任何进一步的建议,我愿意接受。

不是每个人都会同意每条评论或代码风格推荐,但这些对我有用:

#include <string>
#include <climits>
#include <fstream>
#include <sstream>
#include <iomanip>
#include <vector>

extern std::ostream& LOGE;


// See below for the reasons I write this
struct unwrapper {

    void print_exception(std::ostream&os, const std::exception& e, int level =  0) const
    {
        os << level << " : " << e.what() << '\n';
        try {
            std::rethrow_if_nested(e);
        } catch(const std::exception& e) {
            print_exception(os, e, level+1);
        } catch(...) {
            os << level << " : nonstandard exception";
        }
    }    

    std::exception const& initial_exception;

    friend std::ostream& operator<<(std::ostream& os, unwrapper const& u)
    {
        u.print_exception(os, u.initial_exception);
        return os;
    }
};

unwrapper unwrap(std::exception const& e)
{
    return unwrapper { e };
}

// Comment #3 : separate concerns - opening the outfile is a separate job for which
//              we will want a discrete error. So wrap that logic into a function
// Comment #4 : separate exception handling from code
// Comment #5 : nested exceptions are great for providing a forensic trail
//              which helps us to solve runtime errors
// Comment #2 : an ofstream has no business being open for input 
//              std::ios::out is assumed
// Comment #6 : since we're using exceptions, let the stream object raise them for us
std::ofstream openOutput(std::string const& fileName)
try
{

    std::ofstream file(fileName, std::ios::binary);
    file.exceptions(std::ios::badbit | std::ios::failbit);    
    return file;
}
catch(std::exception&)
{
    auto message = [&]() {
        std::ostringstream ss;
        ss << "openOutput: fileName = " << std::quoted(fileName);
        return std::move(ss).str();
    } ();
    LOGE << message;
    std::throw_with_nested(std::runtime_error(message));
}

void writeBytesFrom(std::string fileName, uint64_t fromPosition, uint8_t* buffer, int numBytes)
try  // function try blocks separate exception handling from logic
{
    // Comment #1 : prefer non-compilation over runtime failures.
    // Comment #7 if too few bytes are written, this will now throw
    //            you don't need to check
    // Comment #8 file.close() is automatic, but harmless to leave in    

    static_assert(CHAR_BIT == 8, "Can not use this platform since CHAR_BIT size is not 8");

    std::ofstream file(fileName, std::ios::binary); // Open binary, for output, for input
    file.seekp(fromPosition); // put pointer
    file.write((char*)buffer, numBytes);
    file.close();
}
catch(std::exception&)
{
    auto message = [&]() {
        std::ostringstream ss;
        ss << "writeBytesFrom: fileName = " << std::quoted(fileName) << ", fromPosition = " << fromPosition;
        return std::move(ss).str();
    } ();
    LOGE << message;
    std::throw_with_nested(std::runtime_error(message));
}


void test()
{
    extern std::string getfile();
    extern std::vector<std::uint8_t>& getbuffer();
    extern uint64_t getpos();

    auto&& file = getfile();
    auto&& buffer = getbuffer();
    auto pos = getpos();

    try
    {
        writeBytesFrom(file, pos, buffer.data(), buffer.size());
    }
    catch(std::exception& e)
    {
        // Comment #9 we can unwrap nested exceptions into one log line to
        // provide a complete history of the error
        LOGE << "test failed to write: " << unwrap(e);
    }
}

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2018-11-18
    • 1970-01-01
    • 2016-07-19
    • 2010-12-11
    • 1970-01-01
    • 2012-06-14
    • 2018-11-14
    • 2012-10-05
    相关资源
    最近更新 更多