自定义流操纵器答案

【问题标题】：Custom stream manipulators自定义流操纵器
【发布时间】：2014-07-05 16:22:30
【问题描述】：

我有一个继承自 std::ostream 的自定义流 CFileManagerOStream，它从网络流类 CTcpStream 获取 Unicode UTF16 或 UTF32 字符串，并将它们作为 UTF8 字符串存储在磁盘上。这些字符串可能太大（可能是多个 TB），无法在内存中转换为 UTF8，所以我认为我需要为这项工作实现一个 C++ 流操纵器。我发现的所有操纵器示例都采用整个字符串并对其进行处理，由于内存要求低，在我的情况下不会这样做。我已经准备好所有的 Unicode 转换代码，我要解决的问题是用尽可能少的内存进行转换。

我希望像这样使用机械手；

CFileManagerOStream outFile("MultipleUtf8Strings.dat"); // Custom std::ostream
...
#ifdef _WINDOWS
CTcpStreamUtf16 largeBlobUtf16Stream;
...
outFile << ToUTF8FromUtf16 << largeBlobUtf16Stream;
#else
CTcpStreamUtf32 largeBlobUtf32Stream;
...
outFile << ToUTF8FromUtf32 << largeBlobUtf32Stream;
#endif

这是可能的还是我错误地接近这个？

【问题讨论】：

为什么不out << ToUTtf8(large) - 将它包装到一个接受参数的操纵器类中？
@DieterLücking 你有一个例子可以解决流式大数据集的问题吗？

标签： c++ stream ostream manipulators

【解决方案1】：

我发现使用std::ios_base::iword 存储请求的字符编码是手头问题的最佳解决方案：

#include <iostream>

/*!
\brief Unicode encoding
*/
enum EUnicodeEnc
{
    /** UTF-8 character encoding */
    EUnicodeEnc_UTF8 = 1,

    /** UTF-16 character encoding */
    EUnicodeEnc_UTF16 = 2,

    /** UTF-32 character encoding */
    EUnicodeEnc_UTF32 = 3
};

/** Allocate the \c std::ios_base::iword storage for use with \c SourceStreamEncoding object instances */
int SourceStreamEncoding::sourceEnc_xalloc = std::ios_base::xalloc();

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-8
*/
std::ios_base& FromUtf8(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF8;
    return os;
}

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-16
*/
std::ios_base& FromUtf16(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF16;
    return os;
}

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-32
*/
std::ios_base& FromUtf32(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF32;
    return os;
}

/*!
\brief Overrides \c std::ostream::flush()
\details Converts the buffer to the correct character encoding then flushes buffer
after writing its content to a storage device
*/
std::ostream &CFileManagerOStream::flush()
{
    switch (os.iword(SourceStreamEncoding::sourceEnc_xalloc))
    {
        case EUnicodeEnc_UTF8:
            characterEncoder.FromUTF8(...);
        break;
        case EUnicodeEnc_UTF16:
            characterEncoder.FromUTF16(...);
        break;
        case EUnicodeEnc_UTF32:
            characterEncoder.FromUTF32(...);
        break;
    }
    return (*this);
}

// Now I can do as follows:
int main()
{
    CFileManagerOStream outFile("MultipleUtf8Strings.dat"); // Custom std::ostream
    ...
#ifdef _WINDOWS
    CTcpStreamUtf16 largeBlobUtf16Stream;
    ...
    outFile << FromUtf16 << largeBlobUtf16Stream;
#else
    CTcpStreamUtf32 largeBlobUtf32Stream;
    ...
    outFile << FromUtf32 << largeBlobUtf32Stream;
#endif
}

此外，我添加了以下采用单个参数的操纵器：

class FromEnc
{
    public:
        explicit FromEnc(int i) : i_(i) {}
        int i_;
    private:
        template <class charT, class Traits>
        friend std::basic_ostream<charT, Traits>& operator<<(std::basic_ostream<charT, Traits>& os, const FromEnc& w) {
            os.iword(SourceStreamEncoding::sourceEnc_xalloc) = w.i_;
            return os;
        }
};

，所以现在我也可以这样做：

outFile << FromEnc(EUnicodeEnc_UTF16) << largeBlobUtf16Stream;

【讨论】：