【问题标题】:Increasing max size of Mongo BSON Objects results in Boost library compile error增加 Mongo BSON 对象的最大大小会导致 Boost 库编译错误
【发布时间】:2014-12-22 10:08:25
【问题描述】:

我需要增加 Mongo 中 BSON 对象的最大大小。由于 BSON 对象的大小不能直接配置,我试图通过更改源中的常量并重新编译来做到这一点。

问题是任何试图改变 const int BSONObjMaxUserSize = 16 * 1024 * 1024;到 16 in this file 以外的任何内容都会在 boost 库进行编译时导致崩溃。

将 BSONObjMaxUserSize 保留为 16 可以完美编译,但将其更改为其他任何值都会失败。我已经尝试了很多选项(8,32,甚至 7 个很好的衡量标准。)

我在 2.6.6 稳定版中遇到过这种情况。

    In file included from src/third_party/boost/boost/type_traits/type_with_alignment.hpp:19:0,
                 from src/third_party/boost/boost/smart_ptr/make_shared.hpp:17,
                 from src/third_party/boost/boost/make_shared.hpp:15,
                 from src/third_party/boost/boost/smart_ptr.hpp:28,
                 from src/mongo/pch.h:50,
                 from src/mongo/db/dur_commitjob.cpp:31:
src/third_party/boost/boost/static_assert.hpp:124:85: error: invalid application of 'sizeof' to incomplete type 'boost::STATIC_ASSERTION_FAILURE<false>'
       sizeof(::boost::STATIC_ASSERTION_FAILURE< BOOST_STATIC_ASSERT_BOOL_CAST( B ) >)>\
                                                                                     ^
src/mongo/db/dur_commitjob.cpp:147:9: note: in expansion of macro 'BOOST_STATIC_ASSERT'
         BOOST_STATIC_ASSERT( sizeof(void*)==4 || UncommittedBytesLimit > BSONObjMaxInternalSize * 6 );
         ^
src/third_party/boost/boost/static_assert.hpp:124:86: error: template argument 1 is invalid
       sizeof(::boost::STATIC_ASSERTION_FAILURE< BOOST_STATIC_ASSERT_BOOL_CAST( B ) >)>\
                                                                                      ^
src/mongo/db/dur_commitjob.cpp:147:9: note: in expansion of macro 'BOOST_STATIC_ASSERT'
         BOOST_STATIC_ASSERT( sizeof(void*)==4 || UncommittedBytesLimit > BSONObjMaxInternalSize * 6 );
         ^
src/mongo/db/dur_commitjob.cpp:147:102: error: invalid type in declaration before ';' token
         BOOST_STATIC_ASSERT( sizeof(void*)==4 || UncommittedBytesLimit > BSONObjMaxInternalSize * 6 );
                                                                                                      ^
In file included from src/third_party/boost/boost/filesystem/v3/path_traits.hpp:23:0,
                 from src/third_party/boost/boost/filesystem/v3/path.hpp:25,
                 from src/third_party/boost/boost/filesystem/path.hpp:31,
                 from src/mongo/util/paths.h:21,
                 from src/mongo/db/storage/durable_mapped_file.h:35,
                 from src/mongo/db/dur.h:34,
                 from src/mongo/db/dur_commitjob.h:34,
                 from src/mongo/db/dur_commitjob.cpp:33:
src/third_party/boost/boost/system/error_code.hpp:214:36: error: 'boost::system::posix_category' defined but not used [-Werror=unused-variable]
     static const error_category &  posix_category = generic_category();
                                    ^
src/third_party/boost/boost/system/error_code.hpp:215:36: error: 'boost::system::errno_ecat' defined but not used [-Werror=unused-variable]
     static const error_category &  errno_ecat     = generic_category();
                                    ^
src/third_party/boost/boost/system/error_code.hpp:216:36: error: 'boost::system::native_ecat' defined but not used [-Werror=unused-variable]
     static const error_category &  native_ecat    = system_category();
                                    ^
cc1plus: all warnings being treated as errors
scons: *** [build/linux2/normal/mongo/db/dur_commitjob.o] Error 1
scons: building terminated because of errors.

【问题讨论】:

  • 为什么需要更改最大 BSON 大小(以及您要更改为什么)?通常,接近当前 16MB 限制的文档也与性能不佳的架构设计相关(例如,具有无限增长的数组)。如果您需要存储大型二进制数据,有一个很有用的GridFS 约定。
  • 从消息来看,UncommittedBytesLimit 设置为什么?你有没有改变它?

标签: mongodb boost


【解决方案1】:

这里没有问题:

  • 从 github 克隆 0ae940b456ceafc3a0d47
  • 改变了builder.h:

    diff --git a/src/mongo/bson/util/builder.h b/src/mongo/bson/util/builder.h
    index ae31074..8e34aee 100644
    --- a/src/mongo/bson/util/builder.h
    +++ b/src/mongo/bson/util/builder.h
    @@ -63,7 +63,7 @@ namespace mongo {
            update $push (append) operation
            various db.eval() type operations
        */
    -    const int BSONObjMaxUserSize = 16 * 1024 * 1024;
    +    const int BSONObjMaxUserSize = 24 * 1024 * 1024;
    
        /*
            Sometimes we need objects slightly larger - an object in the replication local.oplog
    

使用scons -j 8 干净地编译。这是我的系统信息:

  1. 操作系统版本

    /tmp/mongo$ lsb_release -a
    No LSB modules are available.
    Distributor ID: Ubuntu
    Description:    Ubuntu 14.04.1 LTS
    Release:    14.04
    Codename:   trusty
    
  2. 内核版本

    /tmp/mongo$ uname -a
    Linux desktop 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
    
  3. gcc 版本

    /tmp/mongo$ gcc -v
    Using built-in specs.
    COLLECT_GCC=/usr/bin/gcc
    COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
    Target: x86_64-linux-gnu
    Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.8.2-19ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
    Thread model: posix
    gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) 
    
  4. 增强版

    /tmp/mongo$ dpkg --status libboost-all-dev 
    Package: libboost-all-dev
    Status: install ok installed
    Priority: optional
    Section: libdevel
    Installed-Size: 26
    Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
    Architecture: amd64
    Source: boost-defaults
    Version: 1.54.0.1ubuntu1
    

【讨论】:

    猜你喜欢
    • 2013-07-11
    • 2019-07-21
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2014-08-24
    • 1970-01-01
    相关资源
    最近更新 更多