参考 http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/1.0.1/Installing-and-Using-Impala/ciiu_lzo.html

sudo apt-get install liblzo2-dev
wget http://archive-primary.cloudera.com/gplextras/ubuntu/precise/amd64/gplextras/pool/contrib/h/hadoop-lzo-cdh4/hadoop-lzo-cdh4_0.4.15.orig.tar.gz
tar -zxvf hadoop-lzo-cdh4_0.4.15.orig.tar.gz
cd hadoop-lzo-cdh4_0.4.15.orig
ant package

解决错误(http://hi.baidu.com/qingchunranzhi/item/72155fd1ccf5e05bd73aaebc)
chmod 645 /home/hadoop/soft/hadoop-lzo-cdh4-0.4.15-gplextras/src/get_build_revision.sh
要不报权限错误
另外在普通用户下还是报权限错误, 在root用户下可以,然后再修改权限。

cp hadoop-lzo-0.4.15.jar $HADOOP_HOME/share/hadoop/common/
cp  build/native/Linux-*-*/lib/libgplcompression.* /home/hadoop/hadoop-2.0.0-cdh4.5.0/lib/native/

应该就可以了

使用

conf.setBoolean("mapreduce.map.output.compress", true);
conf.setClass("mapreduce.map.output.compression.codec", LzoCodec.class, CompressionCodec.class);
conf.setBoolean("mapreduce.output.fileoutputformat.compress", true);
conf.setClass("mapreduce.output.fileoutputformat.compress.codec", LzoCodec.class, CompressionCodec.class);

相关文章:

  • 2022-12-23
  • 2021-06-25
  • 2021-08-25
  • 2022-03-03
  • 2021-06-30
  • 2021-09-13
  • 2021-12-17
  • 2021-08-06
猜你喜欢
  • 2022-01-09
  • 2021-05-24
  • 2021-08-05
  • 2021-07-05
  • 2022-12-23
  • 2022-02-01
  • 2022-12-23
相关资源
相似解决方案