View Post
HBase Scan,Get用法
Scan,get用法
1. get help帮助信息
从下列get用法信息可以看出 get 后面可以跟table表名，rowkey,以及column，value.但是如果想通过get直接获取一个表中的全部数据是做不到的，这种情况就要用到另外一个命令scan。
hbase(main):214:0> help \'get\'
Get row or cell contents; pass table name, row, and optionally
a dictionary of column(s), timestamp, timerange and versions. Examples:
hbase> get \'ns1:t1\', \'r1\'
hbase> get \'t1\', \'r1\'
hbase> get \'t1\', \'r1\', {TIMERANGE => [ts1, ts2]}
hbase> get \'t1\', \'r1\', {COLUMN => \'c1\'}
hbase> get \'t1\', \'r1\', {COLUMN => [\'c1\', \'c2\', \'c3\']}
hbase> get \'t1\', \'r1\', {COLUMN => \'c1\', TIMESTAMP => ts1}
hbase> get \'t1\', \'r1\', {COLUMN => \'c1\', TIMERANGE => [ts1, ts2], VERSIONS => 4}
hbase> get \'t1\', \'r1\', {COLUMN => \'c1\', TIMESTAMP => ts1, VERSIONS => 4}
hbase> get \'t1\', \'r1\', {FILTER => "ValueFilter(=, \'binary:abc\')"}
hbase> get \'t1\', \'r1\', \'c1\'
hbase> get \'t1\', \'r1\', \'c1\', \'c2\'
hbase> get \'t1\', \'r1\', [\'c1\', \'c2\']
hbsase> get \'t1\',\'r1\', {COLUMN => \'c1\', ATTRIBUTES => {\'mykey\'=>\'myvalue\'}}
hbsase> get \'t1\',\'r1\', {COLUMN => \'c1\', AUTHORIZATIONS => [\'PRIVATE\',\'SECRET\']}
2. Scan help帮助信息
scan的用法很多，可以直接扫描全表信息也可以通过指定条件来显示我们所需要获取的数据。这里涉及到Filter的用法接下来会逐一演示
hbase(main):221:0> help \'scan\'
Scan a table; pass table name and optionally a dictionary of scanner
specifications. Scanner specifications may include one or more of:
TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, TIMESTAMP, MAXLENGTH,
or COLUMNS, CACHE
If no columns are specified, all columns will be scanned.
To scan all members of a column family, leave the qualifier empty as in
\'col_family:\'.
The filter can be specified in two ways:
1. Using a filterString - more information on this is available in the
Filter Language document attached to the HBASE-4176 JIRA
2. Using the entire package name of the filter.
Some examples:
hbase> scan \'hbase:meta\'
hbase> scan \'hbase:meta\', {COLUMNS => \'info:regioninfo\'}
hbase> scan \'ns1:t1\', {COLUMNS => [\'c1\', \'c2\'], LIMIT => 10, STARTROW => \'xyz\'}
hbase> scan \'t1\', {COLUMNS => [\'c1\', \'c2\'], LIMIT => 10, STARTROW => \'xyz\'}
hbase> scan \'t1\', {COLUMNS => \'c1\', TIMERANGE => [1303668804, 1303668904]}
hbase> scan \'t1\', {REVERSED => true}
hbase> scan \'t1\', {FILTER => "(PrefixFilter (\'row2\') AND
(QualifierFilter (>=, \'binary:xyz\'))) AND (TimestampsFilter ( 123, 456))"}
hbase> scan \'t1\', {FILTER =>
org.apache.hadoop.hbase.filter.ColumnPaginationFilter.new(1, 0)}
For setting the Operation Attributes
hbase> scan \'t1\', { COLUMNS => [\'c1\', \'c2\'], ATTRIBUTES => {\'mykey\' => \'myvalue\'}}
hbase> scan \'t1\', { COLUMNS => [\'c1\', \'c2\'], AUTHORIZATIONS => [\'PRIVATE\',\'SECRET\']}
For experts, there is an additional option -- CACHE_BLOCKS -- which
switches block caching for the scanner on (true) or off (false). By
default it is enabled. Examples:
hbase> scan \'t1\', {COLUMNS => [\'c1\', \'c2\'], CACHE_BLOCKS => false}
Also for experts, there is an advanced option -- RAW -- which instructs the
scanner to return all cells (including delete markers and uncollected deleted
cells). This option cannot be combined with requesting specific COLUMNS.
Disabled by default. Example:
hbase> scan \'t1\', {RAW => true, VERSIONS => 10}
Besides the default \'toStringBinary\' format, \'scan\' supports custom formatting
by column. A user can define a FORMATTER by adding it to the column name in
the scan specification. The FORMATTER can be stipulated:
1. either as a org.apache.hadoop.hbase.util.Bytes method name (e.g, toInt, toString)
2. or as a custom class followed by method name: e.g. \'c(MyFormatterClass).format\'.
Example formatting cf:qualifier1 and cf:qualifier2 both as Integers:
hbase> scan \'t1\', {COLUMNS => [\'cf:qualifier1:toInt\',
\'cf:qualifier2:c(org.apache.hadoop.hbase.util.Bytes).toInt\'] }
Note that you can specify a FORMATTER by column only (cf:qualifer). You cannot
specify a FORMATTER for all columns of a column family.
Scan can also be used directly from a table, by first getting a reference to a
table, like such:
hbase> t = get_table \'t\'
hbase> t.scan
Note in the above situation, you can still provide all the filtering, columns,
options, etc as described above.
3. 通过get，Scan用法来获取表中指定rowkey信息。
1. get 获取table中rowkey语句 于 Scan获取table中rowkey语句
=================================================================================================================
【get】
hbase(main):011:0> get \'liupeng:employee\',\'1001\'
COLUMN CELL
contect:mail timestamp=1522202414649, value=liupliup@cn.ibm.com
contect:phone timestamp=1522202430196, value=15962459503
group:number timestamp=1522202455929, value=1
info:age timestamp=1522202371257, value=34
info:name timestamp=1522202364156, value=liupeng
【Scan】
hbase(main):010:0> scan \'liupeng:employee\',FILTER=>"PrefixFilter(\'1001\')"
ROW COLUMN+CELL
1001 column=contect:mail, timestamp=1522202414649, value=liupliup@cn.ibm.com
1001 column=contect:phone, timestamp=1522202430196, value=15962459503
1001 column=group:number, timestamp=1522202455929, value=1
1001 column=info:age, timestamp=1522202371257, value=34
1001 column=info:name, timestamp=1522202364156, value=liupeng
1 row(s) in 0.0590 seconds
总结：从上述两种不同的方法可以看出Scan的结果包含了rowkey本身。而get获取到的信息不包含rowkey的值。另外get的column于cell是分开的。而Scan是2者结合在一起的。
另外Scan中FILTER过滤“PrefixFilter”关键字是用来筛选rowkey的。
4. get于Scan获取table中单条数据信息中的区别
《相同点》
hbase(main):229:0> get "liupeng:employee",\'1001\',\'info:phone\'
COLUMN CELL
info:phone timestamp=1527914569028, value=15962459503
1 row(s) in 0.0320 seconds
hbase(main):230:0> scan "liupeng:employee",FILTER=>"PrefixFilter(\'1001\')AND ValueFilter(=,\'substring:159\')"
ROW COLUMN+CELL
1001 column=info:phone, timestamp=1527914569028, value=15962459503
1 row(s) in 0.1010 seconds
《不同点》
##注意事项：上述都可以把table中rowkey为1002，元素为\'159\'的信息查询出来。但是查询的方式截然不同。get是通过指定固定的value \'contect:phone\'来获取到的。
而scan是通过PerfixFilter指定固定的rowkey,然后通过AND条件语句结合ValueFilter指定模糊查询的字符串159查出来的。如果不知道对应的value是contect:phone的基础上
显然Scan这种模糊查询的方式更加高效。
另外Scan下面这种相同语句的查询用get语法是做不到的。例如：
=================================================================================================================
hbase(main):026:0> scan \'liupeng:employee\',FILTER=>"ValueFilter(=,\'substring:159\')"
ROW COLUMN+CELL
1001 column=contect:phone, timestamp=1522202430196, value=15962459503
1002 column=contect:phone, timestamp=1522202527866, value=15977634464
##解释：上述是通过模糊查询直接找到了只要包含159的字段的值就全部显示出来。而get的语法如下所视必须指定rowkey的基础上才可以查询columns。这就需要对rowkey定义的时候
考虑全面的涉及才可以做到。因此从这点来看Scan的方法个人认为比get获取信息更加的便捷。
hbase> t.get \'r1\'
hbase> t.get \'r1\', {TIMERANGE => [ts1, ts2]}
hbase> t.get \'r1\', {COLUMN => \'c1\'}
hbase> t.get \'r1\', {COLUMN => [\'c1\', \'c2\', \'c3\']}
hbase> t.get \'r1\', {COLUMN => \'c1\', TIMESTAMP => ts1}
hbase> t.get \'r1\', {COLUMN => \'c1\', TIMERANGE => [ts1, ts2], VERSIONS => 4}
hbase> t.get \'r1\', {COLUMN => \'c1\', TIMESTAMP => ts1, VERSIONS => 4}
hbase> t.get \'r1\', {FILTER => "ValueFilter(=, \'binary:abc\')"}
hbase> t.get \'r1\', \'c1\'
hbase> t.get \'r1\', \'c1\', \'c2\'
hbase> t.get \'r1\', [\'c1\', \'c2\']
5. Scan方法可以不用指定rowkey检索的情况下直接找valuse值。更具体点说也就是我们要找的哪个column中的哪个value值。get方法是无法做到这一点的。
ColumnPrefixFilter(\'列名\')
hbase(main):038:0> scan \'liupeng:employee\',FILTER=>"ColumnPrefixFilter(\'name\')"
ROW COLUMN+CELL
1001 column=info:name, timestamp=1522202364156, value=liupeng
1002 column=info:name, timestamp=1522202474669, value=Jack_Ma
1003 column=info:name, timestamp=1522202561029, value=kevin_shi
3 row(s) in 0.0210 seconds
##注释：ColumnPrefixFilter代表指定具体哪一个column（key（info）对应的value(name)）。
6. Scan方法方便在于它可以随意指定rowkey，column以及value的值来进行查找。还可以结合AND,ORD等条件语句并用来找到自己想要的数据。
下列语法是AND及OR的连用方法。但是同一条语句中相同的条件语句不可以同时使用。例如AND ....AND..这种方法是不允许的。
hbase(main):060:0> scan \'liupeng:employee\',FILTER=>"ColumnPrefixFilter(\'ph\')AND ValueFilter(=,\'substring:15962\')OR ValueFilter(=,\'substring:186\')"
ROW COLUMN+CELL
1001 column=contect:phone, timestamp=1522202430196, value=15962459503
1003 column=contect:phone, timestamp=1522202605976, value=18665851263
2 row(s) in 0.0170 seconds
7. 通过SingleColumnValueFilter类方法指定检索值列举出检索值对应的所有列及value数据
hbase(main):242:0> scan "liupeng:employee",{FILTER=>"SingleColumnValueFilter(\'info\',\'age\',=,\'substring:30\')"}
ROW COLUMN+CELL
1005 column=contect:mail, timestamp=1528420218800, value=zhangsan@163.com
1005 column=info:age, timestamp=1528439967493, value=30
1005 column=info:name, timestamp=1528420218800, value=zhangsan
1008 column=contect:mail, timestamp=1528681786126, value=www.kevin@alibaba.com
1008 column=info:age, timestamp=1528681786126, value=30
1008 column=info:name, timestamp=1528681786126, value=kevin
2 row(s) in 0.0110 seconds
8. SingleColumnValueFilter类还提供正则表达式查询方法。可以通过模糊查询来查找对应的rowkeys,columns以及values。
hbase(main):244:0> scan "liupeng:employee",{FILTER=>"SingleColumnValueFilter(\'info\',\'name\',=,\'regexstring:liu\')"}
ROW COLUMN+CELL
1001 column=contect:mail, timestamp=1527231141046, value=liupliup@cn.ibm.com
1001 column=info:address, timestamp=1527753987327, value=shanghai
1001 column=info:age, timestamp=1527231097033, value=34
1001 column=info:name, timestamp=1527231081262, value=liupeng
1001 column=info:phone, timestamp=1527914569028, value=15962459503
1004 column=contect:mail, timestamp=1527473497956, value=lqdong@jingdong.com
1004 column=info:address, timestamp=1527755135174, value=shenzhen
1004 column=info:age, timestamp=1527473477124, value=40
1004 column=info:name, timestamp=1527415665182, value=liuqiangdong
2 row(s) in 0.0080 seconds