【问题标题】:Hive Server 2 error on python connect with hiveserver2python 上的 Hive Server 2 错误与 hiveserver2 连接
【发布时间】:2023-03-31 12:25:01
【问题描述】:

我正在使用

Centos、Python2.7、hive 2.1、Hadoop 2.7.2、pyHive

这里是代码

from pyhive import hive
from TCLIService.ttypes import TOperationState
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)

#status = cursor.poll().operationState
#while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNI$
#    logs = cursor.fetch_logs()
#    for message in logs:
#        print message

# If needed, an asynchronous query can be cancelled at any time with:
# cursor.cancel()

# status = cursor.poll().operationState

#print cursor.fetchall()

当我在终端中运行 python /usr/local/py/test5.py 时,它的显示 ....

Traceback (most recent call last):
File "/usr/local/py/test5.py", line 3, in <module>
cursor = hive.connect('localhost').cursor()
File "/usr/local/lib/python2.7/site-packages/pyhive/hive.py", line 63, in connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/pyhive/hive.py", line 104, in __init__
self._transport.open()
File "/usr/local/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 80, in open
status, payload = self._recv_sasl_message()
File "/usr/local/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 98, in _recv_sasl_message
header = read_all_compat(self._trans, 5)
File "/usr/local/lib/python2.7/site-packages/thrift_sasl/six.py", line 31, in <lambda>
read_all_compat = lambda trans, sz: trans.readAll(sz)
File "/usr/local/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 120, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

在此之后显示 Hive 服务器错误日志...

ERROR [HiveServer2-Handler-Pool: Thread-41]: server.TThreadPoolServer (:()) - Thrift error occur$
org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
    at    org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
    at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

我也试过pyhs2得到同样的错误

出了什么问题?

谢谢

【问题讨论】:

  • 我也面临同样的问题,我觉得这与 thrift_sasl 包有关。还没有走得更远,这令人沮丧。
  • 你试过重启服务器吗? sudo server hbase-thrift restart
  • 错误信息显示Missing version in readMessageBegin, old client? -- 可能thrift_sasl 使用thrift 的过时协议版本?对我来说,与hbase 的节俭连接工作正常,只是hive 不行。你能确认一下吗,@imran-hassan?
  • 检查您的服务器和客户端是否使用相同版本的 hive thrift 运行
  • 是的,同意 hbase 工作正常,但 hive 服务器在尝试将 hiveserver2 与 python 连接时出现同样的错误

标签: python python-2.7 hadoop hive pyhive


【解决方案1】:

我已经用这些版本解决了这个错误:

Centos 7、Python2.7、hive 2.1、Hadoop 2.7.3 和 java java 1.7.0_91

使用impyla

它对我有用。

【讨论】:

    【解决方案2】:

    thrift.transport.TTransport.TTransportException: TSocket 读取 0 字节

    我遇到了同样的问题并通过

    解决了
    1. 启动 hiveserver2 (how to start hiveserver2)
    2. 修改您的端口。默认端口为 10000,或者您可以将其更改为跟随您的 hive_site.xml

    【讨论】:

    • 你能解释一下你的答案吗?从问题的输出来看,服务器似乎正在运行,或者您会收到类似“连接被拒绝”的错误,但不是“读取 0 字节”。另外,默认端口 10000 有什么问题?假设你的答案是正确的,究竟应该把什么值改成什么?
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2015-06-22
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2019-08-24
    • 1970-01-01
    相关资源
    最近更新 更多