到集群计算节点上运行的服务器的 ssh 隧道

Question

我有一个脚本来启动在我们集群的特定节点上运行的 Jupyter Notebook 服务器，并且需要能够从本地 Windows 机器（浏览器界面）访问正在运行的实例。 ssh 命令是通过 paramiko 处理的，我想要一种类似的 Pythonic 方式来创建隧道，而不仅仅是系统调用。我正在尝试 sshtunnel，不幸的是，这是我目前的绊脚石。

我需要创建两条隧道，如下所述。为清楚起见，HOME 是 windows 机器，HOST 是集群头节点，而 NODE 是运行服务器的特定节点。

服务器绑定localhost，默认8888端口。HOME和HOST下面的端口是任意的，我只是用来说明一下（其实最方便的HOME端口也是8888）

[HOME] <------> [HOST] <------> [NODE]
 1111            2222            8888

我知道下面的代码不完整而且完全是错误的，因为我不知道如何正确组合本地/远程绑定。现在我不知道我是不是找错了树——文档建议我可以一次创建两个隧道，但是我需要两个单独的实例吗？

from sshtunnel import SSHTunnelForwarder

t1 = SSHTunnelForwarder(
     ('headnode.server', 22),
     ssh_username='meagain',
     ssh_pkey='~/.ssh/id_rsa',
     local_bind_address = ('localhost',1111),
     remote_bind_addresses =[ ('localhost',2222), ('compute.node',8888)])
t1.start()

编辑：尝试选项 2，如下所述。使用 SSH 客户端在命令行上设置并正常工作。作为参考，这是在带有 python 3.6 的 Anaconda 下，在 Windows 7 机器上（因此下面有额外的文件路径争论）

print(sshtunnel.__version__)
'0.1.0'
print(paramiko.__version__)
'2.1.2'



t1 = sshtunnel.SSHTunnelForwarder(
    NODE,
    ssh_username=USER,
    ssh_pkey=paramiko.RSAKey.from_private_key_file(os.path.expanduser(os.path.normpath('~/.ssh/id_rsa'))),
    ssh_proxy_enabled=True,  # DEFAULT
    ssh_config_file=os.path.expanduser(os.path.normpath('~/.ssh/config')),  # if rather than default
    local_bind_address=('localhost', 1111),
    remote_bind_address=('localhost', 8889)
)
t1.start()

日志：

2017-04-06 12:28:25,523| INF | MainThrea/0981@sshtunnel | 0 keys loaded from agent
    t1.start()
  File "D:\Anaconda3\lib\site-packages\sshtunnel.py", line 1224, in start
    reason='Could not establish session to SSH gateway')
  File "D:\Anaconda3\lib\site-packages\sshtunnel.py", line 1036, in _raise
    raise exception(reason)
sshtunnel.BaseSSHTunnelForwarderError: Could not establish session to SSH gateway
>>> 2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | paramiko.ssh_exception.ProxyCommandFailure: ('ssh HOST -W NODE:22', 'An operation was attempted on something that is not a socket')
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | 
2017-04-06 12:28:25,538| ERR | MainThrea/1076@sshtunnel | Could not connect to gateway NODE:22 : ssh HOST -W NODE:22
Traceback (most recent call last):
  File "D:\Bioinformatics\Scripts\bsub_jupyter\tunneller.py", line 234, in <module>
2017-04-06 12:28:25,523| INF | MainThrea/0901@sshtunnel | Connecting to gateway: NODE:22 as user 'USER'
2017-04-06 12:28:25,523| DEB | MainThrea/0904@sshtunnel | Concurrent connections allowed: True
2017-04-06 12:28:25,523| DEB | MainThrea/1284@sshtunnel | Trying to log in with key: b'1e44a013d97d417e4900025c6c11d073'
2017-04-06 12:28:25,523| DEB | MainThrea/1047@sshtunnel | Connecting via proxy: 'HOST'
2017-04-06 12:28:25,523| ERR |  Thread-2/1572@transport | Exception: ('ssh HOST -W NODE:22', 'An operation was attempted on something that is not a socket')
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | Traceback (most recent call last):
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |   File "D:\Anaconda3\lib\site-packages\paramiko\proxy.py", line 96, in recv
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |     [self.process.stdout], [], [], select_timeout)
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | OSError: [WinError 10038] An operation was attempted on something that is not a socket
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | 
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | During handling of the above exception, another exception occurred:
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | 
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport | Traceback (most recent call last):
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |   File "D:\Anaconda3\lib\site-packages\paramiko\transport.py", line 1749, in run
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |     self._check_banner()
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |   File "D:\Anaconda3\lib\site-packages\paramiko\transport.py", line 1893, in _check_banner
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |     buf = self.packetizer.readline(timeout)
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |   File "D:\Anaconda3\lib\site-packages\paramiko\packet.py", line 331, in readline
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |     buf += self._read_timeout(timeout)
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |   File "D:\Anaconda3\lib\site-packages\paramiko\packet.py", line 485, in _read_timeout
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |     x = self.__socket.recv(128)
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |   File "D:\Anaconda3\lib\site-packages\paramiko\proxy.py", line 107, in recv
2017-04-06 12:28:25,538| ERR |  Thread-2/1570@transport |     raise ProxyCommandFailure(' '.join(self.cmd), e.strerror)

Answer 1

由于您提到的端口 8888 绑定到本地主机，因此无法从 HOST 访问它。如果是这种情况，您有两种选择：

选项 A

使用sshtunnel的ssh_proxy参数：

proxy = paramiko.ProxyCommand('ssh HOST -l USER -i IDENTITY_FILE -W headnode.server:22')
t1 = SSHTunnelForwarder(
    (NODE, 22),
    ssh_username='meagain',
    ssh_pkey='~/.ssh/id_rsa',
    ssh_proxy=proxy,
    local_bind_address=('localhost', 1111),
    remote_bind_addresses=('localhost', 8888)
)

选项 B

使用 ssh 配置文件。

您需要有一个有效的配置文件，其中包含ProxyCommand 的NODE 条目，例如：

Host HOST
    User HOSTUSER
    IdentityFile HOSTUSER_PKEY
Host NODE
    ProxyCommand ssh HOST -W %h:%p

然后，打开隧道，让它读取ProxyCommand 条目：

t1 = SSHTunnelForwarder(
    (NODE, 22),
    ssh_username='meagain',
    ssh_pkey='~/.ssh/id_rsa',
    ssh_proxy_enabled=True,  # DEFAULT
    ssh_config_file='/path/to/your-config-file',  # if rather than default
    local_bind_address=('localhost', 1111),
    remote_bind_addresses=('localhost', 8888)
)