尝试连接到 azure datalake 时，Kubernetes Pod 中出现“OAuth2Client：获取令牌请求失败”错误

Question

我有一个 docker 映像，它在 python 中使用 adlCreds = lib.auth(tenant_id=tenantId, client_secret=application_key, client_id=application_id) 创建到 Azure Datalake 的连接并访问一个文件。我正在使用 Airflow Kubernetes_Pod_Operator 在 Kubernetes Pod 中执行 docker 映像。当我实例化 pod 操作符时，Kubernetes 中的容器正在终止并出现错误 OAuth2Client:Get Token request failed

我已验证 Azure 凭据，并且当我在本地执行连接时连接正常。仅当我使用 Kubernetes Pod 在集群中运行它时才会收到此错误。

        # Creating ADL connection

        def make_azure_enduser():
        logging.info('Creating the Azure Datalake Client...')
        config = dict(azure_tenantid=<tenant_id>'),
                      application_key=<azure_application_key>),
                      application_id=<azure_applicationid>),
                      subscriptionId=<azure_subid>),
                      adlAccountName=<azure_datalake_accountname>)
                      )
        # Format the tenant ID and Account Name as strings to be passed into the object
        application_id = "{}".format(config['application_id'])
        application_key = "{}".format(config['application_key'])
        tenantId = "{}".format(config['azure_tenantid'])
        store_name = "{}".format(config['adlAccountName'])

        # Create the token and the Azure Datalake File System Client
        adlCreds = lib.auth(tenant_id=tenantId, client_secret=application_key, client_id=application_id)

        adlsFileSystemClient = core.AzureDLFileSystem(adlCreds, store_name=store_name)
        return adlsFileSystemClient

        # instantiating Kubernetes_pod_operator in Airflow DAG

        get_adl_file = KubernetesPodOperator(namespace='default',
                                 image="get_adl_file:latest",
                                 image_pull_policy='IfNotPresent',
                                 image_pull_secrets='acr-registry',
                                 # cmds=["/bin/sh", "-c", "sleep 500"],
                                 # arguments=["sleep 500"],
                                 labels={"foo": "bar"},
                                 name="get_file",
                                 task_id="get_adl_file",
                                 get_logs=True,
                                 dag=dag
                                 )

我希望 pod 创建与 datalake 的连接，但它失败并出现以下错误。

Kubernetes Pod 日志

> Creating the Azure Datalake Client...
> ed2a3672-9d3b-456e-924f-cdf6e5f60ce8 - TokenRequest:Getting token with
> client credentials.
> /opt/conda/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15:
> DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and
> will be removed in 0.23. Please import this functionality directly
> from joblib, which can be installed with: pip install joblib. If this
> warning is raised when loading pickled models, you may need to
> re-serialize those models with scikit-learn 0.21+.  
> warnings.warn(msg, category=DeprecationWarning)
> ed2a3672-9d3b-456e-924f-cdf6e5f60ce8 - OAuth2Client:Get Token request
> failed Traceback (most recent call last):   File
> "/opt/conda/lib/python3.7/site-packages/urllib3/connection.py", line
> 159, in _new_conn
>     (self._dns_host, self.port), self.timeout, extra_kw)   File "/opt/conda/lib/python3.7/site-packages/urllib3/util/connection.py",
> line 57, in create_connection
>     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):   File "/opt/conda/lib/python3.7/socket.py", line
> 748, in getaddrinfo
>     for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name
> resolution During handling of the above exception, another exception
> occurred: Traceback (most recent call last):   File
> "/opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py",
> line 600, in urlopen
>     chunked=chunked)   File "/opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py",
> line 343, in _make_request
>     self._validate_conn(conn)   File "/opt/conda/lib/python3.7/site-packages/urllib3/connectionpool.py",
> line 839, in _validate_conn
>     conn.connect()   File "/opt/conda/lib/python3.7/site-packages/urllib3/connection.py", line
> 301, in connect
>     conn = self._new_conn()   File "/opt/conda/lib/python3.7/site-packages/urllib3/connection.py", line
> 168, in _new_conn
>     self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError:
> <urllib3.connection.VerifiedHTTPSConnection object at 0x7f0941ce8160>:
> Failed to establish a new connection: [Errno -3] Temporary failure in
> name resolution

请帮我解决这个连接错误。谢谢

Answer 1

这似乎是一个非常具体的问题。

试试 Kubernetes 的“Slack”，有熟练的用户乐于助人！