【问题标题】:Serverless - Numpy - Unable to find good bind path format无服务器 - Numpy - 找不到好的绑定路径格式
【发布时间】:2019-05-24 23:50:18
【问题描述】:

我已经解决这个问题一个多星期了,经历了各种各样的论坛问题和帖子,无法解决。我正在尝试将 numpy 打包在一个函数中,单独构建需求(我有多个函数,我想将多个需求分开)。

环境
Windows 10 家庭版

适用于 Windows 的 Docker 工具箱

Client:
Version:       18.03.0-ce
API version:   1.37
Go version:    go1.9.4
Git commit:    0520e24302


Built: Fri Mar 23 08:31:36 2018
 OS/Arch:       windows/amd64
 Experimental:  false
 Orchestrator:  swarm

Server: Docker Engine - Community
 Engine:
  Version:      18.09.0
  API version:  1.39 (minimum version 1.12)
  Go version:   go1.10.4
  Git commit:   4d60db4
  Built:        Wed Nov  7 00:52:55 2018
  OS/Arch:      linux/amd64
  Experimental: false

无服务器版本

serverless version 6.4.1
serverless-python-requirements version 6.4.1

目录结构

|-test
  |-env.yml
  |-serverless.yml
  |-Dockerfile
  |-functions
    |-f1
      |-index.py
      |-requirements.txt
      |-sub_function_1.py
      |-sub_function_2.py
    |-f2
      |-index.py
      |-requirements.txt
      |-sub_function_3.py
      |-sub_function_4.py

serverless.yml

service: test 
plugins:
  - serverless-python-requirements
custom:
  pythonRequirements:
    zip: true                  
    dockerFile: Dockerfile      
    dockerizePip: non-linux     
provider:
  name: aws
  runtime: python3.6
  stage: dev
  environment: ${file(./env.yml):${opt:stage, self:provider.stage}.env}
  region: ${file(./env.yml):${opt:stage, self:provider.stage}.aws.region}
  profile: ${file(./env.yml):${opt:stage, self:provider.stage}.aws.profile}
package:
  individually: true
functions:
  f1:
    handler:index.handler
    module:functions/f1
  f2:
    handler:index.handleer
    module:functions/f2

我的项目文件位于 C:\Serverless\test 中。我运行npm init,然后运行npm i --save serverless-python-requirements,接受所有默认值。我在sls deploy -v 上得到以下信息。即使我已将 C:\ 添加到 VirtualBox 中正在运行的默认 VM 上的共享文件夹中,并选择了自动挂载和永久挂载。

如果我将 dockerizePip 和 dockerFile 都注释掉,我会根据 here 和其他 SO 帖子得到以下预期:

 Serverless: Invoke invoke
{
    "errorMessage": "Unable to import module 'index'"
}

如果我注释掉 dockerfile,我会得到:

Serverless: Docker Image: lambci/lambda:build-python3.6

      Error --------------------------------------------------

    error during connect: Get https://XXXXXX/v1.37/version: dial tcp
    XXXXXXXXXX: connectex: A connection attempt failed because the 
    connected party did not properly respond after a period of time, or
    established connection failed because connected host has failed to 
    respond.

    at dockerCommand (C:\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:20:11)
    at getBindPath (C:\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:100:3)

使用 Dockerfile

# AWS Lambda execution environment is based on Amazon Linux 1
FROM amazonlinux:1

# Install Python 3.6
RUN yum -y install python36 python36-pip

# Install your dependencies
RUN curl -s https://bootstrap.pypa.io/get-pip.py | python3
RUN yum -y install python3-devel mysql-devel gcc

# Set the same WORKDIR as default image
RUN mkdir /var/task
WORKDIR /var/task

.

Serverless: Building custom docker image from Dockerfile...
Serverless: Docker Image: sls-py-reqs-custom

  Error --------------------------------------------------

  Unable to find good bind path format

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: Unable to find good bind path format
    at getBindPath (C:\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:142:9)
    at installRequirements (C:\Serverless\test\node_modules\serverless-python-requirements\lib\pip.js:152:7)
    at installRequirementsIfNeeded (C:\Serverless\test\node_modules\serverless-python-requirements\lib\pip.js:451:3)

如果我将我的项目移动到 C:\Users\,我会得到这个:

Serverless: Docker Image: sls-py-reqs-custom
Serverless: Trying bindPath /c/Users/Serverless/test/.serverless/requirements (run,--rm,-v,/c/Users/Serverless/test/.serverless/req
uirements:/test,alpine,ls,/test/requirements.txt)
Serverless: /test/requirements.txt

  Error --------------------------------------------------

  docker: Error response from daemon: create "/c/Users/Serverless/test/.serverless/requirements": "\"/c/Users/Serverless/test/.serv
erless/requirements\"" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you in
tended to pass a host directory, use absolute path.
See 'docker run --help'.


     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: docker: Error response from daemon: create "/c/Users/Serverless/test/.serverless/requirements": "\"/c/Users/Serverless/test/
.serverless/requirements\"" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If y
ou intended to pass a host directory, use absolute path.
See 'docker run --help'.

    at dockerCommand (C:\Users\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:20:11)
    at getDockerUid (C:\Users\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:162:14)

我已经看到来自@brianz here 的 Makefile 风格推荐,但我不确定如何适应它(Makefile 不是我的强项)。我对下一步该怎么做有点茫然,我们将不胜感激。 TIA。

【问题讨论】:

    标签: python windows numpy lambda serverless-framework


    【解决方案1】:

    我无法使插件工作,但无论如何我找到了一个更好的解决方案 - Lambda 层。这是一个好处,因为它减小了 lambda 的大小并允许代码/文件重用。您可以使用一个用于 numpy 和 scipy 的预构建 lambda 层,但我构建了自己的层以向自己展示它是如何工作的。以下是我的工作方式:

    创建图层包:

    1. 打开 EC2 实例或 Ubuntu 或 Linux 或其他任何东西 - 这是需要的,以便我们可以正确编译运行时二进制文件
    2. 制作一个依赖包zip - 运行时必须使用目录结构python/lib/python3.6/site-packages供python查找

      mkdir -p tmpdir/python/lib/python3.6/site-packages 
      pip install -r requirements.txt --no-deps -t tmpdir/python/lib/python3.6/site-packages 
      cd tmpdir zip -r ../py_dependencies.zip . 
      cd .. 
      rm -r tmpdir
      
    3. 将层 zip 推送到 AWS - 需要最新的 awscli

      sudo pip install awscli --upgrade --user
      sudo aws lambda publish-layer-version \
      --layer-name py_dependencies \
      --description "Python 3.6 dependencies [numpy=0.15.4]" \
      --license-info "MIT" \
      --compatible-runtimes python3.6 \
      --zip-file fileb://py_dependencies.zip \
      --profile python_dev_serverless
      
    4. 要在任何需要 numpy 的函数中使用,只需使用控制台中显示的 arn 或在上面的上传过程中

      f1:
        handler: index.handler_f_use_numpy
        include:
          - functions/f_use_numpy.py
        layers:
          - arn:aws:lambda:us-west-2:XXXXX:layer:py_dependencies:1
      
    5. 另外,您还可以将常量等常用文件推送到图层。以下是我在 Windows 和 lambda 上测试使用的方法:

      import platform
      
      \# Set common path
      COMMON_PATH = "../../layers/common/"
      if platform.system() == "Linux": COMMON_PATH = "/opt/common/"
      
      def handler_common(event, context):
          # Read from a constants.json file
          with open(COMMON_PATH + 'constants.json') as f:
              return text = json.load(f)
      

    【讨论】:

      【解决方案2】:

      当我遇到同样的问题时,我打开 docker 去设置/共享驱动器选择重置凭据,并在应用我的更改后清除错误

      【讨论】:

        【解决方案3】:

        我通过暂时禁用 Windows 防火墙解决了这个问题。

        【讨论】:

          猜你喜欢
          • 2021-05-09
          • 2023-03-31
          • 1970-01-01
          • 2013-01-06
          • 1970-01-01
          • 1970-01-01
          • 1970-01-01
          • 1970-01-01
          • 2017-01-10
          相关资源
          最近更新 更多