【问题标题】:Where are the entities after uploading data?上传数据后的实体在哪里?
【发布时间】:2014-07-06 17:44:01
【问题描述】:

我正在尝试将我的 csv 文件导入 appengine 数据存储区,但没有收到任何错误,但实体在哪里?我在数据存储部分的 appengine 控制台中看不到它。

models.py:

from google.appengine.ext import db

class Glyph(db.Model):
    glyphId = db.IntegerProperty(indexed=True)
    pageNumber = db.IntegerProperty()
    lineNumber = db.IntegerProperty()
    position = db.IntegerProperty()
    min_x = db.IntegerProperty()
    max_x = db.IntegerProperty()
    min_y = db.IntegerProperty()
    max_y = db.IntegerProperty()
    resolution = db.IntegerProperty()

bulkloader.yaml:

python_preamble:
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: google.appengine.ext.db
- import: google.appengine.api.datastore

transformers:
- kind: Glyph
  connector: csv
  connector_options:
    encoding: utf-8
    columns: from_header
  property_map:
  - property: glyph_id
    external_name: glyph_id

  - property: page_number
    external_name: page_number

  - property: line_number
    external_name: line_number

  - property: position
    external_name: position

  - property: min_x
    external_name: min_x

  - property: max_x
    external_name: max_x

  - property: min_y
    external_name: min_y

  - property: max_y
    external_name: max_y

  - property: resolution
    external_name: resolution

字形.csv:

glyph_id,page_number,line_number,position,min_x,max_x,min_y,max_y,resolution
211,1,2,1,188,217,43,58,320


Environment: cygwin
Appengine version: 1.9.6
$ appcfg.py upload_data --config_file=bulkloader.yaml --filename=glyphs.csv --kind Glyph --url=http://webservice.appspot.com/_ah/remote_api --email=email@gmail.com --batch_size=1000 --rps_limit=500

提前致谢。

12:04 PM Uploading data records.
[INFO    ] Logging to bulkloader-log-20140704.120441
[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 500/second
[INFO    ] Batch Size: 1000
[INFO    ] Opening database: bulkloader-progress-20140704.120441.sql3
[INFO    ] Connecting to webservice.appspot.com/_ah/remote_api
[INFO    ] Starting import; maximum 1000 entities per post
.........................................................................................
[INFO    ] 88246 entities total, 0 previously transferred
[INFO    ] 88246 entities (85210 bytes) transferred in 14.2 seconds
[INFO    ] All entities successfully transferred

日志文件:

[DEBUG    2014-07-04 18:41:21,390 adaptive_thread_pool.py] Waiting for Anonymous_WorkerThread to exit
[DEBUG    2014-07-04 18:41:21,390 bulkloader.py] Waiting for progress_thread to terminate...
[DEBUG    2014-07-04 18:41:21,453 bulkloader.py] [Thread-11] ProgressTrackerThread: exiting
[DEBUG    2014-07-04 18:41:21,468 bulkloader.py] ... done.
[INFO     2014-07-04 18:41:21,468 bulkloader.py] 88246 entities total, 0 previously transferred
[INFO     2014-07-04 18:41:21,468 bulkloader.py] 88246 entities (85210 bytes) transferred in 16.2 seconds
[INFO     2014-07-04 18:41:21,468 bulkloader.py] All entities successfully transferred

控制台日志:

2014-07-06 17:07:44.290 /_ah/remote_api 200 19ms 0kb Google-remote_api/1.0 CYGWIN_NT-5.1/1.7.30(0.272/5/3) Python/2.7.3.final.0 gzip module=default version=1
69.247.153.191 - admin [06/Jul/2014:17:07:44 -0700] "POST /_ah/remote_api HTTP/1.1" 200 92 - "Google-remote_api/1.0 CYGWIN_NT-5.1/1.7.30(0.272/5/3) Python/2.7.3.final.0 gzip" "webservice.appspot.com" ms=19 cpu_ms=0 cpm_usd=0.000010 app_engine_release=1.9.6 trace_id=15aab71128af2f7dbbd629ef8fe24297 instance=00c61b117cc1848cde8da4b66f619fbabd1234

解决方案: 事实证明,我的 bulkloader.yaml 的 property_map 部分中缺少列。修复映射解决了这个问题。

【问题讨论】:

    标签: google-app-engine python-2.7 csv google-cloud-datastore bulkloader


    【解决方案1】:

    事实证明,我的 bulkloader.yaml 的 property_map 部分中缺少列。修复映射解决了这个问题。解决方案已添加到原始帖子中。

    【讨论】:

      【解决方案2】:

      数据会按照您的 bulkloader.yaml 中的定义放入您的模型中。要快速查看它们,请转到您的开发者控制台,转到 Datastore Viewer:

      https://appengine.google.com/datastore/explorer?&app_id=s~yourappid

      您可以使用下拉菜单从一个模型转到另一个模型。在你的情况下,你只上传了Glyphs,所以看看那里。

      【讨论】:

      • 感谢@GAEfan。我已经做了,它没有显示任何东西。 “糟糕!我们无法检索您的种类列表。请稍后再试。”对 namespace 的查询也是空的。是否需要通过控制台创建一个实体才能导入数据?
      • 你有一个models.py,定义了Glyph吗?
      • 是的,我愿意。 models.py: from google.appengine.ext import db class Glyph(db.Model): glyphId = db.IntegerProperty() pageNumber = db.IntegerProperty() lineNumber = db.IntegerProperty() position = db.IntegerProperty() min_x = db.IntegerProperty() max_x = db.IntegerProperty() min_y = db.IntegerProperty() max_y = db.IntegerProperty() 分辨率 = db.IntegerProperty()
      • 您的应用程序 ID 是“webservice”吗?那是他们上传的地方。而且您的 glyphs.csv 不是 csv 文件,因此不确定它是如何上传 18k 个项目而没有错误的。您应该尝试将 --application=s~yourappname 标志添加到 appcfg.py 命令
      • 修复 csv 文件并替换所有“|”到 "," 但实体仍然没有出现。我尝试添加 --applicon=s~webservice,但在执行 upload_data 命令时出现异常:[INFO 2014-07-04 18:40:10,687 bulkloader.py] Connecting to webservice.appspot.com/_ah/remote_api [ERROR 2014-07-04 18:40:10,687 bulkloader.py] 身份验证期间出现异常 URLError: [ERROR 2014-07-04 18:40:10,687 bulkloader.py] 身份验证失败:不正确的凭据或不受支持的身份验证类型(例如 OpenId)。
      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2019-09-27
      • 2023-03-09
      • 1970-01-01
      • 2011-02-14
      • 1970-01-01
      相关资源
      最近更新 更多