【发布时间】：2023-02-06 17:24:03
【问题描述】：
我们正在尝试为自定义模型运行批量预测。
培训是在本教程之后完成的： https://codelabs.developers.google.com/codelabs/vertex-ai-custom-code-training#4
在管道中提交作业的代码：
model = aiplatform.Model(model_path)
batch_prediction_job = model.batch_predict(
gcs_source=gcs_source,
gcs_destination_prefix=gcs_destination,
machine_type='n1-standard-4',
instances_format='csv',
sync=False
)
运行批量预测作业失败，管道中出现以下错误：
JobState.JOB_STATE_FAILED
[KFP Executor 2023-01-18 14:08:09,862 INFO]: BatchPredictionJob projects/472254905662/locations/us-central1/batchPredictionJobs/3522181183414730752 current state:
JobState.JOB_STATE_FAILED
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/site-packages/kfp/v2/components/executor_main.py", line 104, in <module>
executor_main()
File "/usr/local/lib/python3.7/site-packages/kfp/v2/components/executor_main.py", line 100, in executor_main
executor.execute()
File "/usr/local/lib/python3.7/site-packages/kfp/v2/components/executor.py", line 309, in execute
result = self._func(**func_kwargs)
File "/tmp/tmp.ZqplJAZqqL/ephemeral_component.py", line 23, in create_batch_inference_component
print(f'Batch prediction job "{batch_prediction_job.resource_name}" submitted')
File "/usr/local/lib/python3.7/site-packages/google/cloud/aiplatform/base.py", line 676, in resource_name
self._assert_gca_resource_is_available()
File "/usr/local/lib/python3.7/site-packages/google/cloud/aiplatform/base.py", line 1324, in _assert_gca_resource_is_available
else ""
RuntimeError: BatchPredictionJob resource has not been created.
失败的批量预测作业中存在错误，但无法理解其含义：
Batch prediction job BatchPredictionJob 2023-01-18 14:21:50.490123 encountered the following errors:
Model server terminated: model server container terminated: exit_code: 1 reason: "Error" started_at { seconds: 1674052639 } finished_at { seconds: 1674052640 }
为相同的泰坦尼克号数据集训练的 AutoML 模型的批量预测有效。
没有办法解决这个问题。我们尝试了不同的instance_format，而不是指定machine_type，改进了预测数据集（指南说所有字符串字段都应该用双引号括起来）但这并没有停止。
【问题讨论】：
-
嗨@havryliuk，你能提供你试过的示例数据和示例代码吗？
-
您好@ShipraSarkar，您是指包含数据准备和模型训练的整个代码吗？我已经分享了我们提交自定义模型批量预测的代码。
-
嗨@havryliuk，你能用不同的数据集重试吗？
-
@ShipraSarkar，我们尝试使用正确格式的 json（实例 json）而不是 csv，因为这是一个 scikit-learn 模型，但错误是相同的：
RuntimeError: BatchPredictionJob resource has not been created. -
嗨@havryliuk，看来这个问题需要进一步调查，所以如果你有支持计划，请创建一个新的GCP support case。否则，您可以在Issue Tracker 上开一个新问题。