【发布时间】:2016-04-27 19:38:54
【问题描述】:
我正在尝试使用 aws sdk ruby 为使用 hadoop 的 Amazon ElasticMapReduce 服务运行一个步骤,虽然我可以创建集群和步骤,但该步骤总是失败,但在使用 Web 界面手动设置时不会失败
emr = Aws::EMR::Client.new
cluster_id = "*******"
resp = emr.add_job_flow_steps({
job_flow_id: cluster_id, # required
steps: [ # required
{
name: "TestStep", # required
action_on_failure: "CANCEL_AND_WAIT", # accepts TERMINATE_JOB_FLOW, TERMINATE_CLUSTER, CANCEL_AND_WAIT, CONTINUE
hadoop_jar_step: { # required
jar: 'command-runner.jar',
args:[
"-files",
"s3://source123/mapper.py,s3://source123/source_reducer.py",
"-mapper",
"mapper.py",
"-reducer",
"source_reducer.py",
"-input",
"s3://source123/input/",
"-output",
"s3://source123/output/"
]
},
},
],
})
我得到的错误是这样的
Cannot run program "-files" (in directory "."): error=2, No such file or directory
有什么线索吗?
【问题讨论】:
标签: ruby-on-rails ruby hadoop amazon-web-services emr