【发布时间】：2016-03-01 14:15:47
【问题描述】：
我正在使用命令行工具将文件从 Amazon S3 上传到 CloudSearch 以使用命令进行索引-
cs-import-documents -d searchdev3 --source s3://mybucket/html
报错-
连接到区域 [us-east-1] 中的 CloudSearch
AWS 身份验证需要有效的 Date 或 x-amz-date 标头（服务：Amazon S 3；状态码：403；错误代码：拒绝访问；请求 ID：506FD29F73XXXXXX)
如何跟踪问题发生的位置？
更新
当使用--verbose 运行时，我收到以下输出-
Looking up DocServiceEndpoint for domain searchdev3 using config service at null
Connecting to CloudSearch in region [us-east-1]
Sending to endpoint [https://cloudsearch.us-east-1.amazonaws.com/] for region [u
s-east-1]
Connection request: [route: {s}->https://cloudsearch.us-east-1.amazonaws.com][to
tal kept alive: 0; route allocated: 0 of 50; total allocated: 0 of 50]
Connection leased: [id: 0][route: {s}->https://cloudsearch.us-east-1.amazonaws.c
om][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
Connecting to cloudsearch.us-east-1.amazonaws.com:443
CookieSpec selected: best-match
Auth cache not set in the context
Proxy auth state: UNCHALLENGED
Sending request: POST / HTTP/1.1
>> "POST / HTTP/1.1[\r][\n]"
>> "Host: cloudsearch.us-east-1.amazonaws.com[\r][\n]"
>> "Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXX/20151130/us-
east-1/cloudsearch/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signa
ture=e93cd0c22189084ff55c1084[\r][\n]"
>> "X-Amz-Date: 20151130T103603Z[\r][\n]"
>> "User-Agent: aws-internal/3[\r][\n]"
>> "Content-Type: application/x-www-form-urlencoded; charset=utf-8[\r][\n]"
>> "Content-Length: 73[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "[\r][\n]"
>> POST / HTTP/1.1
>> Host: cloudsearch.us-east-1.amazonaws.com
>> Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXX/20151130/us-e
ast-1/cloudsearch/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signat
ure=e93cd0c22189084ff55c1084
>> X-Amz-Date: 20151130T103603Z
>> User-Agent: aws-internal/3
>> Content-Type: application/x-www-form-urlencoded; charset=utf-8
>> Content-Length: 73
>> Connection: Keep-Alive
>> "Action=DescribeDomains&Version=2013-01-01&DomainNames.member.1=searchdev3"
<< "HTTP/1.1 200 OK[\r][\n]"
<< "x-amzn-RequestId: 2907d340-974e-11e5-a6e5-33953f55803d[\r][\n]"
<< "Content-Type: text/xml[\r][\n]"
<< "Content-Length: 1635[\r][\n]"
<< "Date: Mon, 30 Nov 2015 10:36:05 GMT[\r][\n]"
<< "[\r][\n]"
Receiving response: HTTP/1.1 200 OK
<< HTTP/1.1 200 OK
<< x-amzn-RequestId: 2907d340-974e-11e5-a6e5-33953f55803d
<< Content-Type: text/xml
<< Content-Length: 1635
<< Date: Mon, 30 Nov 2015 10:36:05 GMT
<< "<"
<< "D"
<< "e"
<< "s"
<< "cribeDomainsResponse xmlns=""
<< "http://cloudsearch.amazonaws.com/doc/2013-01-01/">[\n]"
<< " <DescribeDomainsResult>[\n]"
<< " <DomainStatusList>[\n]"
<< " <member>[\n]"
<< " <Created>true</Created>[\n]"
<< " <RequiresIndexDocuments>false</RequiresIndexDocuments>[\n]"
<< " <SearchInstanceType>search.m1.small</SearchInstanceType>[\n]"
<< " <DomainId>613875336225/searchdev3</DomainId>[\n]"
<< " <DocService>[\n]"
<< " <Endpoint>doc-searchdev3-ytmgti5ijnobqprgg65e4yxrhe.us-east-1.clou
dsearch.amazonaws.com</Endpoint>[\n]"
<< " </DocService>[\n]"
<< " <SearchPartitionCount>1</SearchPartitionCount>[\n]"
<< " <Deleted>false</Deleted>[\n]"
<< " <SearchService>[\n]"
<< " <Endpoint>search-searchdev3-ytmgti5ijnobqprgg65e4yxrhe.us-east-1.c
loudsearch.amazonaws.com</Endpoint>[\n]"
<< " </SearchService>[\n]"
<< " <DomainName>searchdev3</DomainName>[\n]"
<< " <Processing>true</Processing>[\n]"
<< " <SearchInstanceCount>1</SearchInstanceCount>[\n]"
<< " <ARN>arn:aws:cloudsearch:us-east-1:613875336225:domain/searchdev3</A
RN>[\n]"
<< " <Limits>[\n]"
<< " <AvailableInstanceTypes>[\n]"
<< " <member>search.m1.small</member>[\n]"
<< " <member>search.m3.medium</member>[\n]"
<< " <member>search.m3.large</member>[\n]"
<< " <member>search.m3.xlarge</member>[\n]"
<< " <member>search.m3.2xlarge</member>[\n]"
<< " </AvailableInstanceTypes>[\n]"
<< " <MaximumReplicationCount>5</MaximumReplicationCount>[\n]"
<< " <MaximumPartitionCount>10</MaximumPartitionCount>[\n]"
<< " </Limits>[\n]"
<< " </member>[\n]"
<< " </DomainStatusList>[\n]"
<< " </DescribeDomainsResult>[\n]"
<< " <ResponseMetadata>[\n]"
<< " <RequestId>2907d340-974e-11e5-a6e5-33953f55803d</RequestId>[\n]"
<< " </ResponseMetadata>[\n]"
<< "</DescribeDomainsResponse>[\n]"
Connection [id: 0][route: {s}->https://cloudsearch.us-east-1.amazonaws.com] can
be kept alive indefinitely
Connection released: [id: 0][route: {s}->https://cloudsearch.us-east-1.amazonaws
.com][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
Creating S3 source reader
Connection request: [route: {s}->https://mybucket.s3.amazonaws.com][total
kept alive: 0; route allocated: 0 of 50; total allocated: 0 of 50]
Connection leased: [id: 1][route: {s}->https://mybucket.s3.amazonaws.com]
[total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
Connecting to mybucket.s3.amazonaws.com:443
CookieSpec selected: best-match
Auth cache not set in the context
Proxy auth state: UNCHALLENGED
Sending request: GET /?prefix=html HTTP/1.1
>> "GET /?prefix=html HTTP/1.1[\r][\n]"
>> "Host: mybucket.s3.amazonaws.com[\r][\n]"
>> "Authorization: AWS XXXXXXXX:eA+KR8i3SpRQdTS/pxeBeci7iOw=[\r][\n]
"
>> "User-Agent: aws-internal/3[\r][\n]"
>> "Date: Mon, 30 Nov 2015 10:36:05 +00:00[\r][\n]"
>> "Content-Type: application/x-www-form-urlencoded; charset=utf-8[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "[\r][\n]"
>> GET /?prefix=html HTTP/1.1
>> Host: mybucket.s3.amazonaws.com
>> Authorization: AWS XXXXXXXX:eA+KR8i3SpRQdTS/pxeBeci7iOw=
>> User-Agent: aws-internal/3
>> Date: Mon, 30 Nov 2015 10:36:05 +00:00
>> Content-Type: application/x-www-form-urlencoded; charset=utf-8
>> Connection: Keep-Alive
<< "HTTP/1.1 403 Forbidden[\r][\n]"
<< "x-amz-bucket-region: us-east-1[\r][\n]"
<< "x-amz-request-id: DA6567BEA36C46F9[\r][\n]"
<< "x-amz-id-2: kCC6NxI43AJi3G+YB0Dxgoc6ZrCGh/V3Wc4nhSnxk+mxYpdH50am3BBgzXfkLHrj
[\r][\n]"
<< "Content-Type: application/xml[\r][\n]"
<< "Transfer-Encoding: chunked[\r][\n]"
<< "Date: Mon, 30 Nov 2015 10:36:06 GMT[\r][\n]"
<< "Server: AmazonS3[\r][\n]"
<< "[\r][\n]"
Receiving response: HTTP/1.1 403 Forbidden
<< HTTP/1.1 403 Forbidden
<< x-amz-bucket-region: us-east-1
<< x-amz-request-id: DA6567BEA36C46F9
<< x-amz-id-2: kCC6NxI43AJi3G+YB0Dxgoc6ZrCGh/V3Wc4nhSnxk+mxYpdH50am3BBgzXfkLHrj
<< Content-Type: application/xml
<< Transfer-Encoding: chunked
<< Date: Mon, 30 Nov 2015 10:36:06 GMT
<< Server: AmazonS3
<< "117[\r][\n]"
<< "<?xml version="1.0" encoding="UTF-8"?>[\n]"
<< "<Error><Code>AccessDenied</Code><Message>AWS authentication requires a valid
Date or x-amz-date header</Message><RequestId>DA6567BEA36C46F9</RequestId><Host
Id>kCC6NxI43AJi3G+YB0Dxgoc6ZrCGh/V3Wc4nhSnxk+mxYpdH50am3BBgzXfkLHrj</HostId></Er
ror>"
<< "[\r][\n]"
<< "0[\r][\n]"
<< "[\r][\n]"
Connection [id: 1][route: {s}->https://mybucket.s3.amazonaws.com] can be
kept alive indefinitely
Connection released: [id: 1][route: {s}->https://mybucket.s3.amazonaws.co
m][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S
3; Status Code: 403; Error Code: AccessDenied; Request ID: DA6567BEA36C46F9)
Error Type: Client
Error Code: AccessDenied
HTTP Code: 403
Request ID: DA6567BEA36C46F9
[Ljava.lang.StackTraceElement;@12a7737
【问题讨论】：
-
您可以使用
--verbose运行它以获取一些额外的日志记录并发布输出吗？ -
有趣的是，它在第一个请求 (POST /) 中包含 x-amz-date，但在第二个请求 (GET /prefix=html) 中不包含。我反编译了 jar 以查看发生了什么，但找不到应该添加标头的位置。
cs-import-documents只需调用cs-cmd并使用“CSImportDocuments”作为参数，它是 CloudSearchTools-1.0.jar 中的一个类（在您安装工具的 /lib 中）。您可以尝试在那里闲逛或在 AWS 支持论坛上发帖。 -
@kamalpreet - 你搞定了吗？我在使用非常相似的 cs-import 命令时遇到了同样的问题，当我引用 s3:// 作为源时，我得到了同样的错误。我想我需要提供额外的授权参数，但不知道它们是什么？
标签： amazon-s3 amazon-cloudsearch