wget 返回 200/OK 而 org.apache.http.impl.client.CloseableHttpClient 返回 404/NOT_FOUND

【问题标题】：wget returns 200/OK while org.apache.http.impl.client.CloseableHttpClient returns 404/NOT_FOUNDwget 返回 200/OK 而 org.apache.http.impl.client.CloseableHttpClient 返回 404/NOT_FOUND
【发布时间】：2019-10-29 02:26:28
【问题描述】：

我正在尝试使用 http-components/java 读取此 URL ("https://www.encodeproject.org/files/ENCFF784GFP/@@download/ENCFF784GFP.hic")。

使用 wget 可以正常工作：

wget -S -q -O - "https://www.encodeproject.org/files/ENCFF784GFP/@@download/ENCFF784GFP.hic" > /dev/null 
(...)
  HTTP/1.1 200 OK
  x-amz-id-2: 5vWTJmx5PF0uyuR8cUWwtx9zOei2ewX+yskiLlqKMFJOzkBKP/ND3pTKdC2/9+WYjqTIGuQ/Cp4=
  x-amz-request-id: B447183CF808894B
  Date: Fri, 14 Jun 2019 12:29:39 GMT
  Last-Modified: Sat, 09 Feb 2019 09:14:26 GMT
  ETag: "ccaabdcb885389d618bfe826e67fe680-521"
  Content-Disposition: attachment; filename=ENCFF784GFP.hic
  Accept-Ranges: bytes
  Content-Type: binary/octet-stream
  Content-Length: 4367743691
  Server: AmazonS3
(...)

当我得到一个带有以下 java 代码的 404 时：

(...)
        final HttpClientBuilder hb = HttpClients.custom();
        final CloseableHttpClient httpClient = hb.build();

        final HttpHead httpHead = new HttpHead(url.toExternalForm());


            final CloseableHttpResponse response = httpClient.execute(httpHead);
            if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK ) {

                final String msg = "Unexpected Http status code "
                        + response.getStatusLine()+" for \""+ url+"\""
                        ;
                response.close();
                httpClient.close();
                throw new IOException(msg);
                }
(...)

使用以下日志：

2019/06/14 14:46:47:780 CEST [DEBUG] RequestAddCookies - CookieSpec selected: default
2019/06/14 14:46:47:794 CEST [DEBUG] RequestAuthCache - Auth cache not set in the context
2019/06/14 14:46:47:796 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://www.encodeproject.org:443][total kept alive: 0; rou
te allocated: 0 of 2; total allocated: 0 of 20]
2019/06/14 14:46:47:814 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {s}->https://www.encodeproject.org:443][total kept alive: 
0; route allocated: 1 of 2; total allocated: 1 of 20]
2019/06/14 14:46:47:815 CEST [DEBUG] MainClientExec - Opening connection {s}->https://www.encodeproject.org:443
2019/06/14 14:46:47:844 CEST [DEBUG] DefaultHttpClientConnectionOperator - Connecting to www.encodeproject.org/34.211.244.144:443
2019/06/14 14:46:47:845 CEST [DEBUG] SSLConnectionSocketFactory - Connecting socket to www.encodeproject.org/34.211.244.144:443 with timeout 0
2019/06/14 14:46:48:082 CEST [DEBUG] SSLConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
2019/06/14 14:46:48:082 CEST [DEBUG] SSLConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA38
4, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WIT
H_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS
_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_12
8_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_
DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_C
BC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDS
A_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_G
CM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_
ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION
_INFO_SCSV]
2019/06/14 14:46:48:082 CEST [DEBUG] SSLConnectionSocketFactory - Starting handshake
2019/06/14 14:46:48:500 CEST [DEBUG] SSLConnectionSocketFactory - Secure session established
2019/06/14 14:46:48:500 CEST [DEBUG] SSLConnectionSocketFactory -  negotiated protocol: TLSv1.2
2019/06/14 14:46:48:501 CEST [DEBUG] SSLConnectionSocketFactory -  negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
2019/06/14 14:46:48:502 CEST [DEBUG] SSLConnectionSocketFactory -  peer principal: CN=*.encodeproject.org, OU=Domain Control Validated
2019/06/14 14:46:48:503 CEST [DEBUG] SSLConnectionSocketFactory -  peer alternative names: [*.encodeproject.org, encodeproject.org]
2019/06/14 14:46:48:503 CEST [DEBUG] SSLConnectionSocketFactory -  issuer principal: CN=AlphaSSL CA - SHA256 - G2, O=GlobalSign nv-sa, C=BE
2019/06/14 14:46:48:507 CEST [DEBUG] DefaultHttpClientConnectionOperator - Connection established 172.18.240.255:59598<->34.211.244.144:443
2019/06/14 14:46:48:508 CEST [DEBUG] MainClientExec - Executing request HEAD /files/ENCFF784GFP/@@download/ENCFF784GFP.hic HTTP/1.1
2019/06/14 14:46:48:508 CEST [DEBUG] MainClientExec - Target auth state: UNCHALLENGED
2019/06/14 14:46:48:509 CEST [DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
2019/06/14 14:46:48:510 CEST [DEBUG] headers - http-outgoing-0 >> HEAD /files/ENCFF784GFP/@@download/ENCFF784GFP.hic HTTP/1.1
2019/06/14 14:46:48:510 CEST [DEBUG] headers - http-outgoing-0 >> Host: www.encodeproject.org
2019/06/14 14:46:48:510 CEST [DEBUG] headers - http-outgoing-0 >> Connection: Keep-Alive
2019/06/14 14:46:48:511 CEST [DEBUG] headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5 (Java/1.8.0_191)
2019/06/14 14:46:48:511 CEST [DEBUG] headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
2019/06/14 14:46:48:688 CEST [DEBUG] headers - http-outgoing-0 << HTTP/1.1 307 Temporary Redirect
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << Server: nginx/1.14.1
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << Date: Fri, 14 Jun 2019 12:31:57 GMT
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << Content-Type: text/plain; charset=UTF-8
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << Content-Length: 1569
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << Connection: keep-alive
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << X-Request-URL: https://www.encodeproject.org/files/ENCFF784GFP/@@download/ENCFF784GFP.hic
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << X-Stats: es_count=1&es_time=1918&queue_begin=1560515517514062&queue_time=398&rss_begin=398917632&rss
_change=0&rss_end=398917632&wsgi_begin=1560515517514460&wsgi_end=1560515517522664&wsgi_time=8204
2019/06/14 14:46:48:689 CEST [DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Origin: *
2019/06/14 14:46:48:690 CEST [DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Methods: GET, HEAD
2019/06/14 14:46:48:690 CEST [DEBUG] headers - http-outgoing-0 << Access-Control-Allow-Headers: Accept, Origin, Range, X-Requested-With
2019/06/14 14:46:48:690 CEST [DEBUG] headers - http-outgoing-0 << Access-Control-Expose-Headers: Content-Length, Content-Range, Content-Type
2019/06/14 14:46:48:690 CEST [DEBUG] headers - http-outgoing-0 << Location: https://download.encodeproject.org/https://encode-public.s3.amazonaws.com/2018/04/06/8723e
e96-ccee-4384-b5a8-4040262283eb/ENCFF784GFP.hic?response-content-disposition=attachment%3B%20filename%3DENCFF784GFP.hic&Signature=YZmRQyN1SPT0vpRjyscTzLgV3WM%3D&x-amz
-security-token=AgoJb3JpZ2luX2VjEMT%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIBECuWysenOZdkDKbB6XfmcaQ8zul6azEkuNauJcIB91AiEA6%2FLuT9kpu%2BIvggBn3Dp7mJBxg9
2zGio1NlitzdjrJ08q4wMI7f%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDCYx83zFw7htd3syviq3AzISSluwceByIAxIcFaxcorsp2pKt8ZqsPUcgjsIeWJR2CaHnhyxGLXV3ydF8C4CmxYY
bgepXvPKTYy6i0UASGK8YJf2mPlzGoXjdiRZo%2BpjTJmhVO59CYgV%2B%2BoJqHYOkYqEotyfNT%2BOYxaMum0qzeOfpWjrApXqNyrBkkrowu6Ng85eDNjmZTO0DZWGJICZm%2FiBEi%2B6sewE3aGBLdTGiaKhmbDsKj
uNUNcBUwZfYfjBt0OwAASJZrC6C%2BjVSfiLjgBOaeY8pXqKmZ1X218AjX6vVzCSnriYvsLY1fmgCdu66ifTUU5GL5NqyDmS76UVO1PoviFSctLBU%2B%2BJtQBWwk%2BcoLS0oIU1%2FhtnoHYKH4M%2B2ne3z1w5i86L
1zsdUMXqZQ8AZ1A03iCUhQzCnDo%2Bo04njmhlF08voTZKVHbZ2icSEiKYERboR%2BMIhicYDtDhkRCcTZooP55A%2FSD9Ov4YtkwmpRgoBRMCYkiNl4rLD6MNyGipFJdFeoHllVZCfbMSjtOwmZRcOdKzZuJ%2FCDZhgP
njbzGc0oe%2Fp0QpXfYHNN00g6XEQdleW%2BcSGIDx7hMzuZydGxWOf7IwlpuO6AU6tAF4294kivKpwT90N%2FCA%2Fdquas6GmSMLpRWET%2FGcjgu9mhHXH0cZ3OOFHmWs%2Bo4e5o2i7geA7yCALBw4aIf%2BPpRUjR
EIx9IOPzMJHtKqDX8BLgFnjrvR2FXeiCnW%2BN0YxXGDP%2BRrlxLDkxBVbH6afcfV7ysqHIAwkWTUNlz7wKkCy4aIAfB3mPqMp3WsJ5KDx7xSNDfVe1kvsmNv8%2FIWed1K3KlnjG5VVbH2IklxMBJw5%2FYqSaQ%3D&E
xpires=1560645117&AWSAccessKeyId=ASIATGZNGCNXXYADQSVU
2019/06/14 14:46:48:690 CEST [DEBUG] headers - http-outgoing-0 << Strict-Transport-Security: max-age=15768000
2019/06/14 14:46:48:694 CEST [DEBUG] MainClientExec - Connection can be kept alive indefinitely
2019/06/14 14:46:48:695 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection [id: 0][route: {s}->https://www.encodeproject.org:443] can be kept alive indefini
tely
2019/06/14 14:46:48:695 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {s}->https://www.encodeproject.org:443][total kept alive
: 1; route allocated: 1 of 2; total allocated: 1 of 20]
2019/06/14 14:46:48:698 CEST [DEBUG] DefaultRedirectStrategy - Redirect requested to location 'https://download.encodeproject.org/https://encode-public.s3.amazonaws.c
om/2018/04/06/8723ee96-ccee-4384-b5a8-4040262283eb/ENCFF784GFP.hic?response-content-disposition=attachment%3B%20filename%3DENCFF784GFP.hic&Signature=YZmRQyN1SPT0vpRjy
scTzLgV3WM%3D&x-amz-security-token=AgoJb3JpZ2luX2VjEMT%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIBECuWysenOZdkDKbB6XfmcaQ8zul6azEkuNauJcIB91AiEA6%2FLuT9kpu
%2BIvggBn3Dp7mJBxg92zGio1NlitzdjrJ08q4wMI7f%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDCYx83zFw7htd3syviq3AzISSluwceByIAxIcFaxcorsp2pKt8ZqsPUcgjsIeWJR2CaHn
hyxGLXV3ydF8C4CmxYYbgepXvPKTYy6i0UASGK8YJf2mPlzGoXjdiRZo%2BpjTJmhVO59CYgV%2B%2BoJqHYOkYqEotyfNT%2BOYxaMum0qzeOfpWjrApXqNyrBkkrowu6Ng85eDNjmZTO0DZWGJICZm%2FiBEi%2B6sew
E3aGBLdTGiaKhmbDsKjuNUNcBUwZfYfjBt0OwAASJZrC6C%2BjVSfiLjgBOaeY8pXqKmZ1X218AjX6vVzCSnriYvsLY1fmgCdu66ifTUU5GL5NqyDmS76UVO1PoviFSctLBU%2B%2BJtQBWwk%2BcoLS0oIU1%2FhtnoHY
KH4M%2B2ne3z1w5i86L1zsdUMXqZQ8AZ1A03iCUhQzCnDo%2Bo04njmhlF08voTZKVHbZ2icSEiKYERboR%2BMIhicYDtDhkRCcTZooP55A%2FSD9Ov4YtkwmpRgoBRMCYkiNl4rLD6MNyGipFJdFeoHllVZCfbMSjtOwm
ZRcOdKzZuJ%2FCDZhgPnjbzGc0oe%2Fp0QpXfYHNN00g6XEQdleW%2BcSGIDx7hMzuZydGxWOf7IwlpuO6AU6tAF4294kivKpwT90N%2FCA%2Fdquas6GmSMLpRWET%2FGcjgu9mhHXH0cZ3OOFHmWs%2Bo4e5o2i7geA7
yCALBw4aIf%2BPpRUjREIx9IOPzMJHtKqDX8BLgFnjrvR2FXeiCnW%2BN0YxXGDP%2BRrlxLDkxBVbH6afcfV7ysqHIAwkWTUNlz7wKkCy4aIAfB3mPqMp3WsJ5KDx7xSNDfVe1kvsmNv8%2FIWed1K3KlnjG5VVbH2Ikl
xMBJw5%2FYqSaQ%3D&Expires=1560645117&AWSAccessKeyId=ASIATGZNGCNXXYADQSVU'
2019/06/14 14:46:48:707 CEST [DEBUG] RedirectExec - Resetting target auth state
2019/06/14 14:46:48:708 CEST [DEBUG] RedirectExec - Redirecting to 'https://download.encodeproject.org/https:/encode-public.s3.amazonaws.com/2018/04/06/8723ee96-ccee-
4384-b5a8-4040262283eb/ENCFF784GFP.hic?response-content-disposition=attachment%3B%20filename%3DENCFF784GFP.hic&Signature=YZmRQyN1SPT0vpRjyscTzLgV3WM%3D&x-amz-security
-token=AgoJb3JpZ2luX2VjEMT%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIBECuWysenOZdkDKbB6XfmcaQ8zul6azEkuNauJcIB91AiEA6%2FLuT9kpu%2BIvggBn3Dp7mJBxg92zGio1Nli
tzdjrJ08q4wMI7f%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDCYx83zFw7htd3syviq3AzISSluwceByIAxIcFaxcorsp2pKt8ZqsPUcgjsIeWJR2CaHnhyxGLXV3ydF8C4CmxYYbgepXvPKT
Yy6i0UASGK8YJf2mPlzGoXjdiRZo%2BpjTJmhVO59CYgV%2B%2BoJqHYOkYqEotyfNT%2BOYxaMum0qzeOfpWjrApXqNyrBkkrowu6Ng85eDNjmZTO0DZWGJICZm%2FiBEi%2B6sewE3aGBLdTGiaKhmbDsKjuNUNcBUwZ
fYfjBt0OwAASJZrC6C%2BjVSfiLjgBOaeY8pXqKmZ1X218AjX6vVzCSnriYvsLY1fmgCdu66ifTUU5GL5NqyDmS76UVO1PoviFSctLBU%2B%2BJtQBWwk%2BcoLS0oIU1%2FhtnoHYKH4M%2B2ne3z1w5i86L1zsdUMXqZ
Q8AZ1A03iCUhQzCnDo%2Bo04njmhlF08voTZKVHbZ2icSEiKYERboR%2BMIhicYDtDhkRCcTZooP55A%2FSD9Ov4YtkwmpRgoBRMCYkiNl4rLD6MNyGipFJdFeoHllVZCfbMSjtOwmZRcOdKzZuJ%2FCDZhgPnjbzGc0oe
%2Fp0QpXfYHNN00g6XEQdleW%2BcSGIDx7hMzuZydGxWOf7IwlpuO6AU6tAF4294kivKpwT90N%2FCA%2Fdquas6GmSMLpRWET%2FGcjgu9mhHXH0cZ3OOFHmWs%2Bo4e5o2i7geA7yCALBw4aIf%2BPpRUjREIx9IOPzM
JHtKqDX8BLgFnjrvR2FXeiCnW%2BN0YxXGDP%2BRrlxLDkxBVbH6afcfV7ysqHIAwkWTUNlz7wKkCy4aIAfB3mPqMp3WsJ5KDx7xSNDfVe1kvsmNv8%2FIWed1K3KlnjG5VVbH2IklxMBJw5%2FYqSaQ%3D&Expires=15
60645117&AWSAccessKeyId=ASIATGZNGCNXXYADQSVU' via {s}->https://download.encodeproject.org:443
2019/06/14 14:46:48:710 CEST [DEBUG] RequestAddCookies - CookieSpec selected: default
2019/06/14 14:46:48:710 CEST [DEBUG] RequestAuthCache - Auth cache not set in the context
2019/06/14 14:46:48:710 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://download.encodeproject.org:443][total kept alive: 1
; route allocated: 0 of 2; total allocated: 1 of 20]
2019/06/14 14:46:48:711 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 1][route: {s}->https://download.encodeproject.org:443][total kept al
ive: 1; route allocated: 1 of 2; total allocated: 2 of 20]
2019/06/14 14:46:48:711 CEST [DEBUG] MainClientExec - Opening connection {s}->https://download.encodeproject.org:443
2019/06/14 14:46:48:890 CEST [DEBUG] DefaultHttpClientConnectionOperator - Connecting to download.encodeproject.org/34.211.244.144:443
2019/06/14 14:46:48:891 CEST [DEBUG] SSLConnectionSocketFactory - Connecting socket to download.encodeproject.org/34.211.244.144:443 with timeout 0
2019/06/14 14:46:49:048 CEST [DEBUG] SSLConnectionSocketFactory - Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]
2019/06/14 14:46:49:048 CEST [DEBUG] SSLConnectionSocketFactory - Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA38
4, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WIT
H_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS
_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_12
8_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_
DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_C
BC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDS
A_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_G
CM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_
ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION
_INFO_SCSV]
2019/06/14 14:46:49:048 CEST [DEBUG] SSLConnectionSocketFactory - Starting handshake
2019/06/14 14:46:49:388 CEST [DEBUG] SSLConnectionSocketFactory - Secure session established
2019/06/14 14:46:49:388 CEST [DEBUG] SSLConnectionSocketFactory -  negotiated protocol: TLSv1.2
2019/06/14 14:46:49:388 CEST [DEBUG] SSLConnectionSocketFactory -  negotiated cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
2019/06/14 14:46:49:388 CEST [DEBUG] SSLConnectionSocketFactory -  peer principal: CN=*.encodeproject.org, OU=Domain Control Validated
2019/06/14 14:46:49:388 CEST [DEBUG] SSLConnectionSocketFactory -  peer alternative names: [*.encodeproject.org, encodeproject.org]
2019/06/14 14:46:49:388 CEST [DEBUG] SSLConnectionSocketFactory -  issuer principal: CN=AlphaSSL CA - SHA256 - G2, O=GlobalSign nv-sa, C=BE
2019/06/14 14:46:49:389 CEST [DEBUG] DefaultHttpClientConnectionOperator - Connection established 172.18.240.255:59602<->34.211.244.144:443
2019/06/14 14:46:49:389 CEST [DEBUG] MainClientExec - Executing request HEAD /https:/encode-public.s3.amazonaws.com/2018/04/06/8723ee96-ccee-4384-b5a8-4040262283eb/EN
CFF784GFP.hic?response-content-disposition=attachment%3B%20filename%3DENCFF784GFP.hic&Signature=YZmRQyN1SPT0vpRjyscTzLgV3WM%3D&x-amz-security-token=AgoJb3JpZ2luX2VjEM
T%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIBECuWysenOZdkDKbB6XfmcaQ8zul6azEkuNauJcIB91AiEA6%2FLuT9kpu%2BIvggBn3Dp7mJBxg92zGio1NlitzdjrJ08q4wMI7f%2F%2F%2F%
2F%2F%2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDCYx83zFw7htd3syviq3AzISSluwceByIAxIcFaxcorsp2pKt8ZqsPUcgjsIeWJR2CaHnhyxGLXV3ydF8C4CmxYYbgepXvPKTYy6i0UASGK8YJf2mPlzGoXjdi
RZo%2BpjTJmhVO59CYgV%2B%2BoJqHYOkYqEotyfNT%2BOYxaMum0qzeOfpWjrApXqNyrBkkrowu6Ng85eDNjmZTO0DZWGJICZm%2FiBEi%2B6sewE3aGBLdTGiaKhmbDsKjuNUNcBUwZfYfjBt0OwAASJZrC6C%2BjVSf
iLjgBOaeY8pXqKmZ1X218AjX6vVzCSnriYvsLY1fmgCdu66ifTUU5GL5NqyDmS76UVO1PoviFSctLBU%2B%2BJtQBWwk%2BcoLS0oIU1%2FhtnoHYKH4M%2B2ne3z1w5i86L1zsdUMXqZQ8AZ1A03iCUhQzCnDo%2Bo04n
jmhlF08voTZKVHbZ2icSEiKYERboR%2BMIhicYDtDhkRCcTZooP55A%2FSD9Ov4YtkwmpRgoBRMCYkiNl4rLD6MNyGipFJdFeoHllVZCfbMSjtOwmZRcOdKzZuJ%2FCDZhgPnjbzGc0oe%2Fp0QpXfYHNN00g6XEQdleW%
2BcSGIDx7hMzuZydGxWOf7IwlpuO6AU6tAF4294kivKpwT90N%2FCA%2Fdquas6GmSMLpRWET%2FGcjgu9mhHXH0cZ3OOFHmWs%2Bo4e5o2i7geA7yCALBw4aIf%2BPpRUjREIx9IOPzMJHtKqDX8BLgFnjrvR2FXeiCnW
%2BN0YxXGDP%2BRrlxLDkxBVbH6afcfV7ysqHIAwkWTUNlz7wKkCy4aIAfB3mPqMp3WsJ5KDx7xSNDfVe1kvsmNv8%2FIWed1K3KlnjG5VVbH2IklxMBJw5%2FYqSaQ%3D&Expires=1560645117&AWSAccessKeyId=A
SIATGZNGCNXXYADQSVU HTTP/1.1
2019/06/14 14:46:49:389 CEST [DEBUG] MainClientExec - Target auth state: UNCHALLENGED
2019/06/14 14:46:49:389 CEST [DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
2019/06/14 14:46:49:390 CEST [DEBUG] headers - http-outgoing-1 >> HEAD /https:/encode-public.s3.amazonaws.com/2018/04/06/8723ee96-ccee-4384-b5a8-4040262283eb/ENCFF784
GFP.hic?response-content-disposition=attachment%3B%20filename%3DENCFF784GFP.hic&Signature=YZmRQyN1SPT0vpRjyscTzLgV3WM%3D&x-amz-security-token=AgoJb3JpZ2luX2VjEMT%2F%2
F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIBECuWysenOZdkDKbB6XfmcaQ8zul6azEkuNauJcIB91AiEA6%2FLuT9kpu%2BIvggBn3Dp7mJBxg92zGio1NlitzdjrJ08q4wMI7f%2F%2F%2F%2F%2F%
2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDCYx83zFw7htd3syviq3AzISSluwceByIAxIcFaxcorsp2pKt8ZqsPUcgjsIeWJR2CaHnhyxGLXV3ydF8C4CmxYYbgepXvPKTYy6i0UASGK8YJf2mPlzGoXjdiRZo%2B
pjTJmhVO59CYgV%2B%2BoJqHYOkYqEotyfNT%2BOYxaMum0qzeOfpWjrApXqNyrBkkrowu6Ng85eDNjmZTO0DZWGJICZm%2FiBEi%2B6sewE3aGBLdTGiaKhmbDsKjuNUNcBUwZfYfjBt0OwAASJZrC6C%2BjVSfiLjgBO
aeY8pXqKmZ1X218AjX6vVzCSnriYvsLY1fmgCdu66ifTUU5GL5NqyDmS76UVO1PoviFSctLBU%2B%2BJtQBWwk%2BcoLS0oIU1%2FhtnoHYKH4M%2B2ne3z1w5i86L1zsdUMXqZQ8AZ1A03iCUhQzCnDo%2Bo04njmhlF0
8voTZKVHbZ2icSEiKYERboR%2BMIhicYDtDhkRCcTZooP55A%2FSD9Ov4YtkwmpRgoBRMCYkiNl4rLD6MNyGipFJdFeoHllVZCfbMSjtOwmZRcOdKzZuJ%2FCDZhgPnjbzGc0oe%2Fp0QpXfYHNN00g6XEQdleW%2BcSGI
Dx7hMzuZydGxWOf7IwlpuO6AU6tAF4294kivKpwT90N%2FCA%2Fdquas6GmSMLpRWET%2FGcjgu9mhHXH0cZ3OOFHmWs%2Bo4e5o2i7geA7yCALBw4aIf%2BPpRUjREIx9IOPzMJHtKqDX8BLgFnjrvR2FXeiCnW%2BN0Y
xXGDP%2BRrlxLDkxBVbH6afcfV7ysqHIAwkWTUNlz7wKkCy4aIAfB3mPqMp3WsJ5KDx7xSNDfVe1kvsmNv8%2FIWed1K3KlnjG5VVbH2IklxMBJw5%2FYqSaQ%3D&Expires=1560645117&AWSAccessKeyId=ASIATGZ
NGCNXXYADQSVU HTTP/1.1
2019/06/14 14:46:49:390 CEST [DEBUG] headers - http-outgoing-1 >> Host: download.encodeproject.org
2019/06/14 14:46:49:390 CEST [DEBUG] headers - http-outgoing-1 >> Connection: Keep-Alive
2019/06/14 14:46:49:390 CEST [DEBUG] headers - http-outgoing-1 >> User-Agent: Apache-HttpClient/4.5 (Java/1.8.0_191)
2019/06/14 14:46:49:392 CEST [DEBUG] headers - http-outgoing-1 >> Accept-Encoding: gzip,deflate
2019/06/14 14:46:49:548 CEST [DEBUG] headers - http-outgoing-1 << HTTP/1.1 404 Not Found
2019/06/14 14:46:49:548 CEST [DEBUG] headers - http-outgoing-1 << Server: nginx/1.14.1
2019/06/14 14:46:49:548 CEST [DEBUG] headers - http-outgoing-1 << Date: Fri, 14 Jun 2019 12:31:58 GMT
2019/06/14 14:46:49:548 CEST [DEBUG] headers - http-outgoing-1 << Content-Type: text/html
2019/06/14 14:46:49:548 CEST [DEBUG] headers - http-outgoing-1 << Content-Length: 169
2019/06/14 14:46:49:548 CEST [DEBUG] headers - http-outgoing-1 << Connection: keep-alive
2019/06/14 14:46:49:549 CEST [DEBUG] MainClientExec - Connection can be kept alive indefinitely
2019/06/14 14:46:49:549 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection [id: 1][route: {s}->https://download.encodeproject.org:443] can be kept alive ind
efinitely
2019/06/14 14:46:49:549 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection released: [id: 1][route: {s}->https://download.encodeproject.org:443][total kept 
alive: 2; route allocated: 1 of 2; total allocated: 2 of 20]
2019/06/14 14:46:49:549 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection manager is shutting down
2019/06/14 14:46:49:549 CEST [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-1: Close connection
2019/06/14 14:46:49:550 CEST [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-0: Close connection
2019/06/14 14:46:49:550 CEST [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-0: Close connection
2019/06/14 14:46:49:551 CEST [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-1: Close connection
2019/06/14 14:46:49:551 CEST [DEBUG] PoolingHttpClientConnectionManager - Connection manager shut down
[SEVERE][HicFileInfo]Unexpected Http status code HTTP/1.1 404 Not Found for "https://www.encodeproject.org/files/ENCFF784GFP/@@download/ENCFF784GFP.hic"

设置用户代理不起作用。

我该如何解决这个问题？谢谢

【问题讨论】：

  • 您是否检查过@ 符号的编码是否正确？此外，您正在 Java 代码中执行 HEAD 请求（只是告诉您以防以后出现问题）。
  • @biG-AdR 是的，我正在做一个 HEAD，我也用 Get 进行了测试：相同的结果
  • @biG-AdR 我用 '%40%40' 而不是 '%%' ，我也得到了 404
  • 看起来 lib 没有正确理解重定向。如果您查看日志，您可以看到它正在将您重定向到download.encodeproject.org/https:/…4384-b5a8-4040262283eb/ENCFF784GFP.hic?response-content-disposition=attachment%3B%20filename%3DENCFF784GFP.hic&Signature=YZmRQyN1SPT0vpRjyscTzLgV3WM%3D -security -token=
  • 第二个 https 有一个 / 缺失（在 download.encodeproject.org/https:/ 之后）。如果我从日志中输入完全相同的 URL，我也会在 Chrome 中得到 404。添加额外的 / 解决了这个问题。

标签： java http-status-code-404 apache-httpclient-4.x apache-httpcomponents


【解决方案1】：

我在https://issues.apache.org/jira/browse/HTTPCLIENT-1993 发布了我的问题？

我在这里引用 Oleg Kalnichevski 的回答。

服务器返回的重定向 URI 包含未转义的保留 路径组件中的字符。

https://download.encodeproject.org/https://encode-public.s3.amazonaws.com/2018/04/06/8723ee96-ccee-4384-b5a8-4040262283eb/ENCFF784GFP.hic

如果您希望 HttpClient 将该序列传递到服务器而不 验证和规范化它，你可以禁用请求 URI 使用 RequestConfig 进行标准化。

CloseableHttpClient client = HttpClients.createSystem();
RequestConfig requestConfig = RequestConfig.custom().setNormalizeUri(false).build();
HttpClientContext clientContext = HttpClientContext.create();
clientContext.setRequestConfig(requestConfig);
try (CloseableHttpResponse response = client.execute(
        new HttpGet("https://www.encodeproject.org/files/ENCFF784GFP/@@download/ENCFF784GFP.hic"),
        clientContext)) {
    System.out.println(response.getStatusLine());
    HttpEntity entity = response.getEntity();
    EntityUtils.consume(entity);
}

HTTP/1.1 200 OK

【讨论】：

  • HttpClient 提交者在这里，请求修复您的错误。
  • @Michael-O 这不是httpClient的bug，只是我这边配置错误，所以没看懂“request your faulty to be fix”这句话
  • 抱歉，这应该是 URL 提供者。
  • 自我注意：亚马逊不喜欢'HttpHead'
猜你喜欢
  • 1970-01-01
  • 2016-02-10
  • 2015-04-26
  • 2010-09-25
  • 2015-01-04
  • 1970-01-01
  • 2018-06-28
  • 2017-10-26
  • 1970-01-01
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode