Httpfs 使用 httpclient 创建或附加文件答案

【问题标题】：Httpfs Create or append file with httpclientHttpfs 使用 httpclient 创建或附加文件
【发布时间】：2017-11-21 17:08:02
【问题描述】：

我使用 HTTPCLIENT 创建或附加带有 REST 组件 HTTPFS 的文件。

示例 cmd curl - 使用 curl

curl -i -X PUT -s --negotiate -u : "http://httpfsServer:14000/webhdfs/v1/user/a_app_kpi/tmp/testAppend.txt?op=CREATE&data=true" --header "Content-Type:application/octet-stream" --header "Transfer-Encoding:chunked" -T testAppend.txt

return query
HTTP/1.1 100 Continue

HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Content-Type: text/html;charset=utf-8
Content-Length: 997
Date: Tue, 21 Nov 2017 16:07:07 GMT

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Server: Apache-Coyote/1.1
Set-Cookie: hadoop.auth="u=a_app_kpi&p=a_app_kpi@Environnement&t=kerberos-dt&e=1511316427865&s=JIJTCZS31D1wphVORe3ygZ1NBuo="; Path=/; HttpOnly
Content-Type: application/json
Content-Length: 0
Date: Tue, 21 Nov 2017 16:07:07 GMT

但是当我执行请求时，程序返回错误 com.vhl.action.WebHdfsAction - null

我的创建代码：

CloseableHttpClient httpClient = cnxHttp.getHttpClient();
File file = new File(beanActionHttp.getPathLocal());
InputStreamEntity reqEntity = new InputStreamEntity(
                            new FileInputStream(file), -1, ContentType.APPLICATION_OCTET_STREAM);
                    reqEntity.setChunked(true);

HttpUriRequest request = RequestBuilder
                        .put(action_http)
                        .setEntity(reqEntity)
                        .build();

request.addHeader(HttpHeaders.TRANSFER_ENCODING, "chunked");
request.addHeader(HttpHeaders.CONTENT_TYPE, "application/octet-stream");
// HERE EXECUTE RETURN NULL
CloseableHttpResponse response = httpClient.execute(request);

【问题讨论】：

标签： java httpfs

【解决方案1】：

我发现了我的麻烦。首先我更改方法执行请求以捕获一个奇妙的错误 - 谢谢错误

CloseableHttpResponse 响应 = httpClient.execute(request);到 HttpResponse 响应 = httpClient.execute(request);

Error catching 
    org.apache.http.client.ClientProtocolException
            at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
            at com.vhl.action.WebHdfsAction$4.run(WebHdfsAction.java:399)
            at com.vhl.action.WebHdfsAction$4.run(WebHdfsAction.java:326)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:356)
            at com.vhl.action.WebHdfsAction.doCreateAppend(WebHdfsAction.java:326)
            at com.vhl.action.WebHdfsAction.appendFileHdfsHttpfs(WebHdfsAction.java:93)
            at com.vhl.httpclient.HttpClient.main(HttpClient.java:74)
    Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.

所以我的麻烦是我的请求无法用不可重复的请求实体重试请求。

我用这个来解决我的麻烦 Cannot retry request with a non-repeatable request entity

我将我的实体添加到缓冲区 "BufferedHttpEntity"

代码终于可以运行了：

CloseableHttpClient httpClient = cnxHttp.getHttpClient();
File file = new File(beanActionHttp.getPathLocal());
InputStreamEntity reqEntity = new InputStreamEntity(
                            new FileInputStream(file), -1, ContentType.APPLICATION_OCTET_STREAM);
                    reqEntity.setChunked(true);

BufferedHttpEntity reqEntityBuf = new BufferedHttpEntity(reqEntity);

HttpUriRequest request = RequestBuilder
                        .put(action_http)
                        .setEntity(reqEntityBuf)
                        .build();


// HERE EXECUTE RETURN NULL
CloseableHttpResponse response = httpClient.execute(request);

【讨论】：