HTTP Post 到 API 返回 403 FORBIDDEN

Question

我必须将 XML 文件上传到 API。这是 API 由我从 API 的颁发者处获得的签名证书保护。

现在，我有两个用例。首先，我必须从这个 API 下载一些文件。这与以下代码完美配合：

final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ip, port));

final URL url = new URL(linkToFile);
final HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(proxy);
conn.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
try (
            InputStream inputStream = zipUrlConn.getInputStream();
            ZipInputStream stream = new ZipInputStream(inputStream);) {

    // Do stuff with ZipInputStream here
}

createSSLContext() 方法如下所示：

public SSLContext createSSLContext() throws NoSuchAlgorithmException, CertificateException, IOException, KeyStoreException,
        UnrecoverableKeyException, KeyManagementException {

    final KeyStore clientStore = KeyStore.getInstance("PKCS12");
    clientStore.load(new FileInputStream(this.certificateResource.getFile()), this.p12PW.toCharArray());

    final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
    keyManagerFactory.init(clientStore, this.p12PW.toCharArray());
    final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();

    final KeyStore trustStore = KeyStore.getInstance("JKS");
    trustStore.load(new FileInputStream(this.trustStoreResource.getFile()), this.trustStorePW.toCharArray());

    final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
    trustManagerFactory.init(trustStore);
    final TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();

    final SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(keyManagers, trustManagers, new SecureRandom());

    return sslContext;
}

我遵循了从发行者那里获得的指南，该指南显示了如何使用 cUrl 命令执行此操作：

curl --cert Certificate-<id>.pem[:pem_password] https://api.url.com/

所以我基本上是在尝试在 java 中重建这个命令，它正在工作。

现在是不工作的部分，即文件上传。同样，我得到了一个 cUrl 命令，我必须重建它：

curl --cert Certificate-<id>.pem[:pem_password] -F upload=@<Path_to_file>\DS<PartnerId>_<Timestamp>.XML https://api.url.com/in/upload.php

我尝试了几件事来实现这一目标：

1. “普通”Java

首先，我用标准HttpsURLConnection试了一下，如下：

final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.1.25", 3128));
final HttpsURLConnection connection = (HttpsURLConnection) new URL(ARGE_UPLOAD_URL).openConnection(proxy);

connection.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");

try (OutputStream outputStream = connection.getOutputStream()) {
    outputStream.write(Files.readAllBytes(new File("src/main/resources/XML/example.xml").toPath()));
}

final InputStream result = connection.getInputStream();

但这总是导致java.io.IOException: Server returned HTTP response code: 403 for URL: https://api.url.com/in/upload.php，即使我使用相同的配置，我可以从 API 下载。

1. Apache HttpClient

我发现一些资源声称HttpClient 更容易配置和使用，所以我试了一下：

final CloseableHttpClient httpClient = HttpClientBuilder
            .create()
            .setSSLContext(this.http.createSSLContext())
            .setProxy(new HttpHost(InetAddress.getByName(ip), port))
            .build();
final HttpEntity requestEntity = MultipartEntityBuilder
            .create()
            .addBinaryBody("upload=@example.xml", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
            .build();

final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);

try (CloseableHttpResponse response = httpClient.execute(post)) {
    this.logger.info(response.getStatusLine().toString());
    EntityUtils.consume(response.getEntity());
}

导致HTTP/1.1 403 FORBIDDEN

1. HttpClient（FileEntity 而不是 MultipartEntity）

最后我尝试了FileEntity：

final HttpPost httpPost = new HttpPost(https://api.url.com/in/upload.php);
httpPost.addHeader("content-type", "application/x-www-form-urlencoded;charset=utf-8");

final FileEntity fileEntity = new FileEntity(new File("src/main/resources/XML/example.xml"));
httpPost.setEntity(fileEntity);

System.out.println("executing request " + httpPost.getRequestLine() + httpPost.getConfig());
try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
    final HttpEntity responseEntity = response.getEntity();

    System.out.println("Status: " + response.getStatusLine());
    if (responseEntity != null) {
        System.out.println("Entity: " + EntityUtils.toString(responseEntity));
    }
}

导致Status: HTTP/1.1 403 FORBIDDEN

我只是不明白，尽管使用完全相同的配置，我如何能够从 API 下载，但不能上传到它。

如果您需要更多信息，我很乐意提供。

编辑

按照oli 的建议，我使用 Fiddler 来捕获 HTTPS 请求。这是方法 1（普通 Java）的结果：

POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept-Charset: utf-8
Accept: */*
Content-Type: application/x-www-form-urlencoded;charset=utf-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6738

这是通过谷歌浏览器手动上传的结果：

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 6948
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryxZxIJ5h19MEFbZQs
Cookie: cookie_consent=accepted
Host: hrbaxml.arbeitsagentur.de
Origin: https://hrbaxml.arbeitsagentur.de
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

编辑 2

补充一下，这是使用方法2（HttpClient with MultipartEntity）的结果：

POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Content-Length: 7025
Content-Type: multipart/form-data; boundary=1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Host: hrbaxml.arbeitsagentur.de
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_161)
Accept-Encoding: gzip,deflate

--1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Content-Disposition: form-data; name="DSV000306700_2018-08-23_09-00-00.xml";    filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

编辑 3

我尝试从 Chrome 请求中复制所有 HTTP 标头，因此来自 Java 的请求如下所示：

POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Content-Type: multipart/form-data; boundary=1535095530678
Cookie: cookie_consent=accepted
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6944

--1535095530678
Content-Disposition: form-data; name="uploadFile"; filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/xml
Content-Transfer-Encoding: binary

.. xml data ..

--1535095530678--

但仍然没有成功。还有其他可能的解决方案吗？也许不是上传的问题，而是其他问题？

Answer 1

我会捕获您使用 Java 应用程序发送到服务器的 HTTP 请求（例如使用 Wireshark），并将其与您从浏览器发送的 HTTP 请求进行比较（您可以使用内置浏览器轻松捕获它工具，请尝试按 F12)。

我 100% 确信您会发现一些差异，这对我来说总是有效的。

编辑：

还有另一个可能的问题。请尝试添加

connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0");

或相同的，您的浏览器在您的第一个实现中发送。还要确保您的 SSL 证书和加密算法没有问题（您使用默认的一种，在您的情况下是哪一种）。此外（如果没有帮助）您还可以检查协商握手密钥的密钥长度。

Answer 2

这个信息来自服务器端，所以你应该向服务提供商寻求帮助，也许上传需要认证

Answer 3

好吧，我终于想通了。正如预期的那样，错误出现在我的 POST 请求中。首先，这是工作代码（使用方法2）：

final CloseableHttpClient httpClient = HttpClientBuilder
        .create()
        .setSSLContext(this.http.createSSLContext())
        .setProxy(new HttpHost(InetAddress.getByName(ip), port))
        .build();
final HttpEntity requestEntity = MultipartEntityBuilder
        .create()
        .addBinaryBody("upload", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
        .build();

final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);

try (CloseableHttpResponse response = httpClient.execute(post)) {
    this.logger.info(response.getStatusLine().toString());
    EntityUtils.consume(response.getEntity());
}

如您所见，唯一更改的行是.addBinaryBody("upload", new File("src/main/resources/XML/example.xml"))。 "upload" 至关重要，因为我试图重建的 cUrl 调用具有 -F 标志。来自cUrl manual：

(HTTP SMTP IMAP) 对于 HTTP 协议族，这让 curl 模拟用户按下提交按钮的填写表单。这会导致 curl 根据 RFC 2388 使用 Content-Type multipart/form-data 发布数据。

每个部分都由文件名和数据组成。我正在使用的 API 依赖于“上传”的名称，以便处理请求。否则它不知道该怎么做并返回403 FORBIDDEN。