javax.net.ssl.HttpsURLConnection 返回火星诗歌

Question

我正在编写一个简单的 https 客户端，它将通过 https 拉下网页的 html。我可以很好地连接到网页，但是我下拉的 html 是乱码。

public String GetWebPageHTTPS(String URI){
    BufferedReader read;
    URL inputURI;
    String line;
    String renderedPage = "";
    try{
        inputURI = new URL(URI);
        HttpsURLConnection connect;
        connect = (HttpsURLConnection)inputURI.openConnection();
        connect.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401");
        read = new BufferedReader (new InputStreamReader(connect.getInputStream()));
        while ((line = read.readLine()) != null)
            renderedPage += line;
        read.close();
    }
    catch (MalformedURLException e){
        e.printStackTrace();
    }
    catch (IOException e){
        e.printStackTrace();
    }
    return renderedPage;
}

当我传递一个类似https://kat.ph/ 的字符串时，会返回大约 10,000 个字符的乱码

编辑 这是我修改后的自签名证书代码，但我仍然得到加密流：

public String GetWebPageHTTPS(String URI){
    TrustManager[] trustAllCerts = new TrustManager[] { 
            new X509TrustManager() {     
                public java.security.cert.X509Certificate[] getAcceptedIssuers() { 
                    return null;
                } 
                public void checkClientTrusted( 
                    java.security.cert.X509Certificate[] certs, String authType) {
                    } 
                public void checkServerTrusted( 
                    java.security.cert.X509Certificate[] certs, String authType) {
                }
            } 
        }; 
        try {
            SSLContext sc = SSLContext.getInstance("SSL"); 
            sc.init(null, trustAllCerts, new java.security.SecureRandom()); 
            HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
        } catch (GeneralSecurityException e) {
        } 
        try { 
            System.out.println("URI: " + URI);
            URL url = new URL(URI); 
        } catch (MalformedURLException e) {
        } 
    BufferedReader read;
    URL inputURI;
    String line;
    String renderedPage = "";
    try{
        inputURI = new URL(URI);
        HttpsURLConnection connect;
        connect = (HttpsURLConnection)inputURI.openConnection();
        read = new BufferedReader (new InputStreamReader(connect.getInputStream()));
        while ((line = read.readLine()) != null)
            renderedPage += line;
        read.close();
    }
    catch (MalformedURLException e){
        e.printStackTrace();
    }
    catch (IOException e){
        e.printStackTrace();
    }
    return renderedPage;
}

Answer 1

“它被压缩了吗？stackoverflow.com/questions/8249522/...” – Mahesh Guruswamy

是的，原来它只是 gzip 压缩，这是我的解决方法

public String GetWebPageGzipHTTP(String URI){ 
    String html = "";
    try {
        URLConnection connect = new URL(URI).openConnection();                        
        BufferedReader in = null;
        connect.setReadTimeout(10000);
        connect.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401");
        if (connect.getHeaderField("Content-Encoding")!=null && connect.getHeaderField("Content-Encoding").equals("gzip")){
            in = new BufferedReader(new InputStreamReader(new GZIPInputStream(connect.getInputStream())));            
        } else {
            in = new BufferedReader(new InputStreamReader(connect.getInputStream()));            
        }          
        String inputLine;
        while ((inputLine = in.readLine()) != null){
        html+=inputLine;
        }
    in.close();
        return html;
    } catch (Exception e) {
        return html;
    }
}

}

Answer 2

HTTPS 始终提供证书，并且进一步的通信发生在安全的加密通道上。这就是为什么你收到的东西看起来像胡言乱语。

对于任何签名的证书，HttpsURLConnection 将为您完成工作并且一切正常。当证书未由证书颁发机构签名时，事情变得混乱。在这种情况下，如果您从浏览器打开该 URL，它将显示证书供您检查和接受，然后再继续。

看起来您在这里遇到了类似的问题。您需要做的是告诉 Java 接受自签名证书而不抱怨。您在这里有两个选择，要么下载证书（只需在任何浏览器中打开 URL，它就会向您展示如何操作）并将其添加到您的 JVM 的密钥库中，或者创建您自己的 TrustManager 并禁用证书验证。

有关这两个选项的详细信息，请参阅此 SO 答案。 https://stackoverflow.com/a/2893932/2385178