需求:
基于jsoup或者httpclient爬取https数据的时候涉及到权限认证
实现:
直接在HttpsURLConnection的时候加入check操作
public static void checkQuietly() { try { HttpsURLConnection .setDefaultHostnameVerifier(new HostnameVerifier() { public boolean verify(String hostname, SSLSession session) { return true; } }); SSLContext context = SSLContext.getInstance("TLS"); context.init(null, new X509TrustManager[] { new X509TrustManager() { public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException { } public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException { } public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; } } }, new SecureRandom()); HttpsURLConnection.setDefaultSSLSocketFactory(context .getSocketFactory()); } catch (Exception e) { e.printStackTrace(); } }
PS:
涉及到证书或者自己业务单独剥离实现,原理类似
调用的时候:
checkQuietly() ;Connection conn = Jsoup.connect(url).userAgent(MOZILLA_AGENT) .timeout(TIME_OUT);