【发布时间】:2020-07-19 01:19:57
【问题描述】:
我正在编写一个程序,该程序接受用户输入以连接到站点,将其 html 下载到文本中,并每天两次从表中检索数据。我知道代码不会适合任何页面(一旦我开始工作,我可能会将 url “硬连线”到代码中)。我目前的问题是我的 jsoup 解析器没有正确读取表格数据。我不确定我的元素选择器是否过于通用?该表看起来像标准表/tr/td 格式,但我的行数组填充大小为 0。如果有人可以帮助我调试我的解析器并可能提供一些关于在哪里寻找让它每天两次静默抓取数据的建议,我真的很感激!没有运行时/编译错误,只需要更正输出。
来源网站:https://www.cnbc.com/us-markets/ 表格源代码(片段):
<table class="BasicTable-table"><thead class="BasicTable-tableHeading BasicTable-tableHeadingSortable"><tr><th class="BasicTable-textData"><span>SYMBOL <span class="icon-sort undefined"></span></span></th><th class="BasicTable-numData"><span>PRICE <span class="icon-sort undefined"></span></span></th><th class="BasicTable-numData">
我的代码:
public class StockScraper {
public static void main(String[] args) {
Scanner input = new Scanner (System.in);
System.out.println("Enter the complete url (including http://) of the site you would like to parse:");
String html = input.nextLine();
try {
Document doc = Jsoup.connect(html).get();
System.out.printf("Title: %s", doc.title());
//Try to print site content
System.out.println("");
System.out.println("Writing html contents to 'html.txt'...");
//Save html contents to text file
PrintWriter outputfile = new PrintWriter("html.txt");
outputfile.print(doc.outerHtml());
outputfile.close();
//Select stock data you want to retrieve
System.out.println("Enter the name of the stock you want to check");
String name = input.nextLine();
//Pull data from CNBC Markets
Element table = doc.select("table").get(0);
Elements rows = table.select("tr");
System.out.println(rows.size());
for(int i = 1; i < rows.size(); i++) {
Element rowx = rows.get(i);
Elements col = rows.select("td");
if(col.get(0).equals(name)) {
System.out.println("I worked!");
System.out.println(col.get(1));
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
【问题讨论】:
标签: java html web-scraping automation jsoup