【发布时间】:2019-11-17 10:09:08
【问题描述】:
我处于 C# 的早期学习曲线。我有一个将网络链接存储到文本文件中的代码。如何将它们存储到变量中,以便稍后在代码中循环访问它们并分别访问每一个?
string pdfLinksUrl = "https://www.nordicwater.com/products/waste-water/";
// Load HTML content
var webGet = new HtmlAgilityPack.HtmlWeb();
var doc = webGet.Load(pdfLinksUrl);
// select all <A> nodes from the document using XPath
// (unfortunately we can't select attribute nodes directly as
// it is not yet supported by HAP)
var linkNodes = doc.DocumentNode.SelectNodes("//a[@href]");
// select all href attribute values ending with '.pdf' (case-insensitive)
var pdfUrls = from linkNode in linkNodes
let href = linkNode.Attributes["href"].Value
where href.ToLower().StartsWith("https://www.nordicwater.com/product/")
select href;
// write all PDF links to file
System.IO.File.WriteAllLines(@"c:\temp\pdflinks.txt", pdfUrls.ToArray());
【问题讨论】:
标签: c# web-crawler