如何从 ASP.NET 获取网页的 HTML 内容答案

WebRequest request = WebRequest.Create(sURL); request.Method = "Get"; //Get the response WebResponse response = request.GetResponse(); //Read the stream from the response StreamReader reader = new StreamReader(response.GetResponseStream(), System.Text.Encoding.UTF8);

【解决方案1】：

使用HttpWebRequest获取任何网页的内容...

// We will store the html response of the request here
string siteContent = string.Empty;

// The url you want to grab
string url = "http://google.com";

// Here we're creating our request, we haven't actually sent the request to the site yet...
// we're simply building our HTTP request to shoot off to google...
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.AutomaticDecompression = DecompressionMethods.GZip;

// Right now... this is what our HTTP Request has been built in to...
/*
    GET http://google.com/ HTTP/1.1
    Host: google.com
    Accept-Encoding: gzip
    Connection: Keep-Alive
*/


// Wrap everything that can be disposed in using blocks... 
// They dispose of objects and prevent them from lying around in memory...
using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())  // Go query google
using(Stream responseStream = response.GetResponseStream())               // Load the response stream
using(StreamReader streamReader = new StreamReader(responseStream))       // Load the stream reader to read the response
{
    siteContent = streamReader.ReadToEnd(); // Read the entire response and store it in the siteContent variable
}

// magic...
Console.WriteLine (siteContent);

【讨论】：

如果你要否决它...写一个评论至少解释原因以便我可以改进它，而其他人不会觉得这个解决方案不起作用
谢谢你，但我只需要知道为什么来自浏览器的 URL 请求返回正确的结果 (HTML)，而 ASP.Net Web 请求不能。这是因为请求的站点是在 ASP.Net MVC 中开发的吗？
它将能够...可能还有其他事情发生，但无论是什么...我可以向您保证，如果您可以在网络浏览器中访问它...您可以使用 HttpWebRequest 类访问它，因为它们都使用 HTTP。问题可能是通过 JavaScript 进行重定向...或者可能在您请求的页面加载后触发 AJAX 请求...下载 Fiddler...并查看访问页面后发生的 HTTP 请求在浏览器中
这非常有效。它通过客户端执行此请求。所以无需在客户端使用 jquery 等进行尝试。