【发布时间】:2018-07-16 05:34:12
【问题描述】:
我正在尝试从 URL 中检索 100 页。 网址格式如下: https://www.someBlogSite.com/thread.php?t=xxxx&page=
所以我的 for 循环基本上遍历这些页面并将结果存储在我本地磁盘上的 html 文件中。
https://www.someBlogSite.com/thread.php?t=xxxx&page=1 https://www.someBlogSite.com/thread.php?t=xxxx&page=2 . . https://www.someBlogSite.com/thread.php?t=xxxx&page=99
这是我的工作代码,有没有办法避免使用 await Task.Delay(10000)? 我必须使用它的原因是因为否则我的代码会在所有 100 页内容之前退出已取回。
using System;
using System.Linq;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json;
using System.Threading;
using System.Threading.Tasks;
using System.IO;
namespace RetrieveImages
{
public class PerformingGet
{
static void Main(string[] args) => MainAsync(args).Wait();
static async Task MainAsync(string[] args)
{
//Instantiating HttpClient once in the main method since this is costly
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
//Base Url--going forward only this will need to be edited per collection
string pageBaseUrl = "https://www.someBlogSite.com/thread.php?t=xxxx&page=";
var collectionOfUrls= new List<string>();
for (int pageNum = 1; pageNum < 100; pageNum++)
{
string pageUrl = pageBaseUrl + pageNum;
collectionOfUrls.Add(pageUrl);
}
var tasks = collectionOfUrls.Select(url => GetRequestString(client, url));
var results = await Task.WhenAll(tasks);
var text = string.Join("\r\n", results);
WriteTextAsync(text);
foreach (var x in collectionOfUrls)
{
Console.WriteLine(x);
//await GetRequest(client, x);
}
Console.WriteLine("Task completed");
await Task.Delay(10000);
}
async static Task<string> GetRequestString(HttpClient client, string Url)
{
using (HttpResponseMessage response = await client.GetAsync(Url))
{
if (response.IsSuccessStatusCode)
{
using (HttpContent content = response.Content)
{
return await content.ReadAsStringAsync();
}
}
return string.Empty;
}
}
static async void WriteTextAsync(string text)
{
// Set a variable to the My Documents path.
string mydocpath = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
// Write the text asynchronously to a new file named "WriteTextAsync.txt".
using (StreamWriter outputFile = new StreamWriter(Path.Combine(mydocpath, "WriteTextAsync.html")))
{
await outputFile.WriteAsync(text);
}
}
}
}
【问题讨论】:
-
不要使用
async void。 -
... ^^ 所以你不会在等待
WriteTextAsync -
你的意思是这样做:await WriteTextAsync(text);以及 WriteTextAsync 应该如何?
标签: c# asp.net .net async-await