Parallel.ForEach 共享资源

Question

如何在 Parallel.ForEach 循环中控制对共享资源的访问？我正在尝试并行下载多个文件，并且我想捕获有关下载失败的信息，以便用户稍后可以重新尝试下载。但是，我担心如果多个下载同时失败，应用程序会抛出异常，因为一个线程会在另一个线程写入文件时尝试访问该文件。

在下面的代码中，我想知道如何控制对RepeateRequestPath 文件的访问。 RequestSet 是代表我尝试下载的资源 ID 的字符串列表。

Dim DownloadCnt As Integer = 0
Dim ParallelOpts As New ParallelOptions()
ParallelOpts.MaxDegreeOfParallelism = 4
Parallel.ForEach(RequestSets, ParallelOpts, Sub(RequestSet)
        Try
            DownloadCnt += 1
            Dim XmlUrl As String = String.Format("{0}{1}{2}", "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=", String.Join(",", RequestSet), "&retmode=xml&rettype=abstract")
            DownloadFile(XmlUrl, String.Format("{0}\TempXML{1}.xml", XMLCacheDir, DownloadCnt))
        Catch ex As WebException
            Using Response As WebResponse = ex.Response
                Dim statCode As Integer = CInt(DirectCast(Response, HttpWebResponse).StatusCode)
                MessageBox.Show(String.Format("Failed to retrieve XML due to HTTP error {0}. Please hit the 'Retrieve XML' button to re-run retrieval after the current set is complete.", statCode))
                If Not File.Exists(RepeatRequestPath) Then
                    File.WriteAllLines(RepeatRequestPath, RequestSet)
                Else
                    File.AppendAllLines(RepeatRequestPath, RequestSet)
                End If
            End Using
        End Try
    End Sub)

Answer 1

在 VB.NET 中保护共享资源的常用方法是使用SyncLock。

因此，您将在Parallel.ForEach() 循环之前创建一个锁对象：

Dim lock = New Object

然后你会在循环中使用它：

SyncLock lock
    File.AppendAllLines(RepeatRequestPath, RequestSet)
End SyncLock

另请注意，即使文件尚不存在，您也可以使用 AppendAllLines()，因此您不必检查。

Answer 2

您需要使用信号量来控制对共享资源的访问。您希望一次只有一个线程访问错误文件，因此将信号量初始化为只允许 1 个线程进入。调用 _pool.WaitOne 应该抓住信号量，然后在完成创建/写入文件后释放它。

Private Shared _pool As Semaphore
_pool = = New Semaphore(0, 1)

Dim DownloadCnt As Integer = 0
Dim ParallelOpts As New ParallelOptions()
ParallelOpts.MaxDegreeOfParallelism = 4
Parallel.ForEach(RequestSets, ParallelOpts, Sub(RequestSet)
        Try
            DownloadCnt += 1
            Dim XmlUrl As String = String.Format("{0}{1}{2}", "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=", String.Join(",", RequestSet), "&retmode=xml&rettype=abstract")
            DownloadFile(XmlUrl, String.Format("{0}\TempXML{1}.xml", XMLCacheDir, DownloadCnt))
        Catch ex As WebException
            Using Response As WebResponse = ex.Response
                Dim statCode As Integer = CInt(DirectCast(Response, HttpWebResponse).StatusCode)
                MessageBox.Show(String.Format("Failed to retrieve XML due to HTTP error {0}. Please hit the 'Retrieve XML' button to re-run retrieval after the current set is complete.", statCode))
                _pool.WaitOne()
                Try
                    If Not File.Exists(RepeatRequestPath) Then
                        File.WriteAllLines(RepeatRequestPath, RequestSet)
                    Else
                        File.AppendAllLines(RepeatRequestPath, RequestSet)
                    End If
                Catch ex as Exception
                    'Do some error handling here.
                Finally
                    _pool.Release()
                End Try
            End Using
        End Try
    End Sub)

Answer 3

svick 的解决方案几乎是正确的。但是，如果您需要保护对共享变量的访问，您还需要在类级别将锁定对象声明为共享。

这可以正常工作：

Friend Class SomeClass
    Private Shared _lock As New Object

    Private Shared sharedInt As Integer = 0

    Sub Main()
        SyncLock _lock
            sharedInt += 1
        End SyncLock
    End Sub
End Class

如果您使用非共享锁对象，synclock 将仅保护变量免受同一实例内的多个访问线程的影响，而不是跨实例。