【问题标题】:Can I call an R function while using open FSharp.Collections.ParallelSeq?我可以在使用打开的 FSharp.Collections.ParallelSeq 时调用 R 函数吗?
【发布时间】:2018-05-22 01:58:48
【问题描述】:

我有一些代码在没有并行化的情况下运行没有问题。但是,如果我尝试使用PSeq 而不是Seq 运行相同的代码,则会产生异常。我收到的消息看起来有点随机,很难准确复制。

这里是代码。当异常发生时,以let tmp2 开头的三行将突出显示。

let frameToRMatrix (df: Frame<'R,string>) =

    let foo k df : float list =
        df
        |> Frame.getCol k
        |> Series.values
        |> List.ofSeq

    let folder acc k = (k, foo k df |> box) :: acc

    let tmp =
        List.fold folder [] (df.ColumnKeys |> List.ofSeq)
        |> namedParams

    let sd = df |> Frame.getCol "Vol0" |> Series.lastValue
    let sd = sd * 1000.0 |> int

    printfn "%s" "I was here"

    let rand = System.Random(sd)
    let rms = rand.Next(500)
    System.Threading.Thread.Sleep rms

    let tmp2 =
        tmp
        |> R.cbind // This line prints something on the console the first time it is executed

    printfn "%s" "And here too"

    tmp2

上面的代码包括随机数生成和对System.Threading.Thread.Sleep的调用。如果我不包含在顺序执行下不需要的代码,我会收到一条消息:

System.ArgumentException: 'An item with the same key has already been added.'

以及控制台上的以下内容:

I was here
I was here
[1] 4095

所以执行永远不会到达And here too 行。

当我包含随机数生成器和对sleep 的调用时,我得到不同的结果,这似乎取决于构建选项。

以下是四个示例,包括构建选项、错误消息以及我在控制台上看到的内容。请注意,在所有示例中都有四个 I was here 实例,但只有三个 And here too 实例。

---------------------------------------
Any CPU with Prefer 32-bit checked 
    System.Runtime.InteropServices.SEHException: 'External component has thrown an exception.'

I was here
I was here
[1] 4095
And here too
I was here
And here too
I was here
And here too
Warning: stack imbalance in 'lazyLoadDBfetch', 66 then 65
Error in value[[3L]](cond) : unprotect_ptr: pointer not found


    ---------------------------------------
Any CPU with Prefer 32-bit unchecked 
    System.Runtime.InteropServices.SEHException: 'External component has thrown an exception.'


I was here
I was here
[1] 1.759219e+13
And here too
I was here
And here too
I was here
And here too
Error: cons memory exhausted (limit reached?)
Error: cons memory exhausted (limit reached?)




    ---------------------------------------
x86
System.AccessViolationException" 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'


I was here
I was here
[1] 4095
And here too
I was here
And here too
I was here
And here too

    ---------------------------------------
x64
Exception thrown: 'System.AccessViolationException' in Unknown Module. Attermpted to read or write protected memory.
$$$ - MachineLearning.signal: Calculating signal for ticker AAPL
$$$ - MachineLearning.signal: Calculating signal for ticker AAPL
I was here
I was here
[1] 1.759219e+13
And here too
I was here
And here too
I was here
And here too
Error in loadNamespace(name) :
  no function to return from, jumping to top level

【问题讨论】:

    标签: r multithreading parallel-processing f# type-providers


    【解决方案1】:

    根据我在 R 类型提供程序中调试线程微妙问题的经验,我认为答案是否定的 - 遗憾的是,R 本机互操作层不是线程安全的,因此您不能从 F# 中的多个线程调用它应用。

    我认为并行运行 R 的标准方法是生成多个 R.exe 进程来完成这项工作。我不认为您可以轻松地从 F# 初始化多个独立的 R 进程,因此您最好的选择可能是创建多个 .NET 进程,每个进程控制一个 R 引擎。

    【讨论】:

    • 在相同的参数上调用 List.map,我创建了两个相同的对象,相同类的实例并为每个对象执行相同的方法(称为 obj.crashable,即崩溃的方法)。我将可崩溃的方法配置为按顺序运行。所以并行化在调用树中是一个更高的层次。有趣的是,程序没有崩溃,但我的 CPU 使用率保持在 15%,就像我做同样的事情但只使用两个对象之一时一样。
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2017-07-22
    • 2013-01-23
    • 2010-09-20
    • 1970-01-01
    • 2013-01-19
    • 2020-05-03
    • 2011-12-15
    相关资源
    最近更新 更多