OPENMP taskyield 指令在混合（openmp + mpi）程序中不产生任务答案

【问题标题】：OPENMP taskyield directive not yielding tasks in hybrid (openmp + mpi) programOPENMP taskyield 指令在混合（openmp + mpi）程序中不产生任务
【发布时间】：2019-01-15 04:30:42
【问题描述】：

有两个任务。一个通过 mpi 进行通信，另一个做一些很酷的事情。我使用 mpi 非阻塞调用以允许正在运行的线程在等待信息的同时执行其他任务。但它不起作用。

对代码的一点解释。如果你愿意，你可以直接跳到代码。它创建了两个进程。第一个进程将休眠 10 秒，然后通过阻塞调用发送一些数据。第二个进程（这是感兴趣的进程）将创建两个具有共享整数的线程（完成）。一个任务将被实例化一次，而另一个任务将被实例化的次数与线程数一样多（在本例中为 2）。第一个任务将使用非阻塞调用接收数据，并且应该让运行的线程可以执行另一个任务（但这不会发生）。此任务仅由一个线程运行。第二个任务只是打印一些很酷的东西。此任务由两个线程实例化。

代码如下：

if (rank == 0)
  {
    buffer[1000] = 5;
    sleep(10);
    printf("Process %d sent data\n", rank);
    MPI_Send(buffer, SIZE, MPI_DOUBLE, 1, 5, MPI_COMM_WORLD);
  }
  else if (rank == 1)
  {
#pragma omp parallel num_threads(2) shared (done)
    {
#pragma omp single nowait
      {
#pragma omp task
        {
          int flag = 0;
          printf("Thread %d is receiving data\n", omp_get_thread_num());
          MPI_Irecv(buffer, SIZE, MPI_DOUBLE, 0, 5, MPI_COMM_WORLD, &request);

          MPI_Test(&request, &flag, &status);
          while (flag == 0)
          {
            #pragma omp taskyield
            MPI_Test(&request, &flag, &status);
            printf("Thread %d is wasting time\n", omp_get_thread_num());
            sleep(1);
          }

          done=1;
          printf("Thread %d received data\n", omp_get_thread_num());
        }
      }

#pragma omp task 
      {
        printf("Thread %d entered to the cool task\n", omp_get_thread_num());
        while (done == 0)
        {
          printf("Thread %d is doing some cool stuff\n", omp_get_thread_num());
          sleep(1); /* Or not */
        }
      }
    }
  }

这是输出：

Thread 0 is receiving data
Thread 1 entered to the cool task
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Process 0 sent data
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 is wasting time
Thread 1 is doing some cool stuff
Thread 0 received data
Thread 0 entered to the cool task

如您所见，线程 0 在完成通信任务之前不会进入酷任务。

编译命令：

mpicc -fopenmp pruebas.c -o prueba

执行命令：

mpirun -np 2 --allow-run-as-root ./prueba (worry not, read below)

其他信息：

The program is being ran in a docker container with ubuntu.
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)
mpicc --show output: gcc -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags -L/usr/lib/openmpi/lib -lmpi

【问题讨论】：

标签： multithreading mpi openmp yield

【解决方案1】：

OpenMP 规范并不要求 taskyield 做任何事情，它说

taskyield 构造指定当前任务可以暂停以执行不同的任务。

所以它允许重新安排但不需要它。（任务调度算法描述在这一点上也很清楚）。因此 gcc 完全有权不重新安排。

不过，我确实知道 LLVM（和 Intel）OpenMP 运行时确实产生了，我们可以看到 gcc is generating a call into the runtime，所以有可能只使用 LLVM 运行时和您的二进制文件就足够了。或者，尝试 clang 或 the Intel compilers

（完整披露：我为英特尔工作 :-)）

【讨论】：