MPI Overlapping 使用 C 语言产生数组求和

Question

我在学习 MPI 编程时遇到了以下 stackoverflow question。我正在使用相同的“已回答”示例来计算每一行的总和，以增加重叠。这是一个 2 x 3 数组，我想在这些数组元素达到MPI_Irecv 时立即计算总和。我在MPI_Irecv 和MPI_Wait 之间编辑了我的代码，以便在数组元素可用时立即开始计算。但是当我运行代码时，我调用MPI_Test 的方式似乎无法正常工作。如果您可以通过示例解决此问题，我将不胜感激。

使用mpirun -np 3 test 0th rank 将生成数组元素。第一和第二排名将计算总和

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char *argv[])
{
        MPI_Init(&argc, &argv);
        int world_rank;
        MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
        int world_size;
        MPI_Comm_size(MPI_COMM_WORLD, &world_size);        
        int tag = 1;        
        int arr[2][3]; 
        MPI_Request request;
        MPI_Status status;
        int source = 0;
        int dest;
        int flag;

        printf ("\n--Current Rank: %d\n", world_rank);
        //To handle the number of process received by the user will be handled here later
        if (world_rank == 0)
        {
            int i = 1;
            int a, b, x, y;

            printf("* Rank 0 excecuting\n");
            for(x=0; x<2; x++)//Generating the whole 2 by 3  2D array
            {   
                i++;
                for ( y = 0; y < 3; y++ )
                {
                    arr[x][y] = i;//first row contain all 2 
                }                 //second row contain all 3
            }

            int subarray_index;
            for(subarray_index=0; subarray_index < 2; subarray_index++)
            {
                dest = subarray_index%(world_size - 1) + 1;     
                tag = subarray_index;
                MPI_Isend(&arr[subarray_index][0], 3, MPI_INT, dest, tag, MPI_COMM_WORLD, &request);
            }
        }
        else 
        {
            int a, b;                   
            for(b=0; b<2/(world_size-1); b++)
            {
                int sum = 0;
                int i;                
                int my_offset = world_rank-1;
                tag = b*(world_size-1) + my_offset;
                int subarray = b;
                MPI_Irecv(&arr[subarray][0], 3, MPI_INT, source, tag, MPI_COMM_WORLD, &request);
                MPI_Test(&request, &flag, &status);//I think there may be an error at MPI_Test too
                while (flag != 1)
                {
                    MPI_Test(&request, &flag, &status); 
                    for(i = 0; i<3; i++)
                    {   
                        //if(!arr[subarray][i])//want to wait till I recive actual array elements
                        //{//This need to start calculating as soon as array element become avilable 
                            printf("%d) arr[subarray][i]:%d at rank %d\n", tag, arr[subarray][i], world_rank);
                            sum = arr[subarray][i]+sum;
                        //}
                    }
                }                
                printf("\nSum is: %d at rank: %d and tag is:%d\n", sum, world_rank, tag);
                MPI_Wait (&request, &status); 
            }           
        }
        MPI_Finalize();
}

当我输入mpirun -np 3 test 时，答案应该是“Sum is 6”和“Sum is 9”

--Current Rank: 2
1) arr[subarray][i]:40896 at rank 2
1) arr[subarray][i]:32767 at rank 2
1) arr[subarray][i]:617513272 at rank 2
1) arr[subarray][i]:40896 at rank 2
1) arr[subarray][i]:32767 at rank 2
1) arr[subarray][i]:617513272 at rank 2
1) arr[subarray][i]:40896 at rank 2
1) arr[subarray][i]:32767 at rank 2
1) arr[subarray][i]:617513272 at rank 2 //all above arr element shows it's empty
1) arr[subarray][i]:3 at rank 2 //following three values are correct and these
1) arr[subarray][i]:3 at rank 2 //are the only three that need to use for summing
1) arr[subarray][i]:3 at rank 2

Sum is: 1909043312 at rank: 2 and tag is:1

--Current Rank: 0
* Rank 0 excecuting

--Current Rank: 1 //here I don't get arr element values as above

Sum is: 0 at rank: 1 and tag is:0

Answer 1

不确定您的问题是什么。但是根据您的描述，我推断进程 1 在已经收到消息时到达MPI_Test。因此，flag 已经被设置并且永远不会进入循环。

另一方面，在排名 2 上，由于在等待消息被接收时无条件地执行求和，因此您将未初始化的值相加。

只有在收到消息后才能进行求和。也就是说，当设置标志时，即在你的 while 循环之后，基本上会使你的整个构造无效，就像你可以做的那样：

MPI_Irecv(&arr[subarray][0], 3, MPI_INT, source, tag, MPI_COMM_WORLD, &request);
MPI_Wait (&request, &status);

一旦消息可用，等待将返回，这就是MPI_Wait 的重点。现在，上面基本上相当于一个阻塞MPI_Recv。

另一方面，您也需要在发送方的某个时间点等待通信完成。在那里，您需要为每个 MPI_Isend 提出单独的请求。然后您可以在发送循环后使用MPI_Waitall。

因此，您可能想要这样的东西（带有阻塞接收）：

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char *argv[])
{
        MPI_Init(&argc, &argv);
        int world_rank;
        MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
        int world_size;
        MPI_Comm_size(MPI_COMM_WORLD, &world_size);
        int tag = 1;
        int arr[2][3];
        MPI_Request request[2];
        int source = 0;
        int dest;
        int flag;

        printf ("\n--Current Rank: %d\n", world_rank);
        //To handle the number of process received by the user will be handled here later
        if (world_rank == 0)
        {
            int i = 1;
            int a, b, x, y;
            MPI_Status status[2];

            printf("* Rank 0 excecuting\n");
            for(x=0; x<2; x++)//Generating the whole 2 by 3  2D array
            {
                i++;
                for ( y = 0; y < 3; y++ )
                {
                    arr[x][y] = i;//first row contain all 2 
                }                 //second row contain all 3
                dest = x%(world_size - 1) + 1;
                tag = x;
                MPI_Isend(&arr[x][0], 3, MPI_INT, dest, tag, MPI_COMM_WORLD, &request[x]);
            }

            MPI_Waitall(2, &request[0], &status[0]);

        }
        else
        {
            int a, b;
            MPI_Status status;

            for(b=0; b<2/(world_size-1); b++)
            {
                int sum = 0;
                int i;
                int my_offset = world_rank-1;
                tag = b*(world_size-1) + my_offset;
                int subarray = b;
                MPI_Recv(&arr[subarray][0], 3, MPI_INT, source, tag, MPI_COMM_WORLD, &status);
                for(i = 0; i<3; i++)
                {
                    //if(!arr[subarray][i])//want to wait till I recive actual array elements
                    //{//This need to start calculating as soon as array element become avilable 
                        printf("%d) arr[subarray][i]:%d at rank %d\n", tag, arr[subarray][i], world_rank);
                        sum = arr[subarray][i]+sum;
                    //}
                }
                printf("\nSum is: %d at rank: %d and tag is:%d\n", sum, world_rank, tag);
            }
        }
        MPI_Finalize();
}

这会尽快发送值，并在收到它们后立即对行进行操作。如果该过程要处理多行，您可能仍希望在接收端发布多个接收，但在这种情况下，您再次需要一组请求，并且您将使用 MPI_Waitany 或 MPI_Waitsome 对收到的消息采取行动尽快。

这里是MPI_Waitany的接收部分：

{
    int a, b;
    MPI_Status status;
    MPI_Request request[2/(world_size-1)];

    for(b=0; b<2/(world_size-1); b++)
    {
        int my_offset = world_rank-1;
        tag = b*(world_size-1) + my_offset;
        int subarray = b;
        MPI_Irecv(&arr[subarray][0], 3, MPI_INT, source, tag, MPI_COMM_WORLD, &request[b]);
    }
    for(b=0; b<2/(world_size-1); b++)
    {
        int sum = 0;
        int i;
        MPI_Waitany(2/(world_size-1), &request[0], &a, &status);
        for(i = 0; i<3; i++)
        {
                printf("%d) arr[subarray][i]:%d at rank %d\n", status.MPI_TAG, arr[a][i], world_rank);
                sum = arr[a][i]+sum;
        }
        printf("\nSum is: %d at rank: %d and tag is:%d\n", sum, world_rank, status.MPI_TAG);
    }
}