1. 概述

本篇博客主要介绍在centos7.9部署和测试mpi并行程序开发环境，并通过module加载不同的环境。

2. 部署过程

2.1 安装mpich

节点安装相关依赖环境：yum -y install gcc-gfortran

下载mpich-4.0.2.tar.gz，解压

创建编译安装脚本liwl.sh

#!/bin/bash
INSTALL=/opt/hpc/mpi/mpich/4.0.2
./configure --prefix=${INSTALL}

执行bash liwl.sh && make && make install，完成mpich编译安装

2.2 安装openmpi

下载openmpi-4.1.4.tar.gz源码包，解压后，进入解压目录，在目录下创建编译安装脚本liwl.sh

#!/bin/bash
INSTALL=/opt/hpc/mpi/openmpi/4.1.4
./configure --prefix=${INSTALL}

执行bash liwl.sh && make && make install

2.3 安装module

yum -y install environment-modules

配置moudle，增加mpi：vim /etc/modulefiles/mpi/mpich/4.0.2，内容如下

#%Module
prepend-path PATH /opt/hpc/mpi/mpich/4.0.2/bin
prepend-path LIBRARY_PATH /opt/hpc/mpi/mpich/4.0.2/lib
prepend-path LD_LIBRARY_PATH /opt/hpc/mpi/mpich/4.0.2/lib

配置module，增加openmpi：vim /etc/modulefiles/mpi/openmpi/4.1.4/bin，内容如下：

#%Module
prepend-path PATH /opt/hpc/mpi/openmpi/4.1.4/bin
prepend-path LIBRARY_PATH /opt/hpc/mpi/openmpi/4.1.4/lib
prepend-path LD_LIBRARY_PATH /opt/hpc/mpi/openmpi/4.1.4/lib

2.4 环境测试

执行module avi

$ module avi

------------------------------- /usr/share/Modules/modulefiles ----------------------------------------------------------
dot         module-git  module-info modules     null        use.own

-------------------------------- /etc/modulefiles -------------------------------------------------------------------------
mpi/mpich/4.0.2   mpi/openmpi/4.1.4

执行module load mpi/openmpi/4.1.4，加载选择的环境，通过which mpicc验证

2.5 编程测试

创建hello.c

#include <mpi.h>
#include <unistd.h>
#include <stdio.h>

int main(int argc, char** argv) {
    // Initialize the MPI environment
    MPI_Init(NULL, NULL);

    // Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    // Get the name of the processor
    char processor_name[MPI_MAX_PROCESSOR_NAME];
    int name_len;
    MPI_Get_processor_name(processor_name, &name_len);

    // Print off a hello world message
    printf("Hello world from processor [%s], rank [%d] out of [%d] processors.begint to sleep...\n", processor_name, world_rank, world_size);
    sleep(60);

    // Finalize the MPI environment.
    MPI_Finalize();
    printf("sleep over.\n");
}

先用mpich编译

[liwl@node17][~/online1/mpich]
$ module load mpi/mpich/4.0.2
[liwl@node17][~/online1/mpich]
$ which mpicc
/opt/hpc/mpi/mpich/4.0.2/bin/mpicc
[liwl@node17][~/online1/mpich]
$ mpicc -o hello hello.c 
[liwl@node17][~/online1/mpich]
$ ldd hello
        linux-vdso.so.1 =>  (0x00007ffe7bdf6000)
        libmpi.so.12 => /opt/hpc/mpi/mpich/4.0.2/lib/libmpi.so.12 (0x00007f89b9803000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f89b9435000)
        libhwloc.so.5 => /lib64/libhwloc.so.5 (0x00007f89b91f8000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f89b8ef6000)
        librdmacm.so.1 => /lib64/librdmacm.so.1 (0x00007f89b8cdf000)
        libibverbs.so.1 => /lib64/libibverbs.so.1 (0x00007f89b8ac6000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f89b88aa000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f89b86a6000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f89b849e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f89bc226000)
        libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f89b8292000)
        libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f89b8088000)
        libnl-route-3.so.200 => /lib64/libnl-route-3.so.200 (0x00007f89b7e1b000)
        libnl-3.so.200 => /lib64/libnl-3.so.200 (0x00007f89b7bfa000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f89b79e4000)

再用openmpi编译

[liwl@node17][~/online1/openmpi]
$ module load mpi/openmpi/4.1.4
[liwl@node17][~/online1/openmpi]
$ which mpicc
/opt/hpc/mpi/openmpi/4.1.4/bin/mpicc
[liwl@node17][~/online1/openmpi]
$ mpicc  -o hello hello.c 
[liwl@node17][~/online1/openmpi]
$ ldd hello
        linux-vdso.so.1 =>  (0x00007fff47ffa000)
        libmpi.so.40 => /opt/hpc/mpi/openmpi/4.1.4/lib/libmpi.so.40 (0x00007fd8ac2d3000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fd8ac0b7000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fd8abce9000)
        libopen-rte.so.40 => /opt/hpc/mpi/openmpi/4.1.4/lib/libopen-rte.so.40 (0x00007fd8aba32000)
        libopen-pal.so.40 => /opt/hpc/mpi/openmpi/4.1.4/lib/libopen-pal.so.40 (0x00007fd8ab722000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fd8ab51e000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fd8ab316000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fd8ab014000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007fd8aae11000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fd8aabfb000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd8ac5eb000)

很明显能够看出可执行程序hello链接到了不同的库。最后使用mpirun或者slurm调度系统提交作业即可。