MN4

The MareNostrum 4 (MN4) supercomputer is designated as a Special Scientific/Technical Infrastructure Facility by the Spanish Ministry of Economy, Industry and Competitiveness and is part of the PRACE Research Infrastructure as one of the 7 Tier-0 Systems currently available for European scientists.

More details on its structure and specifications can be found in the MN4 documentation.

Requirements

The main requirement is to have access to the MN4 Supercomputer.

With the access granted, the first step is to connect to the MN4 Supercomputer. Specific instructions to this end are provided in the MN4 documentation.

Deployment

It is not necessary to deploy the Building Blocks, neither the Workflows, since they have already been deployed and are available to all users by means of a module.

The users only need to load the required workflow manager (which in this example is PyCOMPSs), singularity, and the permedcoe module, which enables to get any of the available workflows:

$ export COMPSS_PYTHON_VERSION=3
$ module load COMPSs/3.2
load java/8u131 (PATH, MANPATH, JAVA_HOME, JAVA_ROOT, JAVA_BINDIR, SDK_HOME, JDK_HOME, JRE_HOME)
load papi/5.5.1 (PATH, LD_LIBRARY_PATH, C_INCLUDE_PATH)
load PYTHON/3.7.4 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME, PYTHONPATH)
load COMPSs/3.2 (PATH, CLASSPATH, MANPATH, GAT_LOCATION, COMPSS_HOME, JAVA_TOOL_OPTIONS, LDFLAGS, CPPFLAGS)
$ module load singularity/3.5.2
load SINGULARITY/3.5.2 (PATH)
$ module use /apps/modules/modulefiles/tools/COMPSs/libraries
$ module load permedcoe
load permedcoe (PATH, MANPATH, IT_HOME)
Using Python 3.7.4.

Now, the permedcoe tool is available, as well as some other commands that will ease the workflow get. In particular, the available commands are:

get_covid19workflow

Invoking this command will copy the Covid19 workflow in the current directory. Since all building blocks are available, there is no need to do anything else.

get_drug_synergies_workflow

Invoking this command will copy the Drug Synergies workflow in the current directory. Since all building blocks are available, there is no need to do anything else.

get_single_drug_prediction_workflow

Invoking this command will copy the Single Drug Prediction workflow in the current directory. Since all building blocks are available, there is no need to do anything else.

get_cancer_invasion_workflow

Invoking this command will copy the Cancer Invasion workflow in the current directory. Since all building blocks are available, there is no need to do anything else.

For example, if the target workflow is Cancer Invasion, it is only necessary to:

$ get_cancer_invasion_workflow
$ ls
cancer-invasion-workflow
$ cd cancer-invasion-workflow
$cancer-invasion-workflow> ls
BuildingBlocks  LICENSE  README.md  Resources  Tests  Workflow

The result is the same as in the local deployment.

Execution

Once achieved the workflow sources, the next step is to go inside the Workflow folder, and inside the folder for the selected workflow manager (which in this case is PyCOMPSs):

$cancer-invasion-workflow> cd Workflow/PyCOMPSs
$cancer-invasion-workflow/Workflow/PyCOMPSs> ls
clean.sh  launch.sh  README.md  results  run.sh  src

Inside this folder, there is a launch.sh script that submits the execution of the Cancer Invasion workflow to the queuing system (SLURM).

Important

This script contains the workflow input parameters. So, if you want to run it with different configurations, it will be necessary to edit it and update the required parameters (e.g. other dataset, more replicas, or more simulation time).

$cancer-invasion-workflow/Workflow/PyCOMPSs> ./launch.sh
load java/8u131 (PATH, MANPATH, JAVA_HOME, JAVA_ROOT, JAVA_BINDIR, SDK_HOME, JDK_HOME, JRE_HOME)
load papi/5.5.1 (PATH, LD_LIBRARY_PATH, C_INCLUDE_PATH)
load PYTHON/3.7.4 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME, PYTHONPATH)
load COMPSs/3.2 (PATH, CLASSPATH, MANPATH, GAT_LOCATION, COMPSS_HOME, JAVA_TOOL_OPTIONS, LDFLAGS, CPPFLAGS)
load SINGULARITY/3.5.2 (PATH)
remove permedcoe (PATH, MANPATH, IT_HOME)
Using Python 3.7.4.
load permedcoe (PATH, MANPATH, IT_HOME)
Using Python 3.7.4.
SC Configuration:          default.cfg
JobName:                   COMPSs
Queue:                     default
Deployment:                Master-Worker
Reservation:               disabled
Num Nodes:                 3
Num Switches:              0
GPUs per node:             0
Job dependency:            None
Exec-Time:                 02:00:00
QoS:                       debug
Constraints:               disabled
Storage Home:              null
Storage Properties:
Storage container image:   false
Storage cpu affinity:      disabled
Other:                     --wall_clock_limit=7140
            --sc_cfg=default.cfg
            --qos=debug
            --worker_working_dir=/gpfs/projects/bscXX/bscXXYYY/PROYECTS/PerMedCoE/cancer-invasion-workflow/Workflow/PyCOMPSs
            --log_level=off
            --graph
            --tracing
            --generate_trace=true
            --python_interpreter=python3 /gpfs/projects/bscXX/bscXXYYY/PROYECTS/PerMedCoE/cancer-invasion-workflow/Workflow/PyCOMPSs/src/cancer_invasion.py /gpfs/projects/bscXX/bscXXYYY/PROYECTS/PerMedCoE/cancer-invasion-workflow/Workflow/PyCOMPSs/../../Resources/data//parameters_small.csv /gpfs/projects/bscXX/bscXXYYY/PROYECTS/PerMedCoE/cancer-invasion-workflow/Workflow/PyCOMPSs/results/ 5 4500

Temp submit script is: /scratch/tmp/tmp.dxXN9zJOu7
Requesting 144 processes
Submitted batch job 30404848

The result of submitting the job is its identifier, which in this example is 30404848. The job status can be checked with squeue:

$ squeue
JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
30404848      main   COMPSs bscXXYYY  R       1:37      3 s01r1b25,s24r1b[60,63]

Which in this example, shows that it has already started and the workflow is being executed.

After executing the whole workflow, three main elements will appear:

compss-30404848.err

This file contains the standard error messages shown during the execution. Any issue will appear in this file.

compss-30404848.out

This file contains the standard output messages shown during the execution. In particular, it displays the PyCOMPSs execution output.

results

This folder will contain the workflow results:

$ cd results
$results> tree
.
├── plots
│   ├── migration_bias.png
│   └── migration_bias_ratio.png
└── simulations
    ├── parameter_0
    │   └── [intermediate results]
    ├── parameter_1
    │   └── [intermediate results]
    ├── parameter_10
    │   └── [intermediate results]
    ├── parameter_2
    │   └── [intermediate results]
    ├── parameter_3
    │   └── [intermediate results]
    ├── parameter_4
    │   └── [intermediate results]
    ├── parameter_5
    │   └── [intermediate results]
    ├── parameter_6
    │   └── [intermediate results]
    ├── parameter_7
    │   └── [intermediate results]
    ├── parameter_8
    │   └── [intermediate results]
    └── parameter_9
        └── [intermediate results]

And thats it!