Local
All PerMedCoE Building Blocks and Workflows can be deployed and executed in local machines. To this end, this section provides step-by-step detailed instructions on the requirements, how to do the deployment, and how to run the workflows.
The first step is to make sure that the target machine has the required requirements.
Requirements
For local installations, the permedcoe
package is REQUIRED to be installed.
Since it is a Python package available in the Pypi repository, it can be easily installed using pip:
$ python3 -m pip install permedcoe
Defaulting to user installation because normal site-packages is not writeable
Collecting permedcoe
Downloading permedcoe-0.0.11-py3-none-any.whl (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.6/40.6 kB 2.7 MB/s eta 0:00:00
Requirement already satisfied: pyyaml in /home/user/.local/lib/python3.10/site-packages (from permedcoe) (6.0)
Installing collected packages: permedcoe
Successfully installed permedcoe-0.0.11
Tip
Alternatively, it is possible to be installed from source
It is also REQUIRED to install Apptainer (for automatic container download).
Please, check the Apptainer installation documentation.
Deployment
The deployment can be done for specific Building Blocks, or for complete Workflows (that automatically deploys its required Building Blocks). This section describes how to deploy a single Building Block and a complete Workflow:
Deploy an existing Building Block
Existing Building Blocks can be deployed automatically with the permedcoe
command
(provided by the permedcoe
package):
$ export PERMEDCOE_IMAGES=/path/where/to/store/the/containers/
$ permedcoe deploy building_block PhysiBoSS
[It may take a while since it downloads the required container]
------------------------------------------
------------ STDOUT ------------
Defaulting to user installation because normal site-packages is not writeable
Collecting git+https://github.com/PerMedCoE/BuildingBlocks.git@main#subdirectory=PhysiBoSS
Cloning https://github.com/PerMedCoE/BuildingBlocks.git (to revision main) to /tmp/pip-req-build-zdw3mlse
Resolved https://github.com/PerMedCoE/BuildingBlocks.git to commit 84071d6665edb4a8ea90249ffb5b8e2f583ff13a
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: permedcoe>=0.0.8 in /home/user/.local/lib/python3.10/site-packages (from meta-analysis-BB==0.0.3) (0.0.8)
Requirement already satisfied: pyyaml in /home/user/.local/lib/python3.10/site-packages (from permedcoe>=0.0.8->meta-analysis-BB==0.0.3) (6.0)
------------ STDERR ------------
Running command git clone --filter=blob:none --quiet https://github.com/PerMedCoE/BuildingBlocks.git /tmp/pip-req-build-zdw3mlse
------------------------------------------
The result of this command will install the PhysiBoSS
Building Block
and downloads its required container (stored in ${PERMEDCOE_IMAGES}
path).
Important
It is recommended to keep a single ${PERMEDCOE_IMAGES}
folder where to
store all Building Block containers.
Tip
A full list of the available Building Blocks can be found in Available Building Blocks Section.
From this point, the Building Block will be available in the machine, and it can be used in two ways: by invoking it directly through command line, or using its Python interface. This is further explained in the execution Section.
Deploy an existing Workflow
Existing Workflows can be deployed automatically with the permedcoe
command
(provided by the permedcoe
package):
$ export PERMEDCOE_IMAGES=/path/where/to/store/the/containers/
$ permedcoe deploy workflow covid-19-workflow
SUCCESS: Workflow deployed.
[It may take a while since it deploys all building blocks required by this workflow]
The result of this command will deploy the covid-19-workflow
Workflow
within the current directory. It further installs the required Building Blocks
as well as downloads their required containers (storing them in ${PERMEDCOE_IMAGES}
path).
Important
It is recommended to keep a single ${PERMEDCOE_IMAGES}
folder where to
store all Building Block containers.
Tip
A full list of the available workflows can be found in Existing Workflows Section.
From this point, the Workflow (as well as all its required Building Blocks) will be available in the machine. Its usage is explained in the usage Section.
Usage
This section aims at showing how to use individual Building Blocks and complete Workflows. Please, remind that Building Blocks are designed for a specific purpose, while the workflows use various Building Blocks in order to perform a particular analysis.
Use a Building Block
Building Blocks can be used in two ways: from the command line or through their Python interface.
- Command line
Each Building Block provides a command (with the same name as the building block followed by
_BB
) that can be launched from the command line. For example, the PhysiBoSS Building block (previously deployed) provides thePhysiBoSS_BB
command. Its usage details can be checked using the--help
or-h
flag:$ PhysiBoSS_BB --help usage: PhysiBoSS_BB [-h] --sample SAMPLE --repetition REPETITION --prefix PREFIX --bnd_file BND_FILE --cfg_file CFG_FILE --parallel PARALLEL --max_time MAX_TIME --out_file OUT_FILE --err_file ERR_FILE --results_dir RESULTS_DIR [-c CONFIG] [-d] [-l {debug,info,warning,error,critical}] [--tmpdir TMPDIR] [--processes PROCESSES] [--gpus GPUS] [--memory MEMORY] [--mount_points MOUNT_POINTS] This building block is used to perform a multiscale simulation of a population of cells using PhysiBoSS. The tool uses the different Boolean models personalised by the Personalise patient building block and with the mutants selected by the High-throughput mutant analysis building block. More information on this tool can be found in [Ponce-de-Leon et al. (2022)](https://www.biorxiv.org/content/10.1101/2022.01.06.468363v1) and the [PhysiBoSS GitHub repository](https://github.com/PhysiBoSS/PhysiBoSS). options: -h, --help show this help message and exit --sample SAMPLE (INPUT - str) Patient's identifier --repetition REPETITION (INPUT - int) Number of repetition to be performed --prefix PREFIX (INPUT - str) Name of the model --bnd_file BND_FILE (INPUT - str (file)) Name of the model's BND file --cfg_file CFG_FILE (INPUT - str (file)) Name of the model's CFG file --parallel PARALLEL (INPUT - int) Internal parallelism --max_time MAX_TIME (INPUT - int) PhysiBoSS simulation maximum time --out_file OUT_FILE (OUTPUT - str) Main output of the PhysiBoSS run --err_file ERR_FILE (OUTPUT - str) Error output of the PhysiBoSS run --results_dir RESULTS_DIR (OUTPUT - str) Results directory -c CONFIG, --config CONFIG (CONFIG) Configuration file path -d, --debug Enable Building Block debug mode. Overrides log_level -l {debug,info,warning,error,critical}, --log_level {debug,info,warning,error,critical} Set logging level --tmpdir TMPDIR Temp directory to be mounted in the container --processes PROCESSES Number of processes for MPI executions --gpus GPUS Requirements for GPU jobs --memory MEMORY Memory requirement --mount_points MOUNT_POINTS Comma separated alias:folder to be mounted in the container
Thanks to the
-h
or--help``flag, the PhysiBoSS Building Block inputs and outputs are shown and described. All ``INPUT
andOUTPUT
flags are required in order to execute the Building Block. The rest of the parameters are optional and can be used to define particular options (e.g.-l
or--log_level
can be used to define the level of information to be printed, or the--tmpdir
can be used to define a specific folder that will be mounted in the associated container – this can be useful if the input parameters are in a non default mounted folder).Caution
All Building Blocks require the
PERMEDCOE_IMAGES
environment variable. Otherwise, it will raise the following exception:Traceback (most recent call last): File "/home/user/.local/bin/meta_analysis_BB", line 5, in <module> from meta_analysis_BB.__main__ import main File "/home/user/.local/lib/python3.10/site-packages/meta_analysis_BB/__init__.py", line 6, in <module> from meta_analysis_BB.main import * File "/home/user/.local/lib/python3.10/site-packages/meta_analysis_BB/main.py", line 13, in <module> from meta_analysis_BB.definitions import CONTAINER File "/home/user/.local/lib/python3.10/site-packages/meta_analysis_BB/definitions.py", line 2, in <module> from permedcoe.bb import CONTAINER_PATH File "/home/user/.local/lib/python3.10/site-packages/permedcoe/bb.py", line 24, in <module> CONTAINER_PATH = get_container_path() File "/home/user/.local/lib/python3.10/site-packages/permedcoe/bb.py", line 21, in get_container_path raise Exception("Please define %s environment variable with the path." % CONTAINER_PATH_VN) Exception: Please define PERMEDCOE_IMAGES environment variable with the path.
The result of executing the Building Block will be a one or more output files that can potentially be used by other Building Block.
- Python interface
Since the Building Blocks are developed in Python, they offer a Python interface that can be used from Python applications.
For example, the PhysiBoSS Building Block can be used as follows:
from PhysiBoSS_BB import physiboss physiboss( sample="C141", repetition=1, prefix="epithelial_cell_2_personalized", bnd_file="/path/to/input_file.bnd", cfg_file="/path/to/input_file.cfg", out_file="/path/to/output_file.txt", err_file="/path/to/output_file.txt", results_dir="/path/to/results", parallel=1, max_time=8640, tmpdir="/path/to/tmpdir" )
Note that this interface requires the same parameters as the command line interface.
Tip
Some Building Blocks may provide more functions since they serve for multiple purposes (e.g. MaBoSS for default behaviour or for sensitivity analysis).
Specific details about the python interface of each Building Block can be consulted in the
main.py
file of each Building Block repository.Important
The functions provided by the Building Blocks have a set of decorators on top of them that are responsible of hiding the management complexities, but also to enable the automatic parallelization using PyCOMPSs. Consequently, any application making use of Building Blocks gets automatically parallelized if run using PyCOMPSs. If the application is run using Python directly, it will be executed sequentially.
Use a Workflow
Existing workflows that have been deployed have a particular structure:
$ cd covid-19-workflow
$covid-19-workflow> ls
BuildingBlocks LICENSE README.md Resources Tests Workflow
The contents are:
- LICENSE
The workflow license file.
- README.md
The workflow description file. It provides details about the workflow.
- BuildingBlocks
This folder contains a set of helper scripts to install or uninstall the Workflow required Building Blocks. They are used by the automatic deployment performed by the
permedcoe
package and command.- Resources
This folder contains a set of minimal data in order to run the Workflow (i.e. testing dataset).
- Tests
This folder contains a set of testing scripts. These scripts are able to run each individual Building Block for debugging purposes.
- Workflow
This is the MAIN FOLDER since it contains the workflow files. In particular, it can contain three subfolders:
$covid-19-workflow> cd Workflow $covid-19-workflow/Workflow> ls NextFlow PyCOMPSs SnakeMake
Each subfolder is aimed at contain the workflow for that particular workflow manager. For example, the covid-19 workflow is available in for the three workflow managers.
One peculiarity of the three workflow managers is that Snakemake and NextFlow workflows use the command line interface of the Building Blocks involved, while PyCOMPSs workflow uses their Python interface.
In addition to the workflow file, each folder also includes a
launch.sh
script in order to ease the workflow execution.
As seen in the contents, the Workflow
folder contains the main
workflow files. Consequently, it is necessary to go inside that
folder in order to execute the workflow.
$covid-19-workflow> cd Workflow
$covid-19-workflow/Workflow>
The next step is to decide which workflow manager is going to be used:
$covid-19-workflow/Workflow> ls
NextFlow PyCOMPSs SnakeMake
The covid-19 workflow is available for PyCOMPSs, NextFlow and SnakeMake.
Important
The workflow manager MUST be installed in the machine in order to run the workflow.
Once decided the workflow manager to be used, specific details about how to run the workflow with each of them is provided in the next drop-down sections:
PyCOMPSs
If the chosen workflow manager is PyCOMPSs, the next step is to go inside the folder:
$covid-19-workflow/Workflow> cd PyCOMPSs
$covid-19-workflow/Workflow/PyCOMPSs> ls
0_prepare_dataset.sh
a_launch.sh
a_run.sh
b_1_launch.sh
b_1_run.sh
b_2_launch_per_patient.sh
b_2_launch.sh
b_2_run_per_patient.sh
b_2_run.sh
b_3_launch.sh
b_3_run.sh
clean.sh
README.md
src
src_split
- 0_prepare_dataset.sh
This script downloads and configures the testing dataset. It just requires to be executed once and without parameters (
./0_prepare_dataset.sh
)- a_launch.sh
Script that launches the workflow with the testing dataset within a supercomputer.
- a_run.sh
Script that runs the workflow with the testing dataset.
- src
Folder that contains the workflow written in Python and using PyCOMPSs.
- clean.sh
Helper script that cleans the current folder after running the workflow. Use with caution since it removes all result files.
- src_split and
b_\*
scripts The
src_split
folder contains the workflow split in three parts, so that it can be executed partially or even in different machines. Accordingly, theb_\*
scripts are aimed at launching or running each part.
The way to run the workflow (automatically parallelized with PyCOMPSs) is:
$covid-19-workflow/Workflow/PyCOMPSs> ./a_run.sh
WARNING: PERMEDCOE_IMAGES environment variable not set. Using default: /home/user/github/projects/PerMedCoE/BuildingBlocks/Resources/images/
[ INFO ] Inferred PYTHON language
[ INFO ] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO ] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO ] Using default execution type: compss
----------------- Executing covid19_pilot.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(834) API] - Starting COMPSs Runtime v3.2.rc2310 (build 20231017-1637.r77b4be4b8ac4f722dd3de105161229b849a545d4)
---------------------------
| Covid-19 Pilot Workflow |
---------------------------
>>> WELCOME TO THE PILOT WORKFLOW
> Parameters:
- metadata file: /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/../../Resources/data/metadata_small.tsv
- model prefix: /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/../../Resources/data/epithelial_cell_2
- output folder: /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/
- ko file: /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/ko_file.txt
- replicates: 2
- model: epithelial_cell_2
- data folder: /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/../../Resources/data
- max time: 100
KO file not detected, running MABOSS
> SINGLE CELL PROCESSING C141
> PERSONALIZING PATIENT C141
>> prefix: epithelial_cell_2_personalized
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized_2.err
>> prefix: epithelial_cell_2_personalized__M_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__M_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__M_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__M_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__M_ko_2.err
>> prefix: epithelial_cell_2_personalized__CASP9_cell_active_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP9_cell_active_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP9_cell_active_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP9_cell_active_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP9_cell_active_ko_2.err
>> prefix: epithelial_cell_2_personalized__CASP8_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP8_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP8_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP8_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP8_ko_2.err
>> prefix: epithelial_cell_2_personalized__FASLG_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FASLG_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FASLG_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FASLG_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FASLG_ko_2.err
>> prefix: epithelial_cell_2_personalized__FADD_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FADD_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FADD_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FADD_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FADD_ko_2.err
>> prefix: epithelial_cell_2_personalized__CASP3_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP3_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP3_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP3_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__CASP3_ko_2.err
>> prefix: epithelial_cell_2_personalized__FAS_FASL_complex_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FAS_FASL_complex_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FAS_FASL_complex_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FAS_FASL_complex_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__FAS_FASL_complex_ko_2.err
>> prefix: epithelial_cell_2_personalized__Apoptosome_complex_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__Apoptosome_complex_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__Apoptosome_complex_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__Apoptosome_complex_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_results/output_C141_epithelial_cell_2_personalized__Apoptosome_complex_ko_2.err
> SINGLE CELL PROCESSING C142
> PERSONALIZING PATIENT C142
>> prefix: epithelial_cell_2_personalized
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized_2.err
>> prefix: epithelial_cell_2_personalized__M_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__M_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__M_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__M_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__M_ko_2.err
>> prefix: epithelial_cell_2_personalized__CASP9_cell_active_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP9_cell_active_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP9_cell_active_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP9_cell_active_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP9_cell_active_ko_2.err
>> prefix: epithelial_cell_2_personalized__CASP8_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP8_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP8_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP8_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP8_ko_2.err
>> prefix: epithelial_cell_2_personalized__FASLG_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FASLG_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FASLG_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FASLG_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FASLG_ko_2.err
>> prefix: epithelial_cell_2_personalized__FADD_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FADD_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FADD_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FADD_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FADD_ko_2.err
>> prefix: epithelial_cell_2_personalized__CASP3_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP3_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP3_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP3_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__CASP3_ko_2.err
>> prefix: epithelial_cell_2_personalized__FAS_FASL_complex_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FAS_FASL_complex_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FAS_FASL_complex_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FAS_FASL_complex_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__FAS_FASL_complex_ko_2.err
>> prefix: epithelial_cell_2_personalized__Apoptosome_complex_ko
>>> Repetition: 1
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__Apoptosome_complex_ko_1.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__Apoptosome_complex_ko_1.err
>>> Repetition: 2
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__Apoptosome_complex_ko_2.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_results/output_C142_epithelial_cell_2_personalized__Apoptosome_complex_ko_2.err
>> prefix: epithelial_cell_2_personalized
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized.err
>> prefix: epithelial_cell_2_personalized__M_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__M_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__M_ko.err
>> prefix: epithelial_cell_2_personalized__CASP9_cell_active_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP9_cell_active_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP9_cell_active_ko.err
>> prefix: epithelial_cell_2_personalized__CASP8_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP8_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP8_ko.err
>> prefix: epithelial_cell_2_personalized__FASLG_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__FASLG_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__FASLG_ko.err
>> prefix: epithelial_cell_2_personalized__FADD_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__FADD_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__FADD_ko.err
>> prefix: epithelial_cell_2_personalized__CASP3_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP3_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP3_ko.err
>> prefix: epithelial_cell_2_personalized__FAS_FASL_complex_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__FAS_FASL_complex_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__FAS_FASL_complex_ko.err
>> prefix: epithelial_cell_2_personalized__Apoptosome_complex_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__Apoptosome_complex_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C141/physiboss_replicates_analysis/epithelial_cell_2_personalized__Apoptosome_complex_ko.err
>> prefix: epithelial_cell_2_personalized
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized.err
>> prefix: epithelial_cell_2_personalized__M_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__M_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__M_ko.err
>> prefix: epithelial_cell_2_personalized__CASP9_cell_active_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP9_cell_active_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP9_cell_active_ko.err
>> prefix: epithelial_cell_2_personalized__CASP8_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP8_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP8_ko.err
>> prefix: epithelial_cell_2_personalized__FASLG_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__FASLG_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__FASLG_ko.err
>> prefix: epithelial_cell_2_personalized__FADD_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__FADD_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__FADD_ko.err
>> prefix: epithelial_cell_2_personalized__CASP3_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP3_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__CASP3_ko.err
>> prefix: epithelial_cell_2_personalized__FAS_FASL_complex_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__FAS_FASL_complex_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__FAS_FASL_complex_ko.err
>> prefix: epithelial_cell_2_personalized__Apoptosome_complex_ko
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__Apoptosome_complex_ko.out
- /home/user/github/projects/PerMedCoE/covid-19-workflow/Workflow/PyCOMPSs/results/C142/physiboss_replicates_analysis/epithelial_cell_2_personalized__Apoptosome_complex_ko.err
[(1277810) API] - Execution Finished
------------------------------------------------------------
And the results will be stored within the current folder within the results
folder.
$covid-19-workflow/Workflow/PyCOMPSs> cd results
$covid-19-workflow/Workflow/PyCOMPSs/results> tree
.
├── C141
│ ├── personalize_patient
│ │ └── [personalize_patient results]
│ ├── physiboss_replicates_analysis
│ │ └── [physiboss_replicates_analysis results]
│ ├── physiboss_results
│ │ └── [physiboss_results results]
│ └── single_cell_processing
│ │ └── [single_cell_processing results]
├── C142
│ ├── personalize_patient
│ │ └── [personalize-patient-results]
│ ├── physiboss_replicates_analysis
│ │ └── [physiboss_replicates_analysis results]
│ ├── physiboss_results
│ │ └── [physiboss_results results]
│ └── single_cell_processing
│ └── [single_cell_processing results]
└── meta_analysis
├── cd8s_C141_Apoptosome_complex_ko_.png
├── cd8s_C141_CASP3_ko_.png
├── cd8s_C141_CASP8_ko_.png
├── cd8s_C141_CASP9_cell_active_ko_.png
├── cd8s_C141_FADD_ko_.png
├── cd8s_C141_FAS_FASL_complex_ko_.png
├── cd8s_C141_FASLG_ko_.png
├── cd8s_C141_M_ko_.png
├── cd8s_C141.png
├── cd8s_C142_Apoptosome_complex_ko_.png
├── cd8s_C142_CASP3_ko_.png
├── cd8s_C142_CASP8_ko_.png
├── cd8s_C142_CASP9_cell_active_ko_.png
├── cd8s_C142_FADD_ko_.png
├── cd8s_C142_FAS_FASL_complex_ko_.png
├── cd8s_C142_FASLG_ko_.png
├── cd8s_C142_M_ko_.png
├── cd8s_C142.png
├── cd8_traces_C141_Apoptosome_complex_ko_.png
├── cd8_traces_C141_CASP3_ko_.png
├── cd8_traces_C141_CASP8_ko_.png
├── cd8_traces_C141_CASP9_cell_active_ko_.png
├── cd8_traces_C141_FADD_ko_.png
├── cd8_traces_C141_FAS_FASL_complex_ko_.png
├── cd8_traces_C141_FASLG_ko_.png
├── cd8_traces_C141_M_ko_.png
├── cd8_traces_C141.png
├── cd8_traces_C142_Apoptosome_complex_ko_.png
├── cd8_traces_C142_CASP3_ko_.png
├── cd8_traces_C142_CASP8_ko_.png
├── cd8_traces_C142_CASP9_cell_active_ko_.png
├── cd8_traces_C142_FADD_ko_.png
├── cd8_traces_C142_FAS_FASL_complex_ko_.png
├── cd8_traces_C142_FASLG_ko_.png
├── cd8_traces_C142_M_ko_.png
├── cd8_traces_C142.png
├── clustermap_genes.png
├── clustermap_patients.png
├── clustermap.png
├── clustermap_traces.png
├── dendogram_genes.png
├── dendogram_patients.png
├── dendogram.png
├── dendogram_traces.png
├── epithelials_C141_Apoptosome_complex_ko_.png
├── epithelials_C141_CASP3_ko_.png
├── epithelials_C141_CASP8_ko_.png
├── epithelials_C141_CASP9_cell_active_ko_.png
├── epithelials_C141_FADD_ko_.png
├── epithelials_C141_FAS_FASL_complex_ko_.png
├── epithelials_C141_FASLG_ko_.png
├── epithelials_C141_M_ko_.png
├── epithelials_C141.png
├── epithelials_C142_Apoptosome_complex_ko_.png
├── epithelials_C142_CASP3_ko_.png
├── epithelials_C142_CASP8_ko_.png
├── epithelials_C142_CASP9_cell_active_ko_.png
├── epithelials_C142_FADD_ko_.png
├── epithelials_C142_FAS_FASL_complex_ko_.png
├── epithelials_C142_FASLG_ko_.png
├── epithelials_C142_M_ko_.png
├── epithelials_C142.png
├── epithelials_traces_C141_Apoptosome_complex_ko_.png
├── epithelials_traces_C141_CASP3_ko_.png
├── epithelials_traces_C141_CASP8_ko_.png
├── epithelials_traces_C141_CASP9_cell_active_ko_.png
├── epithelials_traces_C141_FADD_ko_.png
├── epithelials_traces_C141_FAS_FASL_complex_ko_.png
├── epithelials_traces_C141_FASLG_ko_.png
├── epithelials_traces_C141_M_ko_.png
├── epithelials_traces_C141.png
├── epithelials_traces_C142_Apoptosome_complex_ko_.png
├── epithelials_traces_C142_CASP3_ko_.png
├── epithelials_traces_C142_CASP8_ko_.png
├── epithelials_traces_C142_CASP9_cell_active_ko_.png
├── epithelials_traces_C142_FADD_ko_.png
├── epithelials_traces_C142_FAS_FASL_complex_ko_.png
├── epithelials_traces_C142_FASLG_ko_.png
├── epithelials_traces_C142_M_ko_.png
├── epithelials_traces_C142.png
├── macrophages_C141_Apoptosome_complex_ko_.png
├── macrophages_C141_CASP3_ko_.png
├── macrophages_C141_CASP8_ko_.png
├── macrophages_C141_CASP9_cell_active_ko_.png
├── macrophages_C141_FADD_ko_.png
├── macrophages_C141_FAS_FASL_complex_ko_.png
├── macrophages_C141_FASLG_ko_.png
├── macrophages_C141_M_ko_.png
├── macrophages_C141.png
├── macrophages_C142_Apoptosome_complex_ko_.png
├── macrophages_C142_CASP3_ko_.png
├── macrophages_C142_CASP8_ko_.png
├── macrophages_C142_CASP9_cell_active_ko_.png
├── macrophages_C142_FADD_ko_.png
├── macrophages_C142_FAS_FASL_complex_ko_.png
├── macrophages_C142_FASLG_ko_.png
├── macrophages_C142_M_ko_.png
└── macrophages_C142.png
11 directories, 100 files
NextFlow
If the chosen workflow manager is NextFlow, the next step is to go inside the folder:
$covid-19-workflow/Workflow> cd NextFlow
$covid-19-workflow/Workflow/NextFlow> ls
covid19_pilot.nf launch.sh
- covid-19_pilot.nf
This is the workflow script.
- launch.sh
Script that launches the workflow with the testing dataset.
The way to run the workflow is:
$covid-19-workflow/Workflow/NextFlow> ./launch.sh
[Wait for completion]
And the results will be stored within the current folder within the results
folder.
SnakeMake
If the chosen workflow manager is SnakeMake, the next step is to go inside the folder:
$covid-19-workflow/Workflow> cd SnakeMake
$covid-19-workflow/Workflow/SnakeMake> ls
config.yml launch.sh run.sh Snakefile split.sh
- config.yml
Configuration file.
- launch.sh
Script that launches the workflow with the testing dataset using SLURM.
- run.sh
Script that launches the workflow with the testing dataset.
- Snakefile
This is the workflow script.
- split.sh
Helper script required by the Snakefile.
The way to run the workflow is:
$covid-19-workflow/Workflow/SnakeMake> ./run.sh
[Wait for completion]
And the results will be stored within the current folder within the results
folder.