Application Templates
Application template creation
The first suggested step is to start with a template created with the
permedcoe
command:
permedcoe template application my_application
The result of this command is a folder containing a three folders, one for the three supported workflows managers (PyCOMPSs, Nextflow and Snakemake), where a template for each of them is provided.
Hint
The application template creation provides the main actions to be performed in order to complete your building block:
------------------------------------------
To be completed:
- app.py:(9): TODO: Import the desired building blocks and use invoke or any other function.
- Snakefile:(0): TODO: Declare the building blocks to be used as rules.
- Snakefile:(9): TODO: Change bb to the building block name.
- NextFlow.nf:(7): TODO: Declare the building blocks to be used as process.
- NextFlow.nf:(18): TODO: Change bb to the building block name.
------------------------------------------
Note
The template provides support for PyCOMPSs, Nextflow and Snakemake, but it is not mandatory to implement the application for all of them. However, this decision will tight your application to a single workflow manager that needs to be available where the execution is going to take place.
Folder structure
The application contains the following scripts:
.
├── NextFlow
│ ├── launch.sh
│ └── NextFlow.nf
├── PyCOMPSs
│ ├── app.py
│ ├── launch.sh
│ ├── launch_without_pycompss.sh
└── SnakeMake
├── launch.sh
└── Snakefile
File name |
Description |
NextFlow |
Folder containing the Nextflow application template |
NextFlow/launch.sh |
Launch with Nextflow script |
NextFlow/NextFlow.nf |
Application template for Nextflow |
PyCOMPSs |
Folder containing the PyCOMPSs application template |
PyCOMPSs/app.py |
Application template for PyCOMPSs |
PyCOMPSs/launch.sh |
Launch with PyCOMPSs script |
PyCOMPSs/launch_without_pycompss.sh |
Launch without PyCOMPSs script |
SnakeMake |
Folder containing the Snakemake application template |
SnakeMake/launch.sh |
Launch with Snakemake script |
SnakeMake/Snakefile |
Application template for Snakemake |
The developer responsibility is to complete at least one of the following files:
NextFlow.nf
app.py
Snakefile
And the mainly expected content of the application is the usage of building blocks.
Tip
Due to the Python nature of PyCOMPSs applications, it can also support python code and objects among building blocks, enabling the implementation of complex workflows with inner parallelism.
Execution
The execution of the application depends on the workflow manager choosen. Consequently, the execution will be performed following the worklflow manager instructions.
For example, the execution with PyCOMPSs locally is be performed with the runcompss
command (see
runcompss documentation for
more details), whilst for supercomputers is performed with the enqueue_compss
command (see
enqueue_compss documentation for
more details).
Best practices
There are a set of best practices suggested to application developers: