How to benchmark workflows

This section provides an overview on how to execute and benchmark workflows provided by the Workflomics platform. The execution is performed using the Workflomics Benchmarker command line tool.

Workflomics Benchmarker

The Workflomics Benchmarker is a command-line tool that benchmarks the performance of workflows generated by the Workflomics platform. The workflows must be specified in Common Workflow Language (CWL) format. The Workflomics Benchmarker generates a report containing performance metrics such as execution time, memory usage, and CPU usage for each workflow step, as well as scientific metrics (currently supporting proteomics domain) such as number of proteins identified, number of peptides identified, etc.

Benchmarker relies on cwltool to execute the workflows. The tool is written in Python and can be installed using pip install workflomics-benchmarker.

License

Workflomics Benchmarker is licensed under the Apache 2.0 license.

Contributors

This project was initially developed by students from the bachelor Computer Science at Utrecht University within the Software Project course, and students from Grafisch Lyceum Utrecht:
Silvan Eelman, Koen Haverkort, Alex Janse, Matthijs Rademaker, Megan Tjoeng, Inge van Dam, Jeroen van der Wal, Sem van Nieuwenhuizen, Rense Wolters, Sarrisa Wouts.

The project was later further developed by the following contributors:

  • Vedran Kasalica (v.kasalica[at]esciencecenter.nl),

  • Nauman Ahmed,

  • Peter Kok,

  • Rob Marissen,

  • Magnus Palmblad,

  • Anna-Lena Lamprecht