Experiences with workflows for automating data-intensive bioinformatics.
Ola Spjuth, Erik Bongcam-Rudloff, Guillermo Carrasco Hernández, Lukas Forer, Mario Giovacchini, Roman Valls Guimera, Aleksi Kallio, Eija Korpelainen, Maciej M Kańduła, Milko Krachunov, David P Kreil, Ognyan Kulev, Paweł P Łabaj, Samuel Lampa, Luca Pireddu, Sebastian Schönherr, Alexey Siretskiy, Dimitar Vassilev
Author Information
Ola Spjuth: Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, SE-75124, Uppsala, P.O. Box 591, Sweden. ola.spjuth@farmbio.uu.se.
Erik Bongcam-Rudloff: SLU-Global Bioinformatics Centre, Department of Animal Breeding and Genetics, Swedish University of Agricultural Sciences, Uppsala, Sweden. Erik.Bongcam@slu.se.
Guillermo Carrasco Hernández: Science for Life Laboratory, Karolinska Institutet, SE-17121, Stockholm, P.O. Box 1031, Sweden. guillermo.carrasco@scilifelab.se.
Lukas Forer: Division of Genetic Epidemiology, Medical University of Innsbruck, Innsbruck, 6020, Austria. lukas.forer@i-med.ac.at.
Mario Giovacchini: Science for Life Laboratory, Karolinska Institutet, SE-17121, Stockholm, P.O. Box 1031, Sweden. mario.giovacchini@scilifelab.se.
Roman Valls Guimera: Science for Life Laboratory, Karolinska Institutet, SE-17121, Stockholm, P.O. Box 1031, Sweden. brainstorm@nopcode.org.
Aleksi Kallio: CSC - IT Center for Science Ltd., FI-02101, Espoo, P.O. Box 405, Finland. aleksi.kallio@csc.fi.
Eija Korpelainen: CSC - IT Center for Science Ltd., FI-02101, Espoo, P.O. Box 405, Finland. eija.korpelainen@csc.fi.
Maciej M Kańduła: Chair of Bioinformatics Research Group, Boku University, Vienna, Austria. maciej.kandula@boku.ac.at.
Milko Krachunov: Faculty of Mathematics and Informatics, Sofia University, Sofia, Bulgaria. wfxp@milko.3mhz.net.
David P Kreil: Chair of Bioinformatics Research Group, Boku University, Vienna, Austria. david.kreil@boku.ac.at.
Ognyan Kulev: Faculty of Mathematics and Informatics, Sofia University, Sofia, Bulgaria. okulev@fmi.uni-sofia.bg.
Paweł P Łabaj: Chair of Bioinformatics Research Group, Boku University, Vienna, Austria. pawel.labaj@boku.ac.at.
Samuel Lampa: Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, SE-75124, Uppsala, P.O. Box 591, Sweden. samuel.lampa@it.uu.se.
High-throughput technologies, such as next-generation sequencing, have turned molecular biology into a data-intensive discipline, requiring bioinformaticians to use high-performance computing resources and carry out data management and analysis tasks on large scale. Workflow systems can be useful to simplify construction of analysis pipelines that automate tasks, support reproducibility and provide measures for fault-tolerance. However, workflow systems can incur significant development and administration overhead so bioinformatics pipelines are often still built without them. We present the experiences with workflows and workflow systems within the bioinformatics community participating in a series of hackathons and workshops of the EU COST action SeqAhead. The organizations are working on similar problems, but we have addressed them with different strategies and solutions. This fragmentation of efforts is inefficient and leads to redundant and incompatible solutions. Based on our experiences we define a set of recommendations for future systems to enable efficient yet simple bioinformatics workflow construction and execution.
References
Bioinformatics. 2004 Nov 22;20(17):3045-54
[PMID: 15201187]
