Defining and benchmarking open problems in single-cell analysis
Single-cell genomics has enabled the study of biological processes at an unprecedented scale and resolution. These studies were enabled by innovative data generation technologies coupled with emerging computational tools specialized for single-cell data. As single-cell technologies have become more prevalent, so has the development of new analysis tools, which has resulted in over 1,700 published algorithms (as of February 2024). Thus, there is an increasing need to continually evaluate which algorithm performs best in which context to inform best practices that evolve with the field.
In many fields of quantitative science, public competitions and benchmarks address this need by evaluating state-of-the-art methods against known criteria, following the concept of a common task framework. Here, we present Open Problems, a living, extensive, community-guided platform including 12 current single-cell tasks that we envisage raising standards for the selection, evaluation and development of methods in single-cell analysis.
Citation
@article{luecken2025,
author = {Luecken, Malte D. and Gigante, Scott and Burkhardt, Daniel
B. and Cannoodt, Robrecht and Strobl, Daniel C. and Markov, Nikolay
S. and Zappia, Luke and Palla, Giovanni and Lewis, Wesley and
Dimitrov, Daniel and Vinyard, Michael E. and Magruder, D.S. and
Mueller, Michaela F. and Andersson, Alma and Dann, Emma and Qin,
Qian and Otto, Dominik J. and Klein, Michal and Botvinnik, Olga
Borisovna and Deconinck, Louise and Waldrant, Kai and Yasa, Sai
Nirmayi and Szałata, Artur and Benz, Andrew and Li, Zhijian and
{Open Problems Jamboree Members} and Bloom, Jonathan M. and Oliveira
Pisco, Angela and Saez-Rodriguez, Julio and Wulsin, Drausin and
Pinello, Luca and Saeys, Yvan and Theis, Fabian J and Krishnaswamy,
Smita},
title = {Defining and Benchmarking Open Problems in Single-Cell
Analysis},
journal = {Nature biotechnology},
volume = {43},
number = {7},
pages = {1035-1040},
date = {2025-07-01},
url = {https://doi.org/10.1038/s41587-025-02694-w},
doi = {10.1038/s41587-025-02694-w},
issn = {1087-0156},
langid = {en},
abstract = {Single-cell genomics has enabled the study of biological
processes at an unprecedented scale and resolution. These studies
were enabled by innovative data generation technologies coupled with
emerging computational tools specialized for single-cell data. As
single-cell technologies have become more prevalent, so has the
development of new analysis tools, which has resulted in over 1,700
published algorithms (as of February 2024). Thus, there is an
increasing need to continually evaluate which algorithm performs
best in which context to inform best practices that evolve with the
field. In many fields of quantitative science, public competitions
and benchmarks address this need by evaluating state-of-the-art
methods against known criteria, following the concept of a common
task framework. Here, we present Open Problems, a living, extensive,
community-guided platform including 12 current single-cell tasks
that we envisage raising standards for the selection, evaluation and
development of methods in single-cell analysis.}
}