Real-life Experience with Private Cloud hosting Heterogeneous Scientific Workloads

Warning

This publication doesn't include Faculty of Arts. It includes Institute of Computer Science. Official publication website can be found on muni.cz.
Authors

KLUSÁČEK Dalibor PARÁK Boris CHLUMSKÝ Václav HEJTMÁNEK Lukáš

Year of publication 2018
Type Article in Proceedings
Conference 2018 IEEE/ACM INTERNATIONAL CONFERENCE ON UTILITY AND CLOUD COMPUTING COMPANION (UCC COMPANION)
MU Faculty or unit

Institute of Computer Science

Citation
Web https://ieeexplore.ieee.org/document/8605767
Doi http://dx.doi.org/10.1109/UCC-Companion.2018.00044
Keywords cloud; batch computing; HPC; scheduling; container; Singularity; Docker
Description We present our experience with hosting scientific workloads in a private scientific cloud, where both the infrastructure as well as the workloads are heterogeneous. We support two major classes of workloads-classic virtual machines (VMs) and batch job computations. Furthermore, containerized applications and scientific portals such as Galaxy are also hosted in this infrastructure. Using our experience, we describe the system setup, the technologies used to run this heterogeneous system as well as some of the problems we have faced when managing this system throughout the years. Modern computing environments such as clouds, grids or HPC clusters are both complex and costly installations. Therefore, it has always been a major challenge to utilize them properly. Things like improper setup or bad scheduling policy may easily hamper overall performance of the whole system. We believe that our experience and observations may help other researchers and system administrators identify new research directions and/ or potential weak spots. Importantly, the problems discussed in this paper are based on real-life data from the CERIT Scientific Cloud that we freely offer for further analysis and/ or simulations.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.