Enabling Scientific Collaboration with JupyterHub

Scientific instruments are increasingly producing large amounts of data. However, instrument time at large-scale user facilities is a highly constrained resource. Research teams need to analyze experimental data in real-time to inform future experimentation, which creates challenges, especially when coupled with High Performance Computing (HPC) systems. Our work explores the use of live collaborative data analysis on HPC systems using the new Jupyter Real-Time Collaboration features to address these issues. We discuss enhancements to the Jupyter platform that were co-developed by our team to support collaboration at HPC centers like NERSC. Our work pays special attention to security and auditing requirements around user traceability. We discuss how users at the National Center for Electron Microscopy collaborated on a data collection and analysis run using this approach. Our work demonstrates real-time interactive, collaborative analysis, a critical part of emerging scientific workflows.