Harnessing the power of Jupyter{Hub,Lab} to make Jean Zay HPC resources more accessible

Spawning jupyter servers is at the heart of the JupyterHub. This talk will present the details of the custom spawner that has been implemented on the top of WrapSpawner[1]. This spawner allows the users to launch Jupyter servers either on login nodes or compute nodes of Jean Zay. JupyterHub needs privilege escalation to launch Jupyter servers as users and conventionally, this is done via sudo[2]. An alternative approach using SSH certificates will be demonstrated and how it is integrated with the custom spawner. Thanks to such approach, the entire JupyterHub deployment runs in the userland without any extra privileges. Moreover, hardening approaches on the nginx (used as reverse proxy) will be presented which can further reduce the attack surface.

Given that JupyterLab is highly extensible, several custom extensions were developed for the end users. One such extension is based on a custom implementation of jupyter-server-proxy[3] to provide access to the web servers that are running on Jean Zay to the end users via Jupyter-Lab in a secure way. The lifecycle of these applications are managed by the JupyterLab and

users can access these applications by simply clicking on the launcher icons on the JupyterLab launcher. Some of the applications that are provided are Tensorboard, MLFlow, VSCode, etc. As Jupyter Notebooks are the de-facto standards for the ML/AI workloads, Jean Zay platform provides quite afew pre-built conda environments for the end users. In order to use kernels in these environments, a custom kernel_spec_manager_class has been developed. The kernel spec manager class is capable of adding kernels available both on Jean Zay platform and user's conda environments to the JupyterLab instances in the realtime dynamically.

Finally, certain future directions will be presented. Some of these are using bubblewrap containers to launch Jupyter servers in a network isolated containers, using singularity images for users to customize their JupyterLab instances, etc.

Ref:

[1] Project Jupyter (2023). JupyterHub Wrap Spawner, [Online]. Available: https://github.com/jupyterhub/wrapspawner

[2] Project Jupyter (2023). JupyterHub Documentation, [Online]. Available: https://jupyterhub.readthedocs.io/en/stable/howto/configuration/config-…

[3] Project Jupyter (2023). Jupyter Server Proxy, [Online]. Available: https://github.com/jupyterhub/jupyter-server-proxy