Notice
Harnessing the power of Jupyter{Hub,Lab} to make Jean Zay HPC resources more accessible
- document 1 document 2 document 3
- niveau 1 niveau 2 niveau 3
Descriptif
Spawning jupyter servers is at the heart of the JupyterHub. This talk will present the details of the custom spawner that has been implemented on the top of WrapSpawner[1]. This spawner allows the users to launch Jupyter servers either on login nodes or compute nodes of Jean Zay. JupyterHub needs privilege escalation to launch Jupyter servers as users and conventionally, this is done via sudo[2]. An alternative approach using SSH certificates will be demonstrated and how it is integrated with the custom spawner. Thanks to such approach, the entire JupyterHub deployment runs in the userland without any extra privileges. Moreover, hardening approaches on the nginx (used as reverse proxy) will be presented which can further reduce the attack surface.
Given that JupyterLab is highly extensible, several custom extensions were developed for the end users. One such extension is based on a custom implementation of jupyter-server-proxy[3] to provide access to the web servers that are running on Jean Zay to the end users via Jupyter-Lab in a secure way. The lifecycle of these applications are managed by the JupyterLab and
users can access these applications by simply clicking on the launcher icons on the JupyterLab launcher. Some of the applications that are provided are Tensorboard, MLFlow, VSCode, etc. As Jupyter Notebooks are the de-facto standards for the ML/AI workloads, Jean Zay platform provides quite afew pre-built conda environments for the end users. In order to use kernels in these environments, a custom kernel_spec_manager_class has been developed. The kernel spec manager class is capable of adding kernels available both on Jean Zay platform and user's conda environments to the JupyterLab instances in the realtime dynamically.
Finally, certain future directions will be presented. Some of these are using bubblewrap containers to launch Jupyter servers in a network isolated containers, using singularity images for users to customize their JupyterLab instances, etc.
Ref:
[1] Project Jupyter (2023). JupyterHub Wrap Spawner, [Online]. Available: https://github.com/jupyterhub/wrapspawner
[2] Project Jupyter (2023). JupyterHub Documentation, [Online]. Available: https://jupyterhub.readthedocs.io/en/stable/howto/configuration/config-…
[3] Project Jupyter (2023). Jupyter Server Proxy, [Online]. Available: https://github.com/jupyterhub/jupyter-server-proxy
Thème
Documentation
Dans la même collection
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 7 : Conclusion
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 4 : Des interactions croisées, liens entre le calcul et les données.
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 1 : Introduction et présentation des intervenants
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 6 : L'accompagnement autour de la gestion des données
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 3 : les liens entre les structures
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 5 : Des interactions croisées, à propos des compétences
-
Table ronde et discussions : infrastructures de calcul et ateliers de la donnée de recherche Data G…
CastexStéphanieRenardArnaudAlbaretLucieRenonNicolasDufayardJean-FrançoisPARTIE 2 : Les ateliers de la données
-
Présentation du dashboard ReproVIP pour visualiser la reproductibilité dans l'imagerie médicale
BonnetAxelLa plateforme d'imagerie virtuelle VIP [1] (https://vip.creatis.insa-lyon.fr) est un portail web de simulation et d'analyse d'images médicales. Elle existe depuis plus de 10 ans et a évolué pour
-
Fast Polynomial Evaluation (présentation + demo)
VigneronFrançoisWe propose a new algorithm for quickly evaluating polynomials. The FPE algorithm pre-conditions a complex polynomial P of degree d in time O(d log d), with a low multiplicative constant independent
-
Présentation des performances paralléles du code QDD CUDA Fortran sur différentes Architectures GPU
PaipuriMahendraQDD est l'acronyme de Quantum Dissipative Dynamics, un ensemble de théories développées pour prendre en compte les corrélations dynamiques incohérentes dans les clusters et les molécules.
-
Using High Performance Computing to decipher the impact of ageing process on collagens
DepenveillerCamillePlant cells have to face environmental stress, and in this context, being able to decipher the organization of the plant plasma membrane is a key point to better understand this adaptability of plasma
-
Simulating the Extra Cellular Matrix - Calculations and data from atom to animal
BaudStéphanieThe extracellular matrix (ECM) is a three-dimensional network of macromolecules that is the architectural support for cells and allows tissue cohesion. This dynamic structure regulates many biological
Avec les mêmes intervenants et intervenantes
-
Présentation des performances paralléles du code QDD CUDA Fortran sur différentes Architectures GPU
PaipuriMahendraQDD est l'acronyme de Quantum Dissipative Dynamics, un ensemble de théories développées pour prendre en compte les corrélations dynamiques incohérentes dans les clusters et les molécules.