Jupyter on Frontier

Frontier hosts a JupyterHub server to provide users with an interface for using JupyterLab and Jupyter Notebooks. Jupyter Notebooks allow users to combine live code, visualizations, interactive controls, and textual descriptions in a single, shareable document. Currently, Jupyter on Frontier supports a Python 3.11 kernel, but if additional kernels are needed, please contact the OLCF User Assistance Center at help@olcf.ornl.gov or see the instructions for creating and exposing a custom environment in Jupyter Visibility.

Note

If you are a moderate user accustomed to using Jupyter on Marble then using Jupyter on Frontier will feel familiar to you. The primary difference will be how to request a Jupyter session and you will now use Frontier compute resources instead of Marble.

Access

All OLCF moderate accounts should have access to Jupyter on Frontier at https://jupyter.frontier.olcf.ornl.gov.

After navigating to the JupyterHub URL, you will be prompted to log in using your OLCF account name and PASSCODE on the following page:

../_images/frontier_login.png

After successfully logging in and starting your Jupyter server, you will be presented with account parameters to launch your Jupyter session:

../_images/frontier_server_option.png

When requesting a Jupyter session, it is recommended to use the jupyter QOS since this will launch the session in the readily available service partition on Frontier. The jupyter QOS will allocate 1 node and 1 hour walltime for your session and any values listed in the “Walltime” or “Node Count” specifications will be ignored. If your session requires a different resource configuration, please consider using the debug or normal QOS which will launch your session in Frontier’s batch partition. For resource limitations and queue policy for Jupyter sessions in the batch partition, please see Frontier’s Scheduling Policy.

Warning

If your Jupyter session is not launched in 5 minutes, it will timeout and you will need to re-submit your request.

Launching a Notebook

Once a Jupyter session is running, you will be placed in your Frontier home directory. From here, you can launch a new notebook by clicking the Python 3 icon under the Notebook section of the Launcher tab. The default Jupyter kernel uses a Python 3.11 kernel, or you can launch a custom kernel if you have created and exposed one as described in Jupyter Visibility.

../_images/frontier_launch_page.png

Tip

You can only navigate directories realitve to /ccs/home/user/ using File -> Open from Path…. If you would like to access files in other storage areas (e.g., /lustre/orion/), you will need to create a symbolic link to those directories first.

Closing a Notebook

Please be mindful that the service partition used by the jupyter QOS has a 64 node limit and is shared among other users utilizing the JupyterHub server, users running small debugging jobs, and users that require other on-demand services. If you are not actively using your JupyterHub resources please consider stopping your server by navigating to File -> Hub Control Panel -> Stop My Server.

Warning

Logging out, closing the Jupyter tab in your browser, disconnecting your remote VSCode instance, etc. is not sufficient for ending your JupyterHub server and relinquishing the allocated resources. You must manually stop your server to relinquish the resources.

API Token

Users have the option to interact with their Jupyter session programmatically via the JupyterHub REST API. Tokens can be generated by navigating to File -> Hub Control Panel -> Token or by visiting the Token tab before launching a Jupyter server. Once on the Token page, you can specify the token parameters and click the Request new API Token button to generate a new token.

../_images/frontier_api_token.png

Additionally, any currently active tokens can be viewed and managed in the API Tokens and Authorized Applications subsections on the Token page.

VSCode Example

If you would like to run your Jupyter Notebook from a local instance of VSCode, you may do so following these steps:

  1. Generate and store your API token from the Token tab.

  2. Start a server from https://jupyter.frontier.olcf.ornl.gov

  3. Open VSCode with the Jupyter & JupyterHub extensions.

  4. Open the VSCode Command Pallet (Ctrl+Shift+P or cmd+shift+p).

  5. Run the command for “Create New Jupyter Notebook”.

  6. Using the Kernel Selector in the top right or running the “Notebook: Select Notebook Kernel” command, navigate to “Select Kernel” -> “Existing JupyterHub Server” -> “Enter the URL…”

  7. Enter https://jupyter.frontier.olcf.ornl.gov, you’ll be prompted for a username and a token.

  8. Select kernel. Python 3 (ipykernel) should be readily avaialble. Please see Example PyTorch Jupyter Notebook for how to create a custom kernel.

  9. Run from VSCode!

Example PyTorch Jupyter Notebook

To enable a Jupyter notebook to use a pre-existing PyTorch environment, it should be as simple as installing and configuring the ipykernel package in your conda environment and then selecting that kernel in your notebook.

module load miniforge3
source activate <your_env_name>
pip install ipykernel
python -m ipykernel install --user --name <your_env_name> --display-name <your_env_name>

The following will walk you through creating a simple PyTorch environment on Frontier and making the kernel visible to Jupyter notebooks. For more details on PyTorch including important performance considerations, please see PyTorch on Frontier.

From a Frontier login node, load the following modules:

module load PrgEnv-gnu/8.6.0
module load miniforge3/23.11.0-0
module load rocm/6.4.1
module load craype-accel-amd-gfx90a

Next, create a new conda environment with the ipykernel, then install PyTorch with ROCm support:

conda create -n pytorch_env python=3.12 ipykernel -c conda-forge
source activate pytorch_env
pip install numpy torch==2.8.0 --index-url https://download.pytorch.org/whl/rocm6.4

Finally, make the kernel visible to Jupyter notebooks:

python -m ipykernel install --user --name pytorch_env --display-name "PyTorch (ROCm)"

If successful, you should see the kernel on the JupyterHub launcher page:

../_images/frontier_launch_pytorch.png

Additionally, you can verify that Jupyter notebook is seeing all GPUs on a Frontier compute node by running the following code in a notebook cell:

../_images/frontier_torch_check.png

PyTorch Code Blocks
import torch

torch.__version__

if torch.cuda.is_available():
   num_gpus=torch.cuda.device_count()
   print(f"PyTorch detected {num_gpus} visible ROCm devices.")
   for i in range(num_gpus):
      print(f"Device {i}: {torch.cuda.get_device_name(i)}")
else:
   print("ROCm is not available.")

Note

If you notice old or non-functioning kerenls, they could be residual from Jupyter on Marble or insufficient clean up after removing a conda environment (e.g., conda env remove –name pytorch_env). To remove these from Jupyter on Frontier, you may need to manually delete it’s associated directory in /ccs/home/{userid}/.local/share/jupyter/kernels.