jupyter notebooks
This guide will explore the process of setting up and using Jupyter notebooks remotely on Dartmouth's Discovery computing cluster, for example to create visualizations for the entity code. Whether you are analyzing simulation data, creating plots, or debugging code, remote Jupyter notebooks provide a flexible and interactive environment for your research needs.
Jupyter Notebboks on Dartmouth's Discovery Cluster¶
Dartmouth research computing documents how to remotely access Jupyter notebooks. In the following, we will review the key steps. In your Discovery shell, activate conda:
To avoid this command for each Jupyter instance, you can add it to your ~/.bashrc file on Discovery. This environment is set up each time a new shell logs in to the cluster. The conda environment can be customized as follows:
If you have access to the EPaCO lab space and would like to access environments with pre-loaded packages for post-processing of entity data, you can activate:
First time users should create a new conda environment for using with Jupyter notebooks:
Returning users can activate the custom environment by typing
In this environment, you can install custom python packages that will be available for the kernels of remotely accessed notebooks.
Research with entity
To install the visualization package of the entity toolkit, you can run:
Jupyter Notebooks via Dartmouth OnDemand¶
Dartmouth OnDemand provides seamless, browser-based access to Jupyter Notebooks on Discovery, eliminating the need for Jupyter Notebooks via Shell and SSH Tunnels. You can select the Jupyter Notebook sessions and launch it as follows:
- Account: Select
free. - Partition: Choose
standard. - Number of hours: Set to
1(or longer if needed). - Num Cores: Enter
8(as needed). - Node type: Select
anyfor fastest start time.
First Time Users
First time users need to make the conda environment available to the Jupyter sessions. In a discovery shell, run:
Once your session has started, you can connect to it and use Jupyter directly on the cluster, including seamless access to your data. Check out the Dartmouth OnDemand documentation for further information.
Jupyter Notebooks via Shell and SSH Tunnels¶
Dartmouth Discovery Cluster
To avoid running computations on login nodes and slowing down the system for other users, launch your Jupyter notbook in an interactive session. For example, you can use the following command to request four CPU cores with asssiated memory:
To launch a Jupyter notebook, execute the following in your Discovery shell:
Once the notebook starts, you will receive information about the IP address and port used to connect to the notebook (e.g., of the form 127.0.0.1:888x), as well as a token that will be used for secured access.
In a new shell on your local machine, establish a new connection to the discovery cluster with additional port forwarding (connecting directly to the Jupyter notebook), for example like:
Be sure to update the IP address and port number as generated in the output after launching your notebook. Once the connection is established, you can open the Jupyter notebook in your web browser (e.g., Chrome) by navigating to the following web address:
You can now use a Jupyter notebook running on ths discovery cluster from your web browser. With direct access to all the data stored in your lab space.
Windows Users
When attempting to establish an SSH tunnel for accessing a Jupyter Notebook on a remote cluster, Windows users may encounter a "permission denied" error related to port usage. This can often be caused by firewall or antivirus software blocking the connection. If issues persist, check your firewall settings and explicitly allow incoming and outgoing connections for the chosen local port (e.g., 8888 or 4242). Configure rules for allowing port usage and confirm no other application is using the port. In some cases, switching to a less commonly used port (e.g., 4242) may resolve conflicts with background services or security software.