Setting up a remote jupyter notebook for a website

In-depth tutorial for creating a Jupyter lab accessible from your domain

Introduction

I recently wanted to set up a Jupyter notebook server on my home desktop and serve it from my website. This would allow me to edit Python scripts from my laptop and run them on my beefier home hardware remotely. Setting this up was unfortunately not as straight forward as I would’ve hoped, so I’m outlining this process to make things easier for others getting started.

Before moving forward, identify the machine you’d like your notebook server to run on. For me, this was my home desktop where I have otherwise unused compute power sitting around. Perhaps instead you’d like to set up the notebook server on a web server that you already rent to host your website. Either way, I bring this up because the setup for a home computer is slightly different from that of a server already accessible from the web, so keep that in mind as you follow along.

A Brief Overview

This tutorial will consist of the following steps:

  1. Install Python, Jupyter, and Apache
  2. Set up a subdomain that points to your server’s IP
  3. Configure the Jupyter notebook server to be served securely over the web
  4. Configure Apache to handle incoming connections to the subdomain and reverse proxy the Jupyter notebook server
  5. (Optionally) Create a script to automatically start the Jupyter notebook server when your machine boots up.

Let’s begin!

Installation

On the target machine, you will first need to install Python, Jupyter, and Apache. For this tutorial I will assume use of Ubuntu 16.04, but most steps should be the same on any Linux distribution and for macOS.

Jupyter

Ubuntu 16.04 ships with Python3, so you should be good to go on that front. Before installing Jupyter, I recommend setting up a Python venv to keep everything organized and in one place, including additional Python packages you may want to install in the future. Here we’ll set up a venv named jupyterenv in the home directory and activate it:

cd python3 -m venv jupyterenv
source jupyterenv/bin/activate

You can now install Jupyter by typing

pip install jupyter

We’ll do some more configuration for our Jupyter server later in the tutorial.

Apache

We’ll now go ahead and install Apache. Type the following into your command line:

sudo apt-get update
sudo apt-get install apache2

We will configure Apache more on down the line. For now, double check Apache was installed and started correctly by visiting your server’s IP in the browser. You should see a default Apache2 Ubuntu page if everything is working properly. If you don’t see this page, there are a number of issues that you could be running into; I recommend troubleshooting your installation with this article.

Note: If you are setting this up on your home computer, you won’t have an IP to visit just yet. You should, however, be able to visit localhost in your browser and see the same page described above. We’ll address your machine’s accessibility over the web in the next step.

DNS Configuration

If you haven’t done so already, you’ll now need to set up a subdomain as an endpoint for your notebook server. If you’ve done this or wish to use your root domain name, feel free to skip this step.

To set up a subdomain, visit your DNS provider. Create a new A record, type in your desired subdomain sub.domain.com and point it your server’s IP. This IP will depend on what type of machine you’re setting up your server on:

For a configured web server

If you’re already using a web server to host your main site, you likely already know your server IP. Point the subdomain record to that IP and you should be good to go.

For a computer at home (on LAN)

If you want to set up your notebook server on your home computer, you’ll need a way of accessing your computer over the internet. By default, home routers will ignore attempted external connections to your home network’s public IP. However, with router port forwarding, you can configure your router to forward HTTP connections to a computer on your LAN. Most ISP’s make this easy through your router’s local interface; search the web for your router and ISP to find the port forwarding process particular to you. If you’d like more details on how port forwarding works and how to set it up on your home network, I highly recommend reading this article.

If you know your router’s default gateway and are in a rush, it’s worth browsing around that interface for port forwarding options. You’ll want your router to forward incoming connections on port 80 to your target machine’s port 80.

Jupyter Configuration

We are now going to modify the Jupyter configuration file. If you just installed Jupyter or haven’t configured your Jupyter installation before, this file won’t exist yet. Regardless, it’s worth checking if you have a config file already created at ~/.jupyter/jupyter_notebook_config.py. If that file doesn’t exist, run the following in your terminal:

jupyter notebook --generate-config

This will create a config file at the location mentioned above. There are a few things we’ll want to change here. Open it with your favorite editor:

vim ~/.jupyter/jupyter_notebook_config.py

You’ll notice there are a lot of settings that are commented out. Delete everything in the file and add the following lines:

c.NotebookApp.notebook_dir = '/path/to/notebooks/'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 5000

Having the file contain only the settings that are actually in use makes it easy to maintain. Be sure to replace the notebook directory above with your actual path. It’s worth noting that you don’t have to set the port here, but it’s good security practice to change it to something other than the default port 8888.

Because this notebook server will be accessible over the web, you’ll want to set up a password to keep your server secure. Otherwise, anyone will have free reign over your machine. You can set this up by doing the following:

jupyter notebook password
Enter password:  ****
Verify password: ****
[NotebookPasswordApp] Wrote hashed password to ~/.jupyter/jupyter_notebook_config.json

That takes care of the Jupyter setup! We’ve outlined a fairly minimal but secure setup procedure. Check the Jupyter docs for more details and extensive configuration options, as well as best Jupyter server security measures.

Apache Configuration

It is now time to configure the web server to handle requests to your subdomain and point them to your notebook server. We can do this easily by letting Apache reverse proxy our Jupyter notebook server. Execute the following to move to your Apache sites directory and begin editing your default virtualhosts file (or your config file of choice):

cd /etc/apache2/sites-available/
sudo vim 000-default.conf

Add the following virtualhost config to the file:

<VirtualHost *:80>
    ServerName sub.domain.com
    ProxyPreserveHost On

    ProxyPass /api/kernels/ ws://localhost:5000/api/kernels/
    ProxyPassReverse /api/kernels/ http://localhost:5000/api/kernels/

    ProxyPass / http://localhost:5000/
    ProxyPassReverse / http://localhost:5000/
</VirtualHost>

Be sure to change the port from 5000 to the port you specified in your Jupyter config file. If you did not specify a port, the Jupyter server defaults to port 8888. Also don’t forget to replace sub.domain.com with your actual subdomain.

We now need to enable a few Apache modules to allow proxy functionality. Type the following in your terminal:

sudo a2enmod proxy sudo a2enmod proxy_http sudo a2enmod proxy_wstunnel

We can now restart Apache using

sudo service apache2 restart

Now Apache should be good to go. Any connections to sub.domain.com will be heard by your server and Apache will appropriately reverse proxy your notebook server, allowing your notebooks to be seen over the web.

Start Jupyter Server

Once you’ve completed all the previous steps, you should now be ready to start your Jupyter notebook and visit it over the web! Type the following into your terminal to start the Jupyter server in the background:

jupyter notebook &

The trailing & is what keeps the process running in the background. If you are sshed into the machine you’re starting the Jupyter notebook on, you can disconnect without worrying about the notebook server shutting down. Now visit sub.domain.com and enjoy your Jupyter notebooks!

Start Jupyter Server on Boot (Optional)

If you’re like me and configured the notebook server on your home computer, you likely shut your computer down from time to time. Because it’s rather inconvenient to go in and start up your Jupyter notebook server every time your machine boots up, we’ll automate the process by creating a script as outlined below.

We’ll start by creating a bash script that activates our Python venv and starts the Jupyter notebook server. Here we’ll create the script in the home directory:

vim ~/jupyter.sh

Now that you’re editing the script, add the following to the file:

#!/bin/bash

. /home/<USER>/<jupyterenv>/bin/activate jupyter notebook
--config="/home/<USER>/.jupyter/jupyter_notebook_config.py" &

where <USER> is your user’s name and <jupyterenv> is the name of your Python venv. If you didn’t create the Python venv in the home directory at the beginning of the tutorial, be sure to replace the path with your venv’s actual location.

You now have a jupyter.sh script that activates your Python venv and starts up your Jupyter server. You can go ahead and run this file using source jupyter.sh to ensure everything is working properly.

We’ll now create an rc.local script to execute jupyter.sh under the correct user on boot. Begin editing your rc.local file:

sudo vim /etc/rc.local

Add the following lines to the end of the file:

su -c ". /home/<USER>/jupyter.sh" -s /bin/sh <USER> exit 0

replacing <USER> with your actual username. This will allow rc.local to run your Jupyter script as the specified user when your machine boots up.

Conclusion

Thanks for reading! Trying to set up a remote Jupyter server without a clear resource and little networking knowledge proved difficult for me. I hope this tutorial helped those of you who find yourselves in a similar boat. Feel free to reach out with questions or comments, and share this article if you found it helpful! Cheers