djm-software-llc

What is JupyterLab?

JupyterLab is an open-source, web browser based application that supports a wide range of workflows in data science, scientific computing, and machine learning. The project was created to allow data scientists easily share their findings, code, and models with their community.

JupyterLab is flexible. It allows users to configure and arrange their interface as they see fit. In addition, it is fairly extensible and modular: users can write plugins that add new components and integrate with existing ones.

Visit Jupyter

What's the deal with the file system?

“It’s not an obvious user experience. Our users are generally pretty highly technical people . . . . That’s actually why we need you, because we’re trying to make this better for more people and so figuring out usage patterns that make it a little simpler.” –Ganymede

Through our early interviews with the Jupyter team to understand the system and problem at play, we learned early on that it's difficult for the user to orient themselves in the existing file browser. This is very problematic and it quickly became clear as to why the client approached us with this issue. In order for a user to successfully use JupyterLab, at the very minimum they should be able to understand where they are and where their files are. In addition to poor way-finding, the existing file browser suffers from built-in, but invisible “safety features” that prevent users from accessing locations outside the directory JupyterLab was launched in, and untimely feedback that has eroded users’ trust in existing features.

Our goal was to design a file system that supports the base actions of locating and organizing files, while implementing some of the JupyterLab community’s requested features.

The Methods

Competitive Analysis

Our next effort was a competitive analysis of JupyterLab versus its key competitors. We analyzed nine direct and indirect competitors that we felt occupied a similar space as JupyterLab. Our goal was to understand what JupyterLab could be doing well or poorly and where our competitors excelled otherwise.

Heuristic Evaluation

We began our research phase with a heuristic analysis of JupyterLab. Given the highly-technical nature of JupyterLab as discussed above, this was a lot harder than we had anticipated.

Interviews and Surveys

JupyterLab's open-source community and user base are dedicated to the project and its purpose. They have invested significant time into this software. We were guided early on that anything we design for this community will need to gain their stamp of approval. In turn, we spent a significant amount of time understanding the user base, their feelings, and their concerns. Our conversations and surveys provided great context for the personas and journey maps that would guide our concept ideation.

Concept Ideation

Our design lead created a dedicated design prompt of that which we lived by. The design prompt's specific criteria guided our preliminary concepts in the form of sketches. Out of those concepts, we produced two very different design directions, which we would pressure test with stakeholders and the community. These concepts were posted to the JupyterLab GitHub regularly for community input.

Final Prototype

Per the feedback gained from user testing, we put together a working final prototype in Figma. We've handed off the design to the JupyterLab team and the rest is in their hands.

User Testing

Due to the short-term nature of the project, we ran our testing and design tracks simultaneously, incorporating testing insights as we received them to optimize our time. Our testing team conducted remote, unmoderated tests with a group of volunteers from the community that included a first-click test, and 2 usability tests.

Phase One:

Understand the Problem and Software

Heuristic Evaluation

Our heuristic evaluation was conducted using the Nielson Norman Group’s 10 Heuristics. We evaluated only actions that were related to the file browser, noting the heuristic violated, and the severity rating of the violation. We offered our recommendations to remedy these violations in accordance with the 10 Usability Heuristics.

See on github

Competitve Analysis

In our competitive analysis, we split our competitors into two categories: direct and indirect. Those we categorized as direct competitors were mostly IDE’s with similar code-crunching capabilities, while indirect competitors were evaluated primarily on the way they organized and handled files.

See on github

User Interviews and Surveys

Our first real contact with the JL user base was through an introductory survey. We wanted to better understand how people use the file browser as it is, what they disliked about it, and what they wanted to see next. We quickly discovered that the vast majority of those surveyed were developers or data scientists who considered themselves of mid- to expert-level proficiency.

We interviewed 10 JupyterLab users of varying expertise levels and occupations. They ranged from key stakeholders to students, and revealed three feature-level challenges:

Lack of functionality: Feature does not exist.
‍Lack of discoverability: Feature exists but users do not know it does.
‍Lack of feedback: Feature exists but users do not trust it.

From these interviews, the key quotes, pain points, and touch points would be used to craft personas and journey maps, and we were also able to extract a rudimentary feature request chart.

Personas and Journey Mapping

Our research until this point in the project had revealed that JupyterLab’s user base is diverse in both occupation and expertise. We identified 4 key personas who made up the vast majority of users: the data scientist (expert), the developer (expert), the intern (intermediate), and the student (novice).

Each user’s journey was mapped from spinning JupyterLab up, till the end of their work flow. While the journeys were highly variable, every persona had 2 common pain points: finding files, and organizing them once found. We would center our design prompt and requirements around these two points once we began the problem solving phase of the project.

See the personas

Problem Setting Summary

JupyterLab’s current file browser suffers primarily from a mismatch between the user’s mental model of a file browser, and how the file browser actually works. Currently, the file browser does not differentiate between “remote” and “local” in the traditional sense. Instead, JupyterLab understand files only in relation to the “kernel” running JupyterLab, which presents itself, fundamentally, as a problem of “ceilings and bridges”:

"Ceilings"

The user is trapped in their working directory with no visibility to their outer file system. There is no indication that an outer file system even exists.

"Bridges"

There is a lacking connection to external tools (Google Drive, Github, etc) that would allow users to copy files into their "local" space.

Phase Two:

Problem Solving

Design Prompt

Design a file browser UI for JupyterLab that is recognizable, easily navigable, and offers the user transparency into where their files are located. The design should primarily support location and organization of one’s files. Seamless connectivity to other sources and data, and collaboration with peers and the community should also be considered, but are secondary to the location and organization of files.

Where am I: A file’s location should be readily apparent.
‍
Just in time: Visual cues should occur as needed, and exactly when an action occurs.

Design Criteria

The right place: Coherent information architecture optimized around one-click actions, nested actions, right-click actions.

‍The right tools: Users should be given basic file management features and tools.

See The strategy doc

Concept Ideation

Concept 1: "Mac Finder Lite"

In our competitive analysis, we noted that advanced file browsers tend to have a larger working space, while JupyterLab’s remains tethered to a thin sidebar. We began thinking about how a full-sized file browser might look in JupyterLab, and came up with a very Mac Finder inspired design. This concept would live in the JupyterLab workspace, and would allow users to apply “orbit tags” to files to informally group them.

After initial sentiment testing, we found that, while novice users found this concept very interesting, more experienced users struggled to incorporate this design into their existing mental models. However, there is potential for this concept to be developed as an extension of the sidebar file browser, therefore we have handed off our supporting research and artifacts to the JupyterLab team for future development.

Concept 2: Launcher Add-On

Our second concept leaves the file browser in the sidebar, but adds a Launcher-type extension to the UI that would act as a manager for a new feature called “Spaces”.

We repeatedly heard from our interviewees that they could be working with a dozen data files in a single notebook, and would appreciate a way to easily access all the files they need in a project. Workspaces were a concept that already existed in JupyterLab, but there was no visual way to manage the files you placed in a workspace. Remote connections shared a similar problem. The implementation of “spaces” would give users a way to interface with the files they placed in workspaces while organizing projects for easy access and handoff.

Initial sentiment tests revealed that this concept was much more in line with the advanced users’ mental model, and would be less developmentally taxing to implement, so we chose to proceed with this concept for this project.