Apple Podcasts is currently the leader in podcasts and Spotify wants to change this. So much in fact, that in February 2019, Spotify invested a whopping $500 million on podcasts, both on original content and acquisitions.
There is no doubt that podcasts are taking the spoken word entertainment industry by storm. 165 million people in the US have listened to a podcast.32% of Americans (approximately 150M) are monthly podcast listeners. In 2020, podcasts are expected to generate over $650 million.
Now we ask ourselves, based on this sizable investment, is Spotify optimally designed for the ultimate podcast experience? Spotify is already putting their investment to work.
In June 2019, Spotify redesigned personal content libraries for Premium Users. It focused heavily on separating musical and podcast content, something that hadn’t existed before.
With user testing, we can determine if users are able to navigate Spotify’s Podcasts content and if this ongoing investment is paying off.
We conducted a closed card sort using OptimalSort and 20 users participated. A “Closed Card Sort” asks participants to sort topics from content within Spotify into predefined categories. It is especially good for evaluating the existing information architecture to determine if it is already optimally designed.
Four categories, Podcasts, Discover, Podcast Player, and Social, were defined for users to sort topics into. Topics were possible user actions within Spotify’s current format. A few topics used in the Card Sort were "Add podcast to library", "Rewind by 15 seconds", and "Unfollow a podcast". Overall, a total of 29 topics were sorted into the four categories.
Remote user testing was conducted with the online tools Loop11 and UserTesting.com. User testing allows us to gain human insight into how user-centric a product’s experience is by asking users to complete a series of tasks.
Loop11 provided us with qualitative data. We defined a series of tasks with a successful end page specified. If users reach this page at the end of the task, Loop11 marks it as successful. Resulting data is how many tasks were completed successfully, how many tasks users failed, how long it took and how many clicks it took for a user to complete a task.
UserTesting.com provided us with quantitative data. We once again defined a series of tasks for users to complete. Using recording capabilities, we wanted to see and listen to how users completed tasks, if they were successful, and what their thought process and opinions were as they did them.
Our results were surprising. I thought without seeing Spotify’s UI, users would be confused about sorting tasks into the keyword categories. But rather, users were able to sort the topics accordingly.
Each topic was sorted into the intended category by at least 70% of testing participants. The remaining results were similar and not very scattered. So far, people seem to understand Spotify's existing architecture, although it is important to keep in mind that a good majority of testing participants have used Spotify before. We did see some confusion with the task "Add episode to queue", it was sorted either into Podcasts or Podcast Player.
With the given tasks for Loop11, we’re able to see that most users were able to complete the tasks, at a 60-70% completion rate. The exception is when users were asked to find Educational Podcasts. Most users failed, likely signaling that the current layout of podcast categories is confusing. Or at least finding them is.
Loop11 gave us some metrics that showed us the mental latency users encounter when doing tasks that are meant to be simple, such as viewing their podcasts. It took over 10.5 page views on average for users to view their podcast library. The metrics also showed us that users had a tendency to utilize the search bar to complete their given tasks.
Our Loop11 test showed us that most people are able to accomplish their goals in Spotify’s current design for podcasts, but given the longer click paths and reliance on their search bar, there is room for improvement.
UserTesting.com offered eye opening results about Spotify Podcasts current usability state. In this testing iteration, all three of our participants were actually new users to Spotify. This allowed us to gain more insight into where Spotify’s Podcast design lacks. Users really struggled with adding a Podcast to their library. Once again, we saw that the search bar was heavily utilized in completing their tasks. Otherwise, sharing and finding podcasts were easy, as well as adjusting playback. This is likely because the design for these tasks were similar and consistent to other entertainment applications.
Users struggled greatly with “Adding a Podcast” to their library and all essentially abandoned the task. I would like to point out that this exact task provided different results in testing with Loop11. Technically, 60% of users were able to complete this task in the Loop11 test, while all three participants on UserTesting.com failed. In Loop 11 we did not have screen/audio recording, we could only see that users successfully reached the specified success URL, which was “viewing their library”. Therefore, we are unsure if users were actually able to add a Podcast to their library in our Loop11 testing.
Users got creative when trying to complete this task, they did things such as: Create playlist and add an episode, Download episode to see if it appears in Library, View menu within Podcast show page to find an “Add” button.
There was a great deal of frustration with the lack of intuitive design. The “Follow” button was confusing for users and all said they were looking for something that said “Add”. They felt that was more intuitive to them.
Otherwise, we once again saw the search bar was used to assist users with completing their tasks. They did not struggle with sharing, discovering, and adjusting playback for their podcasts. One user said of these remaining tasks, “Overall this was very easy to me” and he thought it was “familiar”.
Overall, from UserTesting.com, we gathered that people struggled with more detailed tasks and we were able to learn a lot from seeing their screen and hearing their thoughts as they work through scenarios. Their frustration was building when they felt they couldn’t complete simple things and the search bar was a fall back for them when Spotify’s built-in navigation was uneasy for them to use.
With our testing, we realized more separation between Music and Podcasts still needs to happen in Spotify. Click paths and scrolling can be minimized. Make Podcast categories more visible. Users will be able to more easily discover new shows based on categories if the current layout is redesigned. Changes to search can help as well.
The final report of my user evaluation is linked below. There are way more details there, interesting data, and some decent re-design recommendations. Take a look!
Users utilize the search bar to get to where they need quickly in Spotify, capitalize on that. Make it available in more places that make sense. A search bar is already implemented within a musical playlist; mirror that and add it to Podcast show pages so users can look up episodes easily within that podcast.