More than 20 million students are looking to improve their lives through learning, and depend on our ability to help them find the right course. With such a large collection to choose from, search is the easiest & preferred way to find what they’re looking for.

Role & Team

For 1.5 years, I lead the design of the Search experience.

My responsibilities included user research, concept ideation, aligning key stakeholders on product goals, designing user flows, visual design, prototyping, and collaborating with broader Marketplace Discovery teams.

I worked alongside a Product Manager and 5 engineers. Our extended team also included a researcher and data scientist.

Search lives within the Marketplace team; other teams include: Browse, Course Page, Reviews, Mobile, and Tagging.

Search When I Joined

When I first joined the Search team, it was an engineering only team working on ranking and algorithm problems.There was no product / user-facing strategy, and feature ideas were solution-based and difficult to evaluate.

I wanted us to align as a team on how we evaluate and ideate on new features. To do that, we needed research support to understand what our users needed.

Old Search Page

Foundational Research

Who are our researchers?

The Explorer searches with multiple interests in mind, uncommitted to any

The Beginner searches with a vague idea of what they need to learn and limited understanding of how to decide on a course.

Mainly with a career-related goal in mind

Almost ~90% of searches were topics (i.e. photoshop, python)

*

The old search page assumed that as long as we found course matches, students would pick the right one for them.

In reality, most students didn’t even know what they need to learn.

*

Synthesized User Problems

1. Students see irrelevant courses for their goal and assumes Udemy has no courses that can help them

2. Students don’t have enough information to understand if a topic or course will help them achieve their goal

3. Students worry that there is a better course on the marketplace despite finding a good course (FOMO)

4. We have no results that match what they searched for

Mapping out user problems to the user flow

A New Design Approach

Based on the main problems, we formed 4 areas of work:

1. Bring students to relevant results

2. Help students understand their results

3. Bring focus to recommended courses

4. Reduce dead ends

Mapping Out The System

Our old search system was so simplistic, it didn’t reflect the many states that a searcher could be in.

The new search page needed to support both existing features, and potential features we haven't designed yet.

Defining Features & Design Explorations

Countless team brainstorms and prototypes later, we aligned on key features in each bucket of work:

1. Course-Based Filters, Related Searches
to bring students to relevant results

2. Course Quickview, Topic Overview
to help students understand their results

3. Recommendations, Badges
to bring focus to recommended courses

4. Include Lecture Results, No Results Page
to reduce dead ends

Course-Based Filters

One of the main reasons that students weren’t seeing relevant courses is because they searched a broad topic that can be taught for different reasons. For example, students could search the topic “photoshop” for photo editing, graphic design, ux design, mobile design or marketing.

To surface more relevant results, we wanted to enable our students to specify the path they’re interested in.

While I was excited about a couple of directions, in practice, Course Topic Filters was the most promising because they were systematically collected, and had coverage on almost every course in the marketplace.

Course Quickview

Our course descriptions don’t always help students understand if a course can help them achieve their goals. From survey responses, we knew students needed additional pieces of information to properly evaluate a course's helpfulness.

We ultimately decided to go with an option that was fair to all courses, felt lightweight, didn't require an extra step, and was easy to compare between multiple courses.

Lecture Results

One common reason students reach “dead-ends” is because they searched for a niche topic that our search couldn’t find an entire course on. Those topics may be mentioned within a course, but we were only scanning course titles & subtitles to find results. We wanted to see if searching within courses would help reduce dead-ends.

From user-testing, we learned that full course matches were still more compelling. But in cases where there weren’t many full course matches, the lecture matches are also compelling. Thus, we enabled lecture matches only on queries that had less than 12 results.

Layouts that support more features

With all these newly planned features, we also had to start thinking about a new layout that can support it all. The current layout was too simplistic and restricted how the features could work together. The trickiest part with changing the layout was balancing how prominent a feature should be with how often it’s used, how helpful it is to the user, and the conversion impact.

V1. Filters on the Side

V2. Expose Dynamic Filters

User Testing Learnings

• Top area drew a lot of attention and testers were naturally curious and clicked into those filters
• Most of the students who saw dynamic filters were excited to see it 
• Students seem to have developed a blindness to the left-hand side
• Most of our students are beginners who don’t have very strict criteria for what they need so a lot of the filters seemed like a nice-to-have
  
• Some students struggled to understand what the topic is and needed to do research off Udemy

Armed with these insights, I iterated on the 2nd direction "Expose Dynamic Filters" and took designs to high fidelity. During this time we also went through a brand redesign.

Final Designs

Given the broader marketplace strategy for developing higher quality metadata and tags, a more flexible and dynamic filter system seemed like a better bet. During this time, we also went through a brand refresh that gave us a more robust colour palette to work with. 

Final Results

We rolled out the new layout in 3 parts to be able to track more accurately what was working and what wasn’t. After many iterations, these are the final positive RPU numbers on each element. Everything else was neutral.

2.11%+

Filter Bar

2.63%+

Course Cards

1.92%+

Recommendation Unit

(c) Beatrice Law 2018