ENDEAVOR

Our team partnered with a Fortune 500 educational publishing and assessment services company on a 7-month project, aimed at informing features for the next iteration of their reading intervention software program. The goal is to improve student motivation, accountability and enabling a personalized learning experience through the software. We created Endeavor, an adventure-style interactive narrative game to help students become competent and confident in reading.
Date
Jan 2018 – Aug 2018
Team
Lu Yang, Zach Mineroff, Roger Strang, Julia Ridley, Chenxin Wang
Domain
Learning Experience Design; Web Design;  Mobile Design; Game Design

MY ROLE

As the sole designer of the team, I collaborated with project manager, researcher, content director, and developer to build Endeavor. I drove the end-to-end UX design process from research, ideation to implementation. I was responsible for building models, user flows, analyzing competitor products, creating wireframes and designing UI screens. I also designed our spring and summer reports. Please feel free to contact me if you are interested in seeing the reports.

DESIGN PROCESS

REVEALING AREAS FOR IMPROVEMENT

For beginning decoders, reading takes a lot of mental energy. Reading longer passages can be stressful and tiring, leading to demotivation. The reading intervention software supports struggling readers and follows a natural progression of instruction through the skills required to read full paragraphs. However, some students have to force themselves to pay attention to the software. Besides, most of these students are from low-income families, so learning from the software in class tends to be the only way to help them catch up. These students are at-risk for never learning to read because they are not enjoying what they are learning. To reveal areas for improvement of the software, we conducted contextual inquiries to understand how teachers and students are using the software in the classroom.

PERSONAS & LEARNING JOURNEY

Through affinity diagramming, we learned from the data collected in our interviews and observations. We created teacher and student personas based on data, and a learning journey to better understand students’ pain points for each of the key instructional elements in the software. 

UNCOVERING STUDENTS’ NEEDS

The followings are selected insights, tied to our areas of focus: engagement, personalization, and accountability.

INSIGHT 1

Students may lose confidence in their ability to complete certain activities when the task is clearly too difficult. 

I’m not cut out for the task.

A student like Sarah spent well over ten minutes on one trial of a word activity. Without additional scaffolds, she felt as if she would never get it right. A student like Daniel had been stuck on the same spelling module for over two weeks and expressed frustrations and disbelief in himself.

INSIGHT 2

Feedback in the form of direct instruction is less effective if it’s repeated several times after the same mistakes.

I hate spelling. Every time you get it wrong you have to listen to the person all over.

A student like Sarah kept spelling the words “tunnel”, “channel” and ”kennel” as “tunnle”, “channle”, “kennle.” Instead of providing feedback on the common error (flipping “el”), the software simply offered the same direct instruction after every mistake.

I’ve had kids like, “I’ve done this six times already.” And I’m like, “okay, but you’re not getting it. That’s why they’re asking you to do it again.

Teachers like Mrs. Grant would often remedy this issue by employing their own, personalized instructional techniques.

INSIGHT 3

Some students have significant trouble combining low-level skills in more complex tasks. Proper scaffolding is needed to bridge the gap between learning phonemes or word strategies and applying them in the context of a sentence/paragraph.

It's just too hard.

During a sentence-recording activity, a student like Sarah stopped reading at the first word she didn’t recognize. Instead of trying to apply a word strategy she learned earlier, she just stopped and put her head down. Another student like Daniel, experienced this kind of frustration so often that he tried to “glitch out” the software by furiously clicking on the record button.

INSIGHT 4

While students value being able to track their progress and see evidence of their own learning, they feel as if the progress tracking screens are irrelevant.

It just makes me think about how far I have to go and it’s too much.

Students like Sarah, Daniel, and Devon all seemed to have an automatic reflex to click the continue button as soon as a progress-tracking screen appeared, even if just a few minutes earlier they told us that they care about their scores and progress.
Teachers like Ms. Grant have created their own forms of progress tracking that their students respond well to.

IDEATION OF PRODUCT CONCEPT

DESIGNING A STORYTELLING GAME

We presented our pitch ideas to our client at the spring meeting. We evaluated the impact for users and feasibility of each idea with our client, and were able to prioritize features to build. We were all excited to design an adventure-style interactive narrative game for our struggling readers. Our goal through the game is to motivate students to practice skills they struggle with the most, enabling them to become competent and confident in reading long passages.

INTRODUCING: ENDEAVOR

Provide students with stories of various reading levels

By providing students with a choice of stories, Endeavor enables learners to engage with content that appeals to their interests. The available stories are based on the students’ current reading level, ensuring they will not get discouraged by overly-demanding text. 

Empower students with narrative choices 

Students make choices in the narrative, giving them a sense of control over their learning experience. A progress bar at the bottom lets students know how much they have read and how much longer they have to go.

Check students’ reading skills through embedded comprehension questions

Comprehension questions are embedded in the narrative, making them seem more like a game than a test of their reading skills. The choices that students make have consequences in the narrative, where incorrect answers lead to poorer outcomes. 

Assess students’ weak skills through embedded word challenges 

Students who answer a comprehension question incorrectly either do not understand the target word or they could not decode it. Embedded word challenges help make this distinction, and can provide teachers with data on how to personalize future lessons.

Pinpoint the exact confusion through easier word challenges

If a student fails an initial word challenge, we can hypothesize that it was due to a confusion between one or more sounds. To pinpoint the exact confusion, we can examine their answer choices on subsequent word challenges.

Present achievements and performance students care about

At the end of the activity, Endeavor shows students how well they did throughout the story. Metrics like stars, coins, and items encourage students to think about their choices next time.

With the companion mobile app, students can read and interact with stories anywhere and anytime. 

The Product Development Process

COMPETITIVE ANALYSIS

First, I conducted competitive analysis of the following six products to understand best practices for integrating gamification elements into skill training and interactive storytelling. The analysis is based on the gamification model we created from literature review.

QUICK PROTOTYPING

We made a functioning prototype in a week in order to quickly test our branching narrative concept. We used user story to help define what our minimal viable product features should involve.
USER STORY
While I’m reading an interactive story, I want to be engaged by the story and the choices so that I feel motivated practicing words and understand how the same activities in the software help me learn to read.
MVP FEATURE LIST
1. Write a narrative story with multiple paths for students to choose from.
2. Embed word challenge activities from the software into the narrative.
3. Create a helpful learning buddy in the form of a Robot.

PROOF OF CONCEPT

We went to a school district and tested 5 students currently using the software.
We found that…
1. Students like the story and are excited about choosing their own path.

I like choose-my-own-adventure over books.

2. They like “Robby”, the learning buddy.

He was helpful. He helped me figure out things.

Evidence of engagement
1. Students choose to repeat story with different choices.
2. Students put themselves into the story when considering different choices. For example, one student chose “boat” over “rocket ship” because boat was safer to her. Another student didn’t choose to “press the buttons“ because he thought “It’a really bad idea to start pressing buttons when the ship is going to crash.”
3. Mouthing words and using mouse to follow text as they read.

EMBED PERSONALIZED INSTRUCTION & GAMIFICATION

For our mid-fidelity prototype, we added spelling and comprehension questions as students wanted multiple types of challenges. We provided personalized Instruction in the form of videos when students get questions wrong. We also embedded gamification elements to help students stay motivated.
REVISED USER STORY
While I’m reading the adventure story, I want feel engaged and have personalized experiences practicing activities, so that I can read longer passages and increase my confidence in reading.
MVP FEATURE LIST
1. Create personalized instruction on spelling activity and word challenge.
2. Make comprehension questions to test engagement.
3. Use gamification elements (hearts, points, collectibles, and customization of the learning buddy Robby) to hold students accountable for the practice activities.

DESIGNING USER FLOWS

USER TESTING ON ACTIVITIES & GAMIFICATION

We tested the new prototype on 6 students. 
We wanted to understand…
1. How do students feel about the gamification elements?
2. Do the activities feel more integrated or more separate to the story?
We found that…
1. Students were positively engaged by hearts and points. Students prefer to exchange points for items after reading the story because they could get better items with more points. Some students prefer customizing Robby to collecting items.

Hearts make me think more carefully about my choice. It’s motivating. It shows me if I’m doing it right.

A 7th grader trying our prototype

I want Robby to be a girl wearing pink dress.

A 3rd grader trying our prototype
2. Two out of six students said the activities felt separate. Others feel activities went along with the story. .

It makes sense if I don’t do it, I won’t be able to get over the challenge.

A 6th grader trying our prototype

THE DANGER OF CHOCOLATE-COVERED BROCCOLI

As we set out to develop a high-fidelity prototype, we received feedback from our client and faculties. Both raised concern on “The activities felt separate from the story.” 
In the previous prototypes, we interleaved existing practice activities with the story. 
We made this decision because…
1.Our goal was to encourage students to see the value in existing exercises in the reading software, so they would be more likely to engage with these activities outside of the game. 
2. Having a set number of practice activities at fixed points is easy to scale. Content writers could roll out stories on their own timelines and just indicate where to embed the challenges.
However, though students enjoyed the story, and expressed the activities made sense in the context, we were not sure if they enjoyed the activities. Gamification elements held students accountable for the activities, but there is no guarantee that the students actually read the text and comprehend it well. 
The question for us became “How can we make sure our activities are not chocolate-covered broccoli?”

DEVELOP A STORY WITH NARRATIVE CHOICES & CONSEQUENCES

We shifted our focus to integrating practice activities into narrative as a whole. The choices reflect mastery of reading skills and have narrative consequences. 
We made this decision because…
1.This will make sure the core feedback loop is about reading the text. 
2. Using existing practice activities is less focused on engaging with the narrative and feels disconnected with the story.
3. Wedding the assessment to the narrative will limit the practice activities to those who need it.
REVISED USER STORY
STUDENTS: I want to have an immersive experience reading the story and making my choices. I want to feel supported and hold myself accountable in order to go down a good path in the story.
TEACHER: While I’m teaching, I want to use data to inform my instruction, so that I can tailor my instruction to the words my students are struggling with.
MVP FEATURE LIST
1. Narrative consequences that motivate students to read.
2. Comprehension and word assessments that hold students accountable to read the texts.
3. “Read it to me” function that helps students decode words when they need support.

CREATING A STORY ARC WITH SCENES

We took a modular design approach and conducted a co-design session to brainstorm on scenes in the story arc that can serve as assessments across stories. 
A sample story arc with different scenes.
Co-designing story scenes

USER FLOW WITH NARRATIVE CONSEQUENCES

The following user flow shows a students’ journey going through each story scene. The goal of having integrated comprehension questions in the flow is to keep students accountable for reading the text, and the three levels of word challenge questions will help provide data on the specific skill students struggle with. The flow also diverts students into various story branching based on their decoding skills, enabling narrative consequences that are personalized for each student.

DESIGNING STORY SCENES

We created an island treasure hunting adventure story and came up with four story scenes that work together to guide students find the treasure. The story scenes are “Siren’s song”, “Natives”, “Treasure” and “Danger (pirate attack)”. Based on our new user flow, I made the high-fidelity screens for “Treasure” and “Danger”. Then we conducted another round of user testing before committing to designing the four scenes.
The following user flow with hi-fi screens show how the “Treasure” scene would work in the story.

After we validated the story scenes with students through user testing, I continued to design the rest of the scenes. The following shows the other three scenes we weaved into the story as well as items students can collect on the way home.

Credits: The robot, room, island, space rocket, and zoo illustrations are designed by Freepik.

DESIGN DECISIONS

As choices are crucial in our story, we considered carefully how to visually show the choices in the narrative. There are two types of choices throughout the story——real choices and action choices, as shown below. Real choices typically have two to three options with real consequences, while action choice helps students move to the next screen.

DESIGN PRINCIPLES

STYLEGUIDE

USER TESTING ON NARRATIVE CONSEQUENCES

We conducted another three rounds of user testing for the high-fidelity prototype, two with summer campers, one with lifeskill students using the software. The summer campers all showed excitement using Endeavor, and believed the challenges were “realistic” and had “a reason to do them”. However, the lifeskill students needed more support to be able to use the product independently.

I like the cues. You wanna pick the right action. You don’t wanna mess up.

An 8th grader trying Endeavor

0 being the worst thing ever and 10 being the best this ever, I would rate it 21. You should market this!

An 9th grader trying Endeavor

Are we going to do this next year? I would love that!

An 8th grader trying Endeavor

EVALUATING SUCCESS OF OUR PRODUCT

FUTURE FEATURES

Given more time, our prototype can be improved in the following ways. First, more illustrations can be added in the story as image association can aid learning retention. Second, we suggest narrative consequences for choosing collectibles in the story. For example, if the user collects the spear, he or she can use it to fend off the pirate later. Third, helping students spend coins on things they could use in a future story, or give them a status symbol for their continued success will increase engagement. Last, we can help teachers gain more fine-grained data on students’ weak skills. 

FINAL TAKEAWAYS

In retrospect, what really went well in this project is our solid user-centered and evidence-based research process. We seized every chance to head to schools to observe and talk to students and teachers. In the two-month product development phase, we worked in two-week design sprints to iteratively refine Endeavor and obtain feedback through five rounds of user testing. Seeing students engaged using Endeavor is a huge motivator for us to keep improving the product.
We made two major changes in the process, one is changing our focus from designing a teacher-facing software to student-facing one, another is designing an integrated story with narrative consequences rather than a story with separate activities. Both changes are based on our clients’ feedback, which contributes hugely to the success of the product. We learned that asking for feedback frequently in the early and more ambiguous stages can help build a product that satisfies the need of both users and stakeholders.