Customer Development (post #7)

Our next step in the process was customer development. Before we got to that, we did need to pick an exam. The customer was not exactly the same for each exam. What we were looking for was an exam that was big enough to be attractive to us but small enough to keep away the indirect competitors. The direct competitors (HLT, Pocket Prep and Quizlet) were going to be there almost no matter what exam we picked. 

 Additionally, we were interested in trying to pick an exam in which it was possible for us to find and talk to people who were preparing for their exam. Our take was that we were trying to have conversations with millennials and Generation Z. So that meant that ideally we were trying to find a social media platform through which we could talk. Because that’s how they communicate. We typed in the name of several exams that we were considering for Facebook, Twitter, Instagram, and so forth. 

 The combination that seemed by far the most promising was the ASVAB (military entrance exam) and YouTube. What we saw was a lot of videos in which someone was just talking about how they passed the ASVAB. There was no educational content. The YouTube videos were by and large just someone providing emotional support and saying that you actually have to study and if they did it then anyone could do it. There were dozens of these videos. The surprising thing to me was that many of these videos had over 100,000 views, thousands of likes, and hundreds of comments. All of the people who were commenting were potential people that we could talk to about the problems they were encountering while taking the exam. 

 Another potential idea came to my mind as I was looking at these YouTube ASVAB videos.  Referral marketing. The people making these videos had an audience in the hundreds of thousands in many cases. Some type of contact information (usually email) was available on their YouTube page. Once we had an app, would they be willing to make a video recommending our app (assuming that they liked the app). I emailed a few of them offering $200 for a short video. Four of them responded saying they would be interested but of course needed to see the app first. At that point I had to tell them that the app wasn’t available yet, but I would contact them when it was ready. 

 The ASVAB was not really the exam that I had pictured, but the competition was pretty weak, over a million ASVAB exams were taken every year, and this YouTube thing seemed like it would be of value both for customer development and potentially for marketing down the road. Another advantage in my mind was that the questions would not be that hard to write. It would be a lot of high school geometry and algebra. I was pretty sure I knew who was going to be writing those questions. Simple content sounded better to me. If there were videos, then I would probably be making those too. We decided to focus in on the ASVAB customer. 

 Customer development is the practice of establishing a continuous and iterative communication line with your customer so that you can come up with ideas, hypotheses, try them out, get feedback, and adopt your product accordingly. 

The key point of customer development is we were trying to find out if we were talking about building the right thing. We also saw customer development as a tool for risk mitigation and opportunity recognition.

 Customer interviews help you understand the reasons why customers buy or don’t buy your product. In this framework, there are four steps in the product lifestyle (discovery, validation, creation, and building).

 Types of Interviews

There are actually four types of interviews, but we focused just on the exploratory and validation interviews because we were in the pre product phase. Here we really want to focus on pain points and validation. 

 Exploratory Interview: the most free form. You’re trying to establish whether or not they have a certain pain point and are open to certain solutions. You are exploring. Looking for insight. You can use it to come up with ideas. Talk about their day, the context in which they would use your product, see how badly they would want it and if they would pay for it. Ask open-ended questions. 

 Validation Interview: you have a theory and you want to test it out. These interviews are run in a scientific way. They are hypersensitive to bias. Don’t introduce your theory or idea until the very end. You try to be as objective as possible when describing your idea. See if they talk about this problem on their own. 

 Finding People to Interview

We went with the YouTube strategy first. Anyone who commented on a “How to Pass the ASVAB” video we tried to contact. We would go to their YouTube channel and see what information their about page would give us. If they provided an email, a Twitter handle, a Facebook page or an Instagram account that was best. I would estimate that maybe 10% of the people we tried to contact would respond, and maybe 10% of those ended up agreeing to a short interview. So that basically means if we contacted 100 people, then 10 would respond and 1 of those would lead to an actual interview. So a 1% success rate. But there were a lot of YouTubers that we could contact. 

 Another similar strategy that we utilized was to go to the Facebook pages of our competitors. HLT had Facebook pages specifically for each exam, including the ASVAB https://www.facebook.com/pg/ASVABMastery/posts/. This page had around 20,000 likes at the time. They had posts and would receive comments and likes for their posts. We would then send messages to the individuals who had commented. Basically the same percentage played out. It took dedication but in the end we got 20 people to interview with us. 

 Our Questions

We tried to stay from just having a list of questions to go through. Perhaps there was a problem that we were unaware of or had overlooked. So we were willing to let the interview go in any direction as long as we were getting information about the potential customer, their problem, what their average day looked like, and how and when they would use our product. 

 Responses 

Many of the people that we talked to were looking to join the military in that it was a chance at some type of new beginning. Many of them were working a full time job that they didn’t like and it wasn’t going anywhere (fast food, call center representative, cashier, etc.).  They were usually working full time. 

 They often did not have a desktop or laptop computer but did have a pretty nice smartphone. The majority was into video games, both on their phone and had some type of video game console at their house/apartment. 

 Many of them never did very well in school. Especially math. That was consistent. They were bad at math and saw this as a major problem in getting a good score or at least passing the ASVAB (each branch has a minimum score to pass the test). 

 Most of our interviewees had several scheduled breaks during their work day. Breaks would be for ten minutes or so. And a one hour lunch. Could they picture using our product for a few minutes during that time? Lukewarm yes was usually the answer. They would rather be on social media or playing a game. What if we figured out exactly the type of math that you needed to be learning? Maybe helping you to develop some skills so that the questions on the test start to get easier. Yeah, that would make it better. The response was definitely more positive there. 

 What if using this app during the day meant that you didn’t have to study as much during your free time? Now the light started to come on. Yes, that would be much better. If the time that they had to spend studying could be reduced then that would be very attractive. 

 Would you pay for this app? Yes but not much. This group was definitely not used to  having to pay for education. This was very different than our graduate school friends. They were used to owing tens of thousands of dollars for education, and ten bucks for an app would just be a drop in the bucket. This group was more sensitive. Yes they would pay, but they were not used to paying for education.

 While we tended to not pay much attention to features that people suggested, there was one suggestion that caught our eye. This person was really into working out. He used the Apple Watch. He talked about how they had different challenges and badges that you could get for hitting certain goals. He thought the app should be similar to a fitness app in design. You get points for correct answers and so many points equals a badge. And there were daily goals. In fitness, it was more like how many steps did you take. For us, it could be how many questions you answered. 

 We liked the idea. It would make the process more interactive and engaging if done right. We ran it by everyone we had interviewed at the end of the process and it was well received. Of course they would have to see it, they said. If the badges were military oriented (start of as a civilian, work your way up to five star general), that would be much better than the typical generic badges like superstar. It certainly seemed plausible that the more we made it like a game, the better. The fitness apps were a good reference. 

 Conclusions from Customer Development

·math is the specific content problem and pain point

·being able to use the app so that less free time is dedicated to studying is an idea that strongly resonates with this group

·customization combined with micro-learning was validated as being of value. However, customization needs to have the capability to go back to some fairly basic math concepts.  Just asking test questions had the potential for frustrating the user and losing them forever.

·even small successes need to be celebrated. People struggling with this exam likely do not have a lot of previous academic success. 

·price sensitive

·making it more like the fitness apps could make it more interactive and engaging. If this could be crossed by making it more game-like, that would probably be a good idea given the amount of video games this customer enjoyed. Ideas included points, point animations, a progress bar, challenges, badges (military based), and daily goals (number of questions answered, number of correct answers). This would make it so that the app was giving you visual awards as you progressed.