How Optimizely Climbed the Customer Education Curve (Part Two)

Adam Avramescu has provided another great guest post. This one is a bit longer, but worth the read!

​Adam is head of the Customer Education program at Optimizely, the world's leading experimentation platform. The Optimizely Customer Education program is called Optiverse, where customers learn about more than how to use the Optimizely product features, and includes a knowledge base, academy, academy live, community, and certification. I met Adam at the Learning DevCamp 2016, happy to meet someone in a conference full of training professionals who also focused on training customers vs. employees.

Part 2: Crossing the Content Chasm

By the end of 2013, Optimizely had filled the major gaps in its Knowledge Base and produced several high-quality videos that helped customers self serve more easily. But support tickets were still on the rise, and we were hearing a common theme from many segments of our customer base: “It’s great that you have knowledge articles, but we want to learn Optimizely from beginner to advanced. Show us the path to value.”

My background is in instructional design, so this theme really spoke to me. It was time to get to work on our first Optimizely Academy.

In many ways, this felt like the situation that Geoffrey Moore describes in Crossing the Chasm, where technology companies and products move from a user base of self-motivated early adopters to a more risk-averse majority. Users in the majority group are less willing to assume the risk of adoption technology sight unseen, are less tolerant of bugs, and prefer both social proof and more detailed training or instructions.

Image from https://www.support.com/crossing-the-chasm-for-the-internet-of-things/

As Optimizely crossed the chasm with its customers, so did our learning materials. But you don’t just create a full learning center overnight, so here’s how we got to the first version of what we now call Optiverse Academy.

The Big Dig: Training Content Excavation
Before we even put pen to paper (or fingers to keyboard, anyway) on a concept document for our academy, we wanted to get a sense of the core path to value for new Optimizely users. We knew there were different roles at Optimizely delivering different training sessions, but there were a few problems:

• The “trainings” weren’t actually trainings. In many cases, they were knowledge dumps with no interactivity, no chance for the participants to test their skills, and pretty poor retention after the fact.
• Content was different from person to person. There was no guarantee that a customer would learn the same thing from one CSM or Technical Account Manager to the next. Not great for the customer, especially if their account team ever changed hands.
• There was no way to get back to the content after that initial training, aside from watching the recording of your session (if we even did a recording). Not great if a new member of your team came on-board months later.

We decided to do a trial run of the content by hosting three different training webinars, pointed at the core skills that we taught customers as they onboarded with Optimizely. We gave them value-oriented titles and had subject matter experts on our Customer Success team deliver the content. We called this series “Optimizely Launchpad,” and the three courses offered bi-weekly were:

• Setting Up Your First Experiment: The 5 Steps in Every Test
• Using the Optimizely Editor: Best Practices for Editing Like a Pro
• Making an Impact: Testing Strategy, Methodology, and Hypotheses

Note that, at the very least, we tried to make our content more about workflows and best practices -- not just product training, but some focus on how you can more effectively use our product to do your job.

We didn’t market these sessions to our entire customer base, so attendance was low, but we also used the webinars as an opportunity to refine the content so we could record it and post it online. Now, our Customer Success team at least had something to point customers to as a learning resource.

Minimum Viable Academy

Sometimes when I speak to Customer Education leaders who are tasked with building a program from scratch, they say, “I really want to build an academy for my users, but I can’t get the budget for an LMS approved.” Good news -- you don’t need to, as long as you understand the tradeoffs.

We designed and built the first iteration of our Academy on a second Zendesk Help Center instance over the course of four months. With those two restrictions (no new software, and four months to get to release), we made some deliberate decisions about what we would and wouldn’t include. Here’s what made the list (and, maybe more importantly, what didn’t).

We said YES to:

• Content tracks. We created paths spanning four different skill levels, and covering four different subjects (Platform skills, Strategy skills, Implementation, and Results interpretation). Each path featured a combination of text-based lessons, videos, and activities.
• Video. Many, but not all, lessons had corresponding videos. This was a good reason for us to produce more new video content that we could also repurpose in our Knowledge Base. Originally we wanted to add live-action “bumpers” to the beginning and end of the courses where you could see a real person from Optimizely introducing the video, but we cut this from the scope.
• Bright, sharp copy. With time in short supply for us to produce engaging, interactive content, we instead tried to engage users through the power of the copy. Each lesson was written with a strong voice -- friendly, helpful, sometimes a little quirky or funny. We received positive feedback on this, and it ended up being a good way to engage customers and make the content feel more human.
• The “Why”/WIIFM: Instructional Designers are taught to focus on the “WIIFM”: What’s In It For Me? We tried to lead each piece of content with explicit focus on how this would help customers do their jobs better, or why the skill we were teaching is actually important.
• Callout boxes. One of the biggest design changes we made for our Academy was custom-designed callout boxes for notes, tips, warnings, and examples. This added visual contrast and helped to highlight and chunk information. We repurposed these callout boxes back into our Knowledge Base as well, since both the Knowledge Base and Academy were hosted on Zendesk instances.
• Stories and examples. Customers responded well to stories of other companies who were in their position, and other people who were going through the same situations they were. So we created three fictional companies with their own fictional optimization teams, who were learning the same lessons as our actual learners. Each of these fictional companies represented one of our key customer verticals at the time (e-commerce, media, and B2B). If I had to re-do this, I would have used more actual customer stories, but the simulated stories were quicker to produce because it took less sourcing, and we didn’t have to clear them through Marketing.
• Self-assessment. At the very beginning of our Academy, we designed a quick self-assessment that pointed you toward the skill level that was most appropriate for you. This was not interactive or dynamic in any way -- just an HTML document -- but this was effective in giving customers a sense of where to start, and setting expectations correctly.

We said NO to:

• True interactivity. That’s right -- no Articulate or Captivate for this first version. It was painful not to have actual interactivities where users could test their skills and receive meaningful feedback, but e-learning modules would have added too much production time, and we weren’t going to have these SCORM files reporting into an LMS anyways. Instead, we came up with written instructions for activities where customers could go into the Optimizely products, follow the instructions, and apply their knowledge that way. Although we did go back and create more rapid-dev e-learning content; the demise of Flash, limitations of SCORM, and general performance issues make me think that we shouldn’t have gone down that path at all.
• User-based tracking. This was one of the bigger sacrifices we ended up making. User-based tracking seemed essential for us to (1) be able to report on individual users’ progress through the Academy, and (2) flag content as done vs not done, so a user would know what content they had or hadn’t completed. But we launched without this, opting instead to look at aggregate metrics in Google Analytics and get qualitative feedback from our customers. The point of this first Academy was to validate the flow of the content, and once we validated that, we were ready to invest in a system that could support user-based tracking.
• “Next” buttons. Ouch -- this was a miss. That’s the #1 thing that our customers said was missing. Turns out that people want to know where to go next!
• Sandboxes. Having sandbox environments to make the experience truly interactive is what makes platforms like Salesforce Trailhead so powerful. We opted not to do this for our first version of Academy due to the technical complexity.
• Live sessions. Although we had been doing our Launchpad webinars on a recurring basis, we opted not to include these in this Academy -- and, in fact, to discontinue the Launchpad sessions due to low attendance. We wouldn’t bring this back until we launched our Academy Live program in 2016. The biggest lesson we learned here is that just because something seems like a good idea (training webinars seemed to be tried and true when we first built them), it doesn’t mean it will work perfectly for you. You might not be ready for it yet. But the live sessions were a great way to validate new content, and we currently use our Academy Live program to introduce new content as well.

After we released this first version of Academy, we saw engagement steadily grow as CSMs referred customers to it during onboarding, and from in-product promotion. Within the first few months of launch, our Academy was receiving roughly 500 unique visitors per month, representing roughly 10% of Optimizely’s customer base at the time. Considering that Academy was designed primarily for onboarding users -- not mature customers -- we considered that a healthy number for our first version. But more importantly, we learned from the feedback they gave us, as well as which lessons they tended to engage with most and least.

Today’s Academy is hosted on a real LMS called Docebo, and the interface and UX is much different from that first version. We’re also offering multiple avenues for training, including live sessions and custom, private trainings, but our Academy continues to be the most scalable platform for customers to learn at their own pace.

How were we able to justify continued investment? Well, in addition to the adoption metrics I mentioned above, our Academy was a piece of a bigger launch -- our combined customer portal, Optiverse. Optiverse made a meaningful impact to our business when it first launched, and has continued to mature since then. I’ll get more into those details in a future article.