Since the launch of Mailchimp’s Facebook Ad Campaigns, we’ve gotten a lot of great feedback from our clients. Here's what our research team has learned.
With the notable exception of Silly Putty, few successful products are accidents.
Since the launch of Mailchimp’s new Facebook Ad Campaigns, we’ve gotten a lot of great feedback from our clients. Its success is owed, in large part, to the deep digging done by our research team.
If you work at an agency — and if you’re reading this, there’s a fair-to-middling chance that you do — you know the value of research. Whether you’re developing a website, building an app, or launching a product, good research is a requirement for success.
(Unless, of course, you’re selling the failed space shuttle polymer that became toy phenomenon Silly Putty.)
During the development of Mailchimp’s Facebook Ad Campaigns feature, Laurissa Wolfram-Hvass, Senior Research Manager at Mailchimp, and Laura Jones, Researcher, were heavily involved in the collaborative effort. However, as they both pointed out, the idea for something like the feature had been percolating for some time.
“There was not a direct line to this product,” Wolfram-Hvass says. “There were a lot of conversations throughout the company about the needs of small businesses and how Mailchimp could meet them. So we didn’t set out to build Facebook ads at first. We set out to understand how to empower small businesses.”
In conversations and surveys with small business owners, an overwhelming theme emerged: Time and money are extremely tight. So they have to get the most out of the tools at their disposal — and one tool almost all of them use is Facebook.
As the researchers narrowed their focus, it also became apparent where the experience could be improved based on how Mailchimp customers use it.
“One of the surprising things we discovered was that our users were exporting their own segmented lists from Mailchimp, then importing them into Facebook to build an ad audience similar to the people who already engage with their brand,” Jones says. “They were trying to get the same end result we now provide, but without the actual tool.”
Understanding what customers wanted to accomplish — and how they tried to pull it off — led to some of Mailchimp’s Facebook Ad Campaigns’ key features. But to get there, it had to go through development first.
Cliché would have you believe that developers work in solitude. Not so at Mailchimp, where developers, designers, and researchers work together closely on cross-functional product teams and share an interest in customer attitudes and behaviors.
“As researchers, we work closely with developers and designers to share information, user stories, and customer feedback,” Wolfram-Hvass says. “Teams are excited about listening in on a customer interview or observing a usability test.”
The product teams stay steeped in as much detail as possible and make sure feedback and facts are constantly communicated. Think of it less as information flow and more as an information bath.
“I’ve found sharing research in a variety of forms to be the best way to go,” Jones says. “With the Facebook project, we’ll use instant messaging, team chat rooms, and regular stand-up meetings to share updates and research takeaways. After usability testing, we hold debriefs, send out update emails, and share short summary documents with screenshots to make sure everyone feels looped in. Communicating in all these different ways allows people to stay informed without pulling them away from their work for too long.”
“There will always be questions that require more exploration and digging,” Wolfram-Hvass adds. “As members of cross-functional product teams, we’re around to gather those answers and that information quickly to help inform refinements.”
“Our job is to meet their needs, not sell them on something they can’t use."
So how exactly do you know when a product is user-ready? Once again, researchers play a key role — this time, in usability and beta testing.
“Early on, we may simply sit someone down with a design comp and see if they find and understand certain elements within the design. Or, we may have someone work with a rough prototype just to see how they interact with it or how they complete certain tasks,” Jones says. “But eventually, especially during beta testing, what we’re doing is basically like giving someone the keys to a car and letting them take it for a test drive.”
For researchers, designers, and developers, user testing is where things get exciting. It’s the first chance to see just how successful the product really is. Of course, sometimes all this excitement can lead to things getting a little cramped.
“During one of our Facebook beta tester interviews, we had at least a dozen people from 6 different departments crammed into this little meeting space so they could listen in,” Wolfram-Hvass says. “Everybody was so eager to hear the results and feedback.”
As exciting as interviews with beta testers can be, the ultimate feedback comes after a release to the entire customer base. With Mailchimp’s Facebook Ad Campaigns tested, refined, and tested again by a small group of customers, the feature was finally ready for everyone. And that’s where the researchers’ work really paid off.
“The feedback from our customers has been very rewarding,” Jones says. “With this product, we had a very clear goal to streamline the process of creating Facebook ads in a very easy, intuitive way.” We were all happy to hear small businesses get excited about how fast it is to create an ad in Mailchimp and their enthusiasm over the targeting possibilities. This was something our customers wanted, and it feels great to know we’ve delivered a good product.”
For both Jones and Wolfram-Hvass, it all comes back to helping small businesses save time and money.
“We keep harping on this, but time and resources come up in literally every interview we do with small businesses. For them, this stuff is non-negotiable,” Wolfram-Hvass says. “I’ve been so proud that we were able to create a tool that’s going to really help our customers. This just makes Mailchimp that much more powerful for small business.”
And they’re not done just yet. Even after a product launches, a good researcher keeps, well, researching.
“We’re still listening and gathering the information that will feed into future iterations,” Jones says. “We’re always learning. That’s what’s so exciting about this job. There are always new ideas and new ways of thinking about how we can do more. We’ve got exciting things ahead.”
The value of good user research is hard to overstate. When done well, it informs strategy, design, writing, development, and every other element of how you — or your clients — proceed. Here are 5 ways to make the most of it.
1. Research first. Solution second. Some ideas come with their own momentum, and those can be the most exciting. But without the proper research, your idea might be a solution to a problem that doesn’t exist. “Before we develop anything, we spend time understanding the customer,” Wolfram-Hvass says. “Our job is to meet their needs, not sell them on something they can’t use.”
2. Loop in the whole team. Great research doesn’t mean much if it’s not shared. Make sure everyone on your team who’ll be touching the project has access to the insight provided by research. “Our product teams are built to engage people across departments,” Wolfram-Hvass says. “We try to share research through multiple channels to make sure they’re not left out.”
3. Don’t be afraid to summarize. “Everyone gets busy with their own work and needs time and space to get it done,” Jones says. “Sometimes, to help them, you’ve got to break the research down and figure out what’s most important.” So yes, make sure your team has access to your research. But maybe give them a cheat sheet, too.
4. Plan your tests and test your plans. In the early stages, it’s important for teams to plan and identify key points of development when you’ll get the most benefits from user testing. But a good researcher can also take notes on the fly. “Throughout the process, there were times when developers, designers, or writers would run into a surprise,” Jones says. “It’s important to address those things before they become problems, even if they show up outside a planned testing situation.”
5. Be ready to dig deeper. Early in their research, Jones and Wolfram-Hvass realized a large segment of the small business owners they spoke with weren’t using Facebook ads at all. But rather than assume that meant they didn’t want to use them, the team took a closer look. “What we learned was that a lot of them have so much going on they don’t feel they have time to learn something new,” Wolfram-Hvass says. “They imagined Facebook ads to be this difficult thing they’d never get around to.” Instead of assuming a lack of interest, the team identified a problem — one that the new product could solve.