5 myths preventing you from usability testing

As the User Experience team grows at Q2, our usability testing strategy rapidly matures with it. We partnered with 8-Bit-Bear Consulting for formal testing on our developed products every few months, which helps us detect areas of our app that are difficult to use and identify solutions to optimize those features. We are also disrupting the normal testing process by weaving in informal, in-house testing into our sprint cycles, ameliorating problems before they transition from prototyping into development.

8-Bit-Bear Consulting : http://www.8-bitbearconsulting.com/


How have these tactics helped us?

Our efforts have resulted in a significant improvement to our System Usability Scale (SUS) score, but more importantly, we’re beginning to see the trickle-up effect of account holders contacting their financial institutions about their positive online banking experience.  While Q2’s mission is to build stronger communities by strengthening their financial institutions, our design team believes we can strengthen financial institutions by creating happier account holders.


How can they help you?

Dedicating some of your project’s budget to usability testing can boost your bottom line in a powerful way: Jakob Nielsen reported that spending 10 percent of a project’s budget on usability evaluation results in a whopping 84 percent increase in conversion rates. This translates into increased revenue for e-commerce sites, and for software that doesn’t make money directly off of successfully completed tasks, it means less annoyed users, less costly development rework, less support calls, and a greater overall level of user trust in your brand.


What usability myths are keeping you from success?


Myth 1: Usability testing demands an elaborate recruiting process

It’s best practice to test users that are representative of your product’s target market.  That said, testing any users is far better than testing no users, so don’t let the hunt for the perfect user stop you from testing.  A screener, or list of carefully selected questions intended to weed out users that don’t match the profile of your target audience, can affordably optimize your recruiting process.

Infragistics Senior UX Architect Jim Ross’s tips for writing great screeners include keeping the list of questions bearably short, starting with the questions most likely to eliminate testers so as not to waste their time, and recruiting based on the behaviors and attitudes you are looking for as opposed to less meaningful demographics.


Myth 2: We’d have to test a lot of users to get meaningful results, and we don’t have the time

The number of users you need to recruit for a usability study is much smaller than you might assume.  Nielsen advocates testing only five users, as the first handful of users has been shown to uncover the majority of usability issues, and each subsequent user becomes significantly less valuable to the study.  At Q2, my design partner and I employ Nielsen’s suggestion of testing only five users, then using the results to quickly iterate on our prototype, and testing again in a few weeks.  This way, we have continual reassurance that our designs are moving towards optimized usability without wasting time and resources.


Myth 3: Testing requires costly software and recording equipment

While some high-tech testing tools purport to automatically interpret users’ facial expressions, the reality is that such software can only detect strong emotions, and we as humans can do a better job at interpreting each others’ subtle expressions.  Simple, inexpensive programs like Screencast-O-Matic are more than enough, as they allow you to record a screen capture, audio, and a video of the user for a $15/year subscription.  Smashing Magazine also provides a robust comparison of affordable user testing tools that is worth checking out.


Myth 4: We’d have to hire a third party consultant, and it’s not in our budget

There is value in performing formal summative testing with a team of experts, as Q2 does with 8-Bit-Bear.  But if your company doesn’t have room in the budget for such services—and even if you are regularly using third party consultants—I argue that you should be performing informal in-house testing on a regular basis.  Iterative, formative testing allows you to evaluate prototypes while they are in progress, stopping major issues at the design stage before developers waste their time building something that isn’t usable.

Catching issues early isn’t the only benefit to formative testing in-house; involving the project team in usability testing will help them feel more invested in fixing them.  Watching users struggle through a task in real-time has much more of an impact on designers, developers and product owners than reading (or not reading) a long usability test report.


Myth 5: We don’t have the skills to perform in-house testing

While I don’t advocate moderating a usability test without bothering to educate yourself on best practices, it’s really not rocket science.  You will need to learn how to write effective tasks, for which MeasuringU has a great list of tips.  You should also make your tester feel comfortable, assure them that there are no right or wrong answers, allow them to speak without interrupting, and refrain from leading them to solutions.  Pairing up with another moderator in your sessions (as we do at Q2) will keep you honest, and one of you can ask questions while the other takes notes.


Now what?

With a little bit of practice, anyone with listening skills and an ability to remain impartial can learn to conduct a usability testing session. And while fancy equipment and a perfect recruiting process are nice to have, you can still get meaningful results with low-budget testing. Our user experience team is passionate about this topic and we would love to hear your thoughts. Share your insights on our social media pages.

Beyond Faster Horses

I once believed that I wanted more Star Wars films to be made (note: I am, of course, a huge nerd1). I was not happy with the Star Wars film that I eventually received in the summer of 1999 — known in my circle as “The Menace That Shall Not Be Named.” Most of my demographic segment have an attachment to the original Star Wars films that borders on unhealthy, as we saw those films during a part of our pre-adolescence during which the style, tone, and content are maximally impactful. What I wanted was to be five years old again, awed by cinematic spectacle and inspired by uncomplicated heroism. To ask a film to take me to this place again as an adult was admittedly a tall order (it does not help that the prequel films are awful, a note that I cannot resist adding here).

New Coke, Heinz’s EZ Squirt green ketchup, and the McDonald’s Arch Deluxe burger are well-known examples of consumer product failures that arose from research into consumer preferences and interviews with target consumers. Software suffers from a similar problem, as end users crave familiarity in their expressed preferences but find the delivery of that content or experience uninspiring. In the design world, many practitioners go so far as to assert that listening to users directly is a practice that all but guarantees failure. The reason being that it is difficult for users to accurately express insight into their own experiences and to divorce their actual preferences from the preferences that they wish to express in front of others. Few of us would admits publicly to watching “Amish Mafia”, but the show has had four seasons (note: I do not watch “Amish Mafia”2).

This is not a call for all of us to stop listening to our customers, far from it. However, there can be a broad gap between the stated end-user preferences, such as “just make it work exactly like it did before” and the actual end-user performance and experience. Measuring end-user engagement and task completion with software interfaces is critical for arriving at objective assessments of usability and capability, not just using interviews and ad hoc feedback to drive design. The counterexample that illustrates this principle is the extermination of physical keyboards on mobile phones driven by the iPhone’s introduction in 2007. Although many end users expressed strong preferences for a physical keyboard, the industrial design and performance of full-screen touch interfaces is clearly superior for apps (and arguably for messaging). Or as I sometimes like to daydream of Henry Ford lamenting: “If I had asked people what they wanted, they would have said faster horses.” Innovators hear the desire for the faster horse and think about how to fulfill this wish in a better way.

1 I have, as an adult, watched the entire Star Wars Holiday Special (look it up if you dare).

2 I do not watch it, but I would Executive Produce the show out of my own pocket if the Star Wars prequels could vanish from existence.

Are You Teaching Your Account Holders Anything?

One of my most profound experiences—professionally, anyway—occurred about ten years ago while visiting with my doctor. Doc was standing with his back to me, entranced in whatever it was he was fiddling with, when I said to him, “I guess it gets pretty irritating with all those pharmaceutical reps running around here all the time, huh?” It was more small-talk than visiting – I certainly didn’t expect to solicit the response I did. Doc stopped what he was doing, turned, and said, “Actually, no, it’s not irritating at all—I don’t know what I’d do without them.”

Say what!? I thought. Did I hear you correctly, Doc?

“I swear I’ve gotten dumber every day since the day I graduated from medical school, Will; if it weren’t for the drug reps, I wouldn’t know what the heck was going on.”  Needless to say, I was surprised at his response. He went on to explain that he’s so busy messing with insurance companies, paperwork, billing, seeing as many patients as possible and, oh yeah, being the CEO of his medical practice, that he doesn’t have time to keep up with the latest techniques, procedures and clinical trials—he relies on the pharmaceutical reps to keep him informed.

Not coincidentally, the reps who kept him most informed, were the reps for whom he wrote the most scripts. And that’s when it all came together for me: teaching is selling, and selling is teaching. If you’d like to get your tellers selling, get ‘em teaching.

Dictionary.com defines consult as: to give professional or expert advice; to serve as a consultant. Teach, however, is defined as: to impart knowledge or skill; give instruction to. Now, which would you rather receive: knowledge or advice? If you’re like me, the choice is obvious: knowledge! And your account holders undoubtedly feel the same way.

Studies have shown that consumers much prefer to buy things, rather than be soldthings, and they definitely don’t prefer being “advised” of what to do. It’s when your account holders have been taught something—partaken of your knowledge—that they feel empowered to make informed decisions of their own free will.

After more than fifteen years of working with community financial institutions, I’ve heard bankers repeatedly say that selling isn’t easy. Specifically, “We just can’t seem to get our tellers to sell, Will.” The problem, however, isn’t that they can’t sell—it’s that they’re not teaching. I once managed a team of relationship managers and many of them told me they were uncomfortable with “the selling part.” They felt that as relationship managers, they were violating the sacred, unspoken terms of their impartial, objective, client-advocate status. When I told the relationship managers to replace the word selling with teaching, their cross-sell numbers went through the roof.

To remedy the perceived problem with selling, instill a culture of teaching among your tellers. Educate them on the benefits of this new and improved approach—greater trust between you and your account holders; stronger relationships as a result of greater trust; and less attrition as a result of stronger relationships—and they’ll buy into it.

And know that your account holders aren’t the only ones who appreciate knowledge—your employees appreciate it just as much. Assign various folks to become subject matter experts, and then have them teach their peers.

Now is the time to get your tellers teaching. According to a 2013 study by Power Consulting, during which 839 in-person audits were performed by small business owners at 38 financial institutions (FIs) across the country, nearly 40% of FIs audited never even mentioned their FI’s business mobile offerings.

A Cisco IBSG study found that “financial education” was one of Gen ‘Y’ and Gen ‘X’s’ greatest needs; nearly 40% of both groups surveyed desired “help managing their finances.” If you’re the FI that fills that need, you may just have an account holder for life. Just remember: teaching is selling, and selling is teaching. Are you teaching your account holders anything?

Will Ferrell is Vice President of Product Marketing for Q2.  Will is responsible for shaping the messages and painting the pictures that tell the story behind Q2’s passion for strengthening communities by strengthening their financial institutions. A veteran of the virtual banking and payments industry, Will has spoken at numerous industry conferences and tradeshows, teaching anyone who will listen about the power of the virtual channel to improve lives, all the while dodging “thought leaders”.


The Perils of Information vs. Knowledge, Empowerment, February 14, 2011, Stan Lepeak, Managing Director Global Research)
— Power Consulting
— Celent

Social Media Strategy Tips for Your Financial Institution

Is your financial institution using social media yet?

Is your financial institution using social media yet? Some of you might say you have just made your page and posted a few tweets on Twitter or a couple of pictures on Facebook. That’s a great start, but did you think about your strategy and policy before you started? When starting social media it’s important to take a crawl, walk and run approach and I’ll let you know what I mean by that.

1. Crawl – This is where the research begins. For financial institutions there is a great place to start your research and that’s cbanc Network. (If you haven’t heard of it yet, I highly recommend checking it out.) You’ll need to do research within and outside of your company to 1. Build your FIs Social Media Strategy and 2. Create your Social Media Policy.  Your strategy should include who your audience is, how you are going to reach them, what you will post about and how often are you going to post, etc. Your Social Media Policy is more focused on internally how you will protect confidentiality, keep employees engaged with your FIs pages without confining them, etc.

2. Walk – Now that you have completed your research, it’s time to get into the fun part. This is where you create your Facebook Page and Twitter account. This is when you upload your profile picture for your page and get to choose your background for your Twitter page. The creation process has begun! Not only are you creating the pages, you now need to create some posts. After doing your research and creating your strategy you should have an idea of what your FI is going to post about. Maybe it’s about events you are hosting or a local business client’s success. This is the time to take it slow and see what people are interested in hearing from your FI. Make sure you are starting conversations with your customers and members. What do they respond to?

3. Run – Now that you have your Social Media Strategy and policy in place and you have started posting, your FI might have a good group of followers and Likes. You feel comfortable posting and have conversations going on. Don’t settle for what you have though, strive for more. This is the step where you can create a newFacebook App – try showcasing your employees or launch a contest.

Try out this crawl, walk and run approach and see how it works for your FI!