[Sigia-l] Measuring user satisfaction and intuitiveness of asystem

Wed Apr 27 14:15:15 EDT 2011

Hello Darin,

Thank you for your interesting response and for sharing your thoughts. 

I understand that measuring user satisfaction is subjective and limited as well that the wording of such a questionnaire need be carefully worded. I agree on all of those points. 

To me usability testing and a user satisfaction questionnaire get at two different metrics - the former more objective and the latter more subjective. There is definite, indisputable evidence for the use of both together and I won't argue that point. I suppose I don't see, though, why the subjective metric cannot be measured on its own for the purpose of gauging the user experience from the subject's own opinion. We can definitely reference it on various releases of the application to see if we're making progress or not.  

Right now we do not have a grasp on the user experience qua user experience - not the efficiency of the system, whether certain screens are less than usable, etc. This information may be more important to our team right now as a way to gain stakeholder buy-in for further UCD efforts going forward. If we can say that user satisfaction is low then we know we need to drill into it to find out why: trouble points, where we can improve, and what enhancements we need.

We do have customer satisfaction metrics but that is the business customer who is rating their experience with our company and their satisfaction with the functionality our system offers compared to their needs. To me that seems quite different than the actual user who works on the system all day and who may rate the system differently in their experience. For our clients we have the layers of the business customer and the actual system user. 

I suppose I just struggle to understand why measuring user satisfaction alone isn't valuable. I've come across that idea a few times. Maybe I am missing something.

Thanks,

Ron Polka
User Interface Design Engineer

Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipients and may contain confidential and/or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail, delete this message and destroy all copies thereof.

-----Original Message-----
From: sigia-l-bounces at asis.org [mailto:sigia-l-bounces at asis.org] On Behalf Of darin sullivan
Sent: Wednesday, April 27, 2011 1:54 PM
To: SIG Information Architecture
Subject: Re: [Sigia-l] Measuring user satisfaction and intuitiveness of asystem

Hi Ron... At least quantitatively speaking, validation of your UI can be
accomplished utilizing standard techniques such as GOMS
Analysis<http://en.wikipedia.org/wiki/GOMS>, which
will measure the efficiency of a user's interaction with a system. A survey
of your user base can be used to help establish their level of satisfaction,
but questions must be carefully worded so as to minimize bias in the
response. In my experience, only usability testing will effectively measure
the quality of an interaction with a system, or how users generally feel
about their interaction. Usually analysis is done in parallel with usability
testing, to best establish both the efficiency and quality of the UI.

A simple example of why both measures are important: When users move through
open applications using keystroke combinations, such as Alt-Tab on Windows
and Command-Tab on Mac, it is often much more efficient to combine a third
key, Alt-Shift-Tab, to move to the target application. However, most users
prefer to simply repeat the Alt-Tab combination to move through the list of
applications rather than add the third key, because it's easier. There are
many examples of highly efficient systems that are used very little because
users don't like them (not easy or intuitive), and well used systems that
are very inefficient. Analysis and testing combine to provide important
guideposts to engineers, designers and stakeholders and can help to
establish the value of a user centered process.

In terms of measuring the system as a whole, most businesses in my
experience are unwilling to analyze or test a system in its entirety and
prefer to look at trends in the size of their customer base and customer
satisfaction before and after large implementations-are we now adding more
customers than we're losing (churn rate) and how are they rating our product
or service? These are both quantitative and qualitative measures too, and
can help to identify where analysis and testing can provide the greatest
benefit.

Good luck, and keep us posted :)

On Wed, Apr 27, 2011 at 8:32 AM, Polka, Ronald <Ron.Polka at mckesson.com>wrote:

> All,
>
>
>
> We have a system that was designed largely without utilizing a
> user-centered design process. We're constantly facing challenges on how
> to validate our designs and the UI of the system. Some stakeholders
> within the organization think the user experience of the system is
> intuitive and satisfactory while others think the exact opposite.
>
>
>
> My question: is there a standard measure I can use to validate whether
> or not our user base finds their experience with the system in general
> as satisfactory and intuitive.
>
>
>
> I've come across the SUS, Microsoft's Desirability Toolkit, QUIS, USE,
> and other measures out there but it seems these measures are designed to
> be used in conjunction with a usability testing session focused on
> specific screens in the system. I'd like to measure the system as a
> whole so as to gauge where we stand today with our users.
>
>
>
> Any thoughts or comments? Has anyone run into a similar scenario or
> situation - if so what did you do?
>
>
>
> Thanks,
>
>
>
> Ron Polka
>
> User Interface Design Engineer
>
>
>
> McKesson Corporation
>
>
>
> Confidentiality Notice: This e-mail message, including any attachments,
> is for the sole use of the intended recipients and may contain
> confidential and/or privileged information. Any unauthorized review,
> use, disclosure or distribution is prohibited. If you are not the
> intended recipient, please contact the sender by reply e-mail, delete
> this message and destroy all copies thereof.
>
>
>
> ------------
> 2012  IA Summit
> March 21 - 25, 2012
> Hyatt Regency New Orleans, LA
> -----
> When replying, please *trim your post* as much as possible.
> *Plain text, please; NO Attachments
>
> Searchable Archive at http://www.info-arch.org/lists/sigia-l/
> ________________________________________
> Sigia-l mailing list -- post to: Sigia-l at asis.org
> Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
>
------------
2012  IA Summit
March 21 - 25, 2012
Hyatt Regency New Orleans, LA
-----
When replying, please *trim your post* as much as possible.
*Plain text, please; NO Attachments

Searchable Archive at http://www.info-arch.org/lists/sigia-l/
________________________________________
Sigia-l mailing list -- post to: Sigia-l at asis.org
Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l