It’s interesting trying to introduce Cognitive Systems Engineering (CSE) to a new customer meeting (a mix of users, managers, and system developers). In some ways it’s a bit like speaking a foreign language. Terms, concepts, patterns, etc., that are fundamental to us are like puns…they are almost impossible to translate. (I once saw a James Bond film with German subtitles…the translation was accurate…but the puns were gone…the audience thought I was nuts for laughing.) We always get a bit of that when we talk CSE to a new crowd, particularly for something like an evaluation…where they are expecting things like “change font to Arial 12″ or “change this green to brown” and get things like “the workspace doesn’t adequately reflect the underlying domain workflow”.
System developers on many of our other projects are flocking to Agile Development as a way to ‘ensure user acceptance’. The premise is ‘build something’ (almost any design will do) and the users will then try to use it, gripe about it, and iteratively guide the agile team toward converging on a ‘user accepted system’ (assuming that to be synonymous with good Decision Support…but that’s a blog for another day). In this evaluation project, our design methodology is essentially turned inside out, into an evaluation methodology. The initial insights presented to the customers went remarkably well, largely because while we used Cognitive Systems Engineering as the basis for our approach, the resulting insights naturally lent themselves to being expressed from a user’s perspective. With the insights being spread across Usability (the standard ‘knobology’ kinds of comments), Usefulness (efficiencies of just getting common work processes done), and Understanding (related to truly effective decision support for fundamental problem solving…critical during novel, unanticipated situations), we were able to ‘ease’ them into the evaluation findings to date. We opened with the Usability ones, and the discussion naturally broadened to the larger impacts on the user when these flaws confused them. We then transitioned to the more abstract Usefulness and Understanding findings. These more holistic, abstract findings were tough to discuss in depth…but the magic happened when we started showing design recommendations. We call it the “of course test”…once customers see it, they blurt out “of course it should work like that!”
The first review meeting, surprisingly with findings at all three U Levels (Usability, Usefulness, Understanding), was a little slow in starting, but really gained an acceptance for Cognitive Systems Engineering, not so much for its underlying science, but more for its ability to detect issues deep within the conceptual design of the system and offer direct resolution to them. The redesign recommendations are what made Cognitive Systems Engineering valuable to the users, managers, and developers of the system.
Next up: Part 4: Second round of Insights-A Case Study in Cognitive Systems Engineering Usability Evaluation
Stumble it!