After the recent DC Metrorail crash, which tragically took 9 lives, the standard flurry of reporting activity followed. Any time an accident like this happens (see the Air France crash before it), all the reporters and amateur investigators try to pinpoint blame – and the easy answer is almost always to identify the culprit as human error at a single point of failure. However, one recent article looked a little further and began examining the problem from a more powerful point of view – taking the same human-centric approach to systems that we do at RCS.
Mr. Vedantam of the Washington Post explored the problems that come with introducing automation into systems that humans have traditionally controlled. He correctly states that the problem often comes from a poorly designed relationship between the human and the automated system. In fact, the successful operation of ANY system depends on the relationship between the human, the artifacts the human uses, and any software agents (aka automation) that play a role in operation (known as a Joint Cognitive System). This relationship must be properly supported in all systems, not just when interacting with automated systems. Designing from this perspective, which is what the ACSE approach is about, drastically changes how system designers approach designing systems to support users. This approach forces designers to consider the implications of their design decision on the whole system, rather than on the construction of the individual components.
When adding automation, the major implication of this approach is that the human user is not replaced from the system, as some designers want to believe, but instead switches from operator to supervisor. In fact, in some ways the demands on the human *increase* as they now have to know their original role, but also now become expert in how to validate the automation’s decisions and how to interact with it. Automated agents are essentially another form of operator, and thus have to be team players. Users should never be told to not interfere while being expected to be the last line of defense for proper operation. That means users need to have the same information that the automation has to make decisions (and presented in a manner that the user can quickly understand and make decisions from), just in case the unexpected happens and the automation isn’t able to perform. This is essential especially in situations where the unexpected happens, since it is the human ability to think and react in unexpected situations where human judgment and action is unmatched – something no level of automation can ever replace.
During these times, when the user only has seconds (or less) to react, the overall system design must come through. Good informative feedback is important, as is a clear representation of what the automated system is doing, so the user can quickly step in and take the right actions. For example, the user must know when the automation is nearing its operating limits, or when the automation has failed (as it did in the DC Metro crash), so that they can switch back from manager to operator without the overall JCS skipping a beat. This is a lesson spoken frequently throughout the history of Cognitive Systems Engineering (see Lützhöft and Dekker, 2002, for an in depth evaluation of the problem found during the Royal Majesty incident mentioned in the article, where automation failed and the user was unaware), and it appears that others are catching on as the addition of automation has continually failed to lead to an accident-free world.
If no one takes responsibility for designing the JCS as a whole, it is always the human user who has to deal with the consequences. And sadly, they are almost always burdened with the blame. Taking into account the CSE perspective, as Mr. Vedantam has, hopefully that onus will be shared with system designers to build a better JCS.
Lützhöft, M. H. and Dekker, S. W. A. (2002). On Your Watch: Automation on the Bridge. Journal of Navigation, 55(1), 83-96.
Stumble it!