In the March edition of Wired Magazine, two authors touched on Cognitive Systems Engineering in the course of their discussions of the current financial crisis. Daniel Roth wrote an article entitled “The Road Map for Recovery” about what it would take, in the way of strategic changes to rein in the financial markets and prevent the causes of the current market recession. The article focused on what it would take to inform investors adequately of the fundamental makeup (and risks) associated with the “products” they were investing in. Felix Simon wrote an article: “Formula for Disaster” discussing the causes of the collapse. The basic premise of the two articles was that if the true underlying data were available, and visual analytic tools were available, the wisdom of the masses would collectively be a more powerful force in detecting and revealing the underlying truths of the offering, helping would be investing make better decisions (and therefore not buy “CBO Squared Products” and Ponzi schemes).
In that article, the authors unknowingly rediscovered two fundamental elements of Cognitive Systems Engineering: Observability, the principle of exposing the underlying processing and transformations in a way that allows understanding and control; and Emergent Pattern-based Displays, the technique of design where the holistic meaning of the integrated data set reveals itself as patterns in the data itself, employing the perceptual system to detect this holistic meaning from the data presented in an intuitive frame.
In the articles, Roth wrote: “…the era of <data access> has to give way to the era of pixelization; only when we give everyone the tools to see each point of data will the picture become clear…“. This is*exactly* what has been discovered by the Cognitive Systems Engineering community. Filters, Goodness Scoring algorithms, relevance ranks, etc. are knowledge-sparse when compared to a properly framed Emergent Pattern-based Display coupled with the elegance of human perception and pre-cognitive processing.
Salmon also wrote: “…one reason was that the outputs came from ‘black box’ computer models and were hard to subject to a commonsense smell test….” Salmon was describing how systems without Observability are essentially impossible to fathom. The Cognitive Systems Engineering community goes further: users will invent ‘urban legends’ about how such systems work, how they respond to inputs, and how they ‘think’. These ‘urban legends’ tend to be very anthropomorphized and very rarely actually correlate to how the algorithms actually work, particularly for any complex, or non-linear mathematical formulation. These urban legends routinely collapse under atypical situations (since they were developed during ‘normal operations’) resulting in additional user confusion at the worst possible time.
Later in the same article, when talking about attempts to reduce the true underlying complexities to a single, ’simple’ metric (in this case loan correlation), the author states that “…co-association between securities is not measurable using correlation because past history can never prepare you for that one day when everything goes south. Anything that relies on correlation is charlatanism….” Yet traders built massive financial portfolios on exactly this correlation metric, even though it was a flawed simplification. They liked it because of its simplicity, but failed to have any Observability into its underlying brittleness.
In short, you find Cognitive Systems Engineering ‘case studies’ in a variety of places. In this case, studies in Observability and Emergent Pattern Based Displays are evident in the Wired Magazine edition about the current financial crisis.
Stumble it!