Operator Roles, System Complexity, and Failure
2010-03-16 02:31:14 // The Operator
Tagged: failure
Automation does not remove humans but tends to redefine their roles: operators become concerned with maintenance, repair and higher level supervisory control and decision making.
Sociological and memetic organisms supplant individual consciousness.
Operators relegated to central control rooms
Case in Point: 1977 NYC Blackout
- Indirect Information
- Operator followed prescribed procedures
- But electrical system was brought to a complete halt
- Operator could not know there were two relay failures: (1) one leading to a high flow over line normally carrying little or no current (operator would have been alerted) and (2) other blocked flow over line making it appear normal
- Operator had no way of knowing that zero reading would appear normal
- Operators become the “scapegoat” of an automated system
Consider: Assassination [fly-by-wire] and mutually-assured destruction [nuclear; Dead Hand retaliation] increasingly automated – destruction-by-proxy encouraged, full circumvention of individual decision making preferred.
Operators and Embedded Systems
- Embedded systems can mask the occurrence and subsequent development of a problem
- When malfunction is discovered it may be more difficult to control
- Systems may be hidden or distorted
- Such design further limits operator options and hinders broad comprehension
- Safeware by Nancy Leveson
… AZATHOTH — hideous name …
