[conspire] Risks of automation, part 2

[Rick Moen]

Some UI & documentation failures are more serious than others.

https://arstechnica.com/information-technology/2018/11/indonesia-737-crash-caused-by-safety-feature-change-pilots-werent-told-of/

Indonesia 737 crash caused by “safety” feature change pilots weren’t told of
SEAN GALLAGHER - 11/13/2018, 9:40 AM


[...]
Boeing never told pilots about one key new safety feature [of the
brand-new 737 MAX] -- or how to troubleshoot its failure.
[...]

In the past, anti-stall systems have either issued audible warnings to
pilots [..]

Indeed, have heard of those, one upon a time. They sometimes worked,
and sometimes not so much. Go on, please:

[...] or, as in newer systems, used a sensory feedback system that
warns the pilot by putting more resistance on the aircraft's control
stick or yoke when the pilot is approaching the critical angle. But the
new system in the 737 uses data from the aircraft's AOA [angle of
attack] and airspeed sensors to proactively counter pilot error,
adjusting the aircraft's controls to push the nose down if the sensors
indicate the aircraft could stall.

Initial data from the investigation of the crash of Lion Air Flight
610 indicates that the AOA sensor was providing "erroneous input,"
according to a Boeing statement. The aircraft had recently had an
AOA sensor replaced, and had experienced additional unidentified
issues; a maintenance technician was aboard at the time of
the crash, but not because of the AOA sensor.

The Operational Manual Bulletin sent out by Boeing on November 6
provides procedures for dealing with that sort of situation, but no
prior training provided by Boeing ever mentioned the automated system.

Oops.

The faulty AOA sensor data may have caused the aircraft's trim system
to lower the nose down in order to avoid a stall.
[...]

Yeah, guys? If building a novel system into a brand-new airframe that
_could_ spontaneously decide to override pilot input and _could_ act
erroneously, please consider mentioning its existence to aircrews.

Could avert, y'know, 189 fatalities + loss of a $130M airframe.