Overview

• Introduction
• Purpose and Scope
• Procedure
• Results

As the design of the modern flight deck continues, more and more computers are being installed to aid the flightcrew. Accidents and incidents involving automated aircraft have drawn attention to potential problems and concerns with these systems because safety-critical tasks that were once performed by the pilot are now being performed by the automation.

The NASA Aviation Safety Reporting System (ASRS) (http://asrs.arc.nasa.gov) database contains reports of incidents submitted voluntarily and anonymously by pilots and others directly involved with the aircraft operations. These reports cover a wide range of aviation related issues and include many reports on incidents involving an automated flight deck.

The purpose of this research was to use ASRS reports to identify flight deck automation issues and, further, to identify information in the reports that could be considered evidence for automation issues. The scope of this research includes ASRS reports submitted by pilots and covered by Federal Aviation Regulations Part 121/135 crew/automation interaction incidents. FAR Part 121 refers to domestic, flag, and supplemental air carriers and other commercial operators of large aircraft and Part 135 refers to air taxi operators.

In Phase 1 of our study we obtained incident reports from ASRS. 46,798 reports had been submitted to ASRS at the time of our query. To reduce the full database to incidents involving only automated flight decks, we submitted a query to ASRS requesting reports referencing Part 121/135 crew/automation interaction incidents. This allowed us to focus on reports of incidents involving advanced technology aircraft, while excluding reports of incidents involving conventional and general aviation aircraft. This query generated 591 ASRS reports.

We reviewed the narrative sections of the 591 ASRS reports for citations of problems or concerns related to automation and recorded them in a database along with similar citations from other sources. The narrative section of an ASRS report is that part in which the reporter describes the incident in his or her own words. We were specifically looking for excerpts in which the reporter claimed that some automation problem exists or expressed some concern about automation. In identifying these citations, we did not insure that the reporter claimed that the problem or concern contributed to the incident in question. In this phase of the research we were merely attempting to identify issues. In this initial analysis we identified 282 reports citing automation issues.

In Phase 2 we reviewed these 282 reports for excerpts clearly stating that the cited issue contributed to the incident. We recorded these excerpts in our database. We found that 282 of the incident reports cited an automation issue that contributed to the incident.

In Phase 3, we identified additional incident reports that were classified as referencing Part 121/135 crew/automation interaction incidents and were submitted between January 1997 and December 1998. There were a total of 1671 incident reports. We reviewed these 1671 reports for excerpts clearly stating that the cited issue contributed to the incident. We also recorded these excerpts in our database. We found that 608 of those reports had information related to automation issues.

We found evidence related to flight deck automation issues in a total of 890 incident reports. This total is comprised of the 282 reports we reviewed in Phase 2 and 608 reports we reviewed in Phase 3. A total of 64 separate issues were supported by this evidence. Details are presented in the following table. For each issue, we listed its issue identifier, the abbreviated issue statement, the number of incident reports supporting this issue out of 890 total reports, and the percentage that number represents. The link provides a complete set of incident report narratives supporting that issue.

Our results indicate that while pilots are generally very accepting of flight deck automation, there is evidence to suggest that they do have concerns about its design, function, and use. Furthermore, these concerns are justified in that the characteristics of automation underlying them do contribute to incidents.

We understand that ASRS reports are submitted on a voluntary basis and are subject to self-reporting bias. However, incident reports offer perhaps the only source of information on line operations that is open to this kind of analysis. As such, incident reports offer important insights into the operational environment.