FDAI logo   ::  Site Map  ::   
Home  |  About This Website  |  Contact Us
Home » All Evidence from Experiments

All Evidence from Experiments 128 matching pieces of evidence found.


  1.  
  2. Evidence Type: Excerpt from Experiment
    Evidence: "MANOVA results also showed a significant Control Type× ATC interaction for workload (F [1,11] = 4.73, p < .05). Post hoc tests showed that: 1) without ATC, Control Type had no effect on pilot workload, and 2) with ATC, average pilot workload ratings for using the voice interface were 22% lower (average rating = 46) than those for the manual interface (average rating = 59). The weapon delivery workload results are depicted in Figure 9 (next page)."
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  3.  
  4. Evidence Type: Excerpt from Experiment
    Evidence: "The results verified the hypotheses and showed that the pilots were able to designate targets more quickly using voice control coupled with the ATC than with manual control coupled with the ATC (F [1,11] = 4.79, p < .05). Further, pilots reported a decrease in workload when using the voice versus manual interface in combination with the ATC (F [1,11] = 4.73, p < .05)."
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  5.  
  6. Evidence Type: Excerpt from Experiment
    Evidence: "In regard to workload during rerouting, there was a significant two-way interaction for Control Type× TF Task Load (F [1,11] = 4.84, p < .05). Post-hoc tests showed that: 1) during high TF Task Loading, when the voice interface was used to accomplish mission rerouting, it decreased pilot workload by 31% compared to using the manual interface, and 2) during low TF Task Loading, Control Type had no effect on rerouting. During high TF Task Loading, the pilots’ average SWAT ratings to reroute the mission dropped from a 39 with the manual interface to a 27 with the voice interface, whereas during low TF Task Loading, pilots’ average SWAT ratings remained virtually constant at 34."
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  7.  
  8. Evidence Type: Excerpt from Experiment
    Evidence: "In regard to workload during rerouting, there was a significant two-way interaction for Control Type× TF Task Load (F [1,11] = 4.84, p < .05). Post-hoc tests showed that: 1) during high TF Task Loading, when the voice interface was used to accomplish mission rerouting, it decreased pilot workload by 31% compared to using the manual interface, and 2) during low TF Task Loading, Control Type had no effect on rerouting. During high TF Task Loading, the pilots’ average SWAT ratings to reroute the mission dropped from a 39 with the manual interface to a 27 with the voice interface, whereas during low TF Task Loading, pilots’ average SWAT ratings remained virtually constant at 34."
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  9.  
  10. Evidence Type: Excerpt from Experiment
    Evidence: "Results of the weapon delivery questionnaire data were consistent with the navigation segment questionnaire data. During weapon delivery, pilots rated the voice interface combined with the auto-target cue as most effective for identifying targets when compared to the manual or voice interfaces alone or the manual interface combined with ATC. For designating targets, pilots rated both control types effective regardless of whether or not ATC was used, but they rated their performance most effective and gave their highest ratings for the voice interface combined with ATC."
    Issue: automation performance may be limited (Issue #126) See Issue details
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  11.  
  12. Evidence Type: Excerpt from Experiment
    Evidence: "Pilot questionnaire responses confirmed that the voice interface was significantly more effective in completing the mission reroute than the manual interface. The effectiveness scale ranged from 1 (Poor) to 5 (Very Good). Across pilots, the manual interface was rated as neither a hindrance nor an advantage for accomplishing the reroute task (average rating = 3.2). Across pilots, the voice interface was rated as highly effective for accomplishing the task (average rating = 4.8)."
    Issue: automation performance may be limited (Issue #126) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  13.  
  14. Evidence Type: Excerpt from Experiment
    Evidence: "Other tasks during the navigation segment in which the voice and manual interfaces were compared included: 1) correcting system malfunctions, and 2) changing radio frequency channels to place radio calls. For changing radio frequencies—a task similar in nature to rerouting in that a string of alphanumerics was entered into the simulator, the voice interface improved pilots’ speed and accuracy to accomplish the task compared to the manual interface (F [1,11] = 7.63, p < .05). However, the improvement was less than what the voice interface provided for the mission rerouting task. For one system malfunction—flight control pitch fault—voice and manual interfaces demonstrated no differences for accomplishing the pitch fault reset task; for another malfunction—Global Positioning System (GPS) failure—the voice interface increased the time to complete the GPS failure task compared to the manual interface. Possible reasons for these results will be explained in the discussion section."
    Issue: data entry and programming may be difficult and time consuming (Issue #112) See Issue details
    Strength: +3
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  15.  
  16. Evidence Type: Excerpt from Experiment
    Evidence: "The results verified the hypotheses and showed that the pilots were able to designate targets more quickly using voice control coupled with the ATC than with manual control coupled with the ATC (F [1,11] = 4.79, p < .05). Further, pilots reported a decrease in workload when using the voice versus manual interface in combination with the ATC (F [1,11] = 4.73, p < .05)."
    Issue: data entry and programming may be difficult and time consuming (Issue #112) See Issue details
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  17.  
  18. Evidence Type: Excerpt from Experiment
    Evidence: "Other tasks during the navigation segment in which the voice and manual interfaces were compared included: 1) correcting system malfunctions, and 2) changing radio frequency channels to place radio calls. For changing radio frequencies—a task similar in nature to rerouting in that a string of alphanumerics was entered into the simulator, the voice interface improved pilots’ speed and accuracy to accomplish the task compared to the manual interface (F [1,11] = 7.63, p < .05). However, the improvement was less than what the voice interface provided for the mission rerouting task. For one system malfunction—flight control pitch fault—voice and manual interfaces demonstrated no differences for accomplishing the pitch fault reset task; for another malfunction—Global Positioning System (GPS) failure—the voice interface increased the time to complete the GPS failure task compared to the manual interface. Possible reasons for these results will be explained in the discussion section."
    Issue: data entry and programming may be difficult and time consuming (Issue #112) See Issue details
    Strength: -3
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  19.  
  20. Evidence Type: Excerpt from Experiment
    Evidence: "Summary. Navigation segment results showed that voice recognition significantly improved the speed and accuracy of pilot data input during re-route when compared to manual input. Weapon delivery segment results showed that pilot performance was significantly improved by integrating auto-target cueing features with voice recognition when compared to the manual, throttle-mounted switches. In fact, in all but one of the voice interface cases, pilots were able to correctly designate all six of the tanker aircraft in a single pass on the airfield at significantly greater distances from the airfield than when they used the manual interface."
    Issue: data entry and programming may be difficult and time consuming (Issue #112) See Issue details
    Strength: -3
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  21.  
  22. Evidence Type: Excerpt from Experiment
    Evidence: "Navigation Results. For the mission rerouting task, the main effect of Control Type was significant (F [1,11] = 6.77, p < .05). By using the voice interface, pilots completed the rerouting task almost 14 seconds quicker that when using the manual interface."
    Issue: data entry and programming may be difficult and time consuming (Issue #112) See Issue details
    Strength: -3
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  23.  
  24. Evidence Type: Excerpt from Experiment
    Evidence: "Figure 7 illustrates the incidence of three types of speech recognition errors for the navigation segment: insertions, deletions, and substitutions. Insertions are errors where the voice system inserts commands that the pilot did not speak. Deletions are errors where the voice system did not recognize what the pilot correctly spoke and took no action. Substitutions are errors where the voice system misrecognized what the pilot spoke and executed the wrong command. The figure illustrates the voice interface error rate during the navigation segment for nine of the twelve pilots—data from three pilots were not recorded. The figure shows voice interface error rate overall averaged 2.2%."
    Issue: programming may be susceptible to error (Issue #170) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  25.  
  26. Evidence Type: Excerpt from Experiment
    Evidence: "Figure 10 illustrates the speech recognition error rates for weapon delivery. Insertions, deletions, and substitutions (see navigation segment for descriptions of these error types) are shown for nine of the twelve pilots. The voice interface had an overall error rate during weapon delivery of 3.9%."
    Issue: programming may be susceptible to error (Issue #170) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  27.  
  28. Evidence Type: Excerpt from Experiment
    Evidence: "During the navigation segment for nine of the twelve pilots (voice recordings for data from three pilots were not available recorded due to tape machine malfunctionor tabulated), the overall voice interface error rate averaged 2.2%. Many of the errors were "single-repeat" deletion errors. These were resolved with a single repeat of the command. Most of these errors could have been avoided through more extensive voice template training, so another analysis was performed on an adjusted data set in which these errors were removed. This analysis revealed that, minus the single repeat errors, voice interface average error rate dropped to 0.6%."
    Issue: programming may be susceptible to error (Issue #170) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. See Resource details

  29.  
  30. Evidence Type: Excerpt from Experiment
    Evidence: "During the weapon delivery segment for nine of the twelve pilots, the voice interface had an overall error rate of 3.9%. When the single-repeat deletion errors were removed from the data set, the adjusted overall mean error rate dropped to 1.9%. After removing subject two’s data, the largest sources of error during weapon delivery were substitutions and deletions. Although both types of error only occurred 0.4% of the time, it can still be accounted for. Closer examination of the data showed that the substitution errors occurred when designating multiple targets."
    Issue: programming may be susceptible to error (Issue #170) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Barbato, G. (1999). Lessons learned: Integrating voice recognition and automation target cueing symbology for fighter attack. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 203-207. Columbus, OH: The Ohio State University. See Resource details

  31.  
  32. Evidence Type: Excerpt from Experiment
    Evidence: "Ultimately, the most interesting questions about these data are how many pilots successfully recovered from the runaway pitch trim down malfunction and whether the auditory warning materially contributed to safe recoveries. ... Although the small sample size precludes statistical analysis, it appears that neither mode of flight nor presence of an auditory alarm materially affected the distribution [of potential unintentional ground contacts and overspeeds]. ... Although the auditory trim malfunction warning provided an immediate cue, no detectable difference was present in performance between the two alerting groups [aural alert present for one group of pilots and not present for the other]. The fact that some pilots reported a failure to hear the [aural] warning suggests that a re-evaluation of criteria for general aviation cockpit auditory warnings may be warranted, with specific attention to noise-exposed and aging populations." (page 4-6)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Beringer, D.B. (1997). Automation Effects in General Aviation: Pilot Responses to Autopilot Failures and Alarms. See Resource details

  33.  
  34. Evidence Type: Excerpt from Experiment
    Evidence: "Attitude indicator (ADI) failure. When the attitude indicator failed, it drifted slowly to approximately 25-30 degrees right bank when the aircraft was in level flight. The result was that the autopilot attempted to follow the failed instrument, placing the aircraft in a left bank. This was not a failure of the AP system but rather, a failure of the sensor feeding data to the system and was comparatively subtle. We were particularly interested in how long pilots took to diagnose the problem. Initial diagnosis (recognition of the general problem) times ranged from 12.7 to 263 seconds (mean = 48.83; median = 34.82). Times to positively identify the failed ADI ranged from 13.83 to 264.6 seconds (mean = 58.79; median= 39.63). Regarding return of the aircraft to level flight, first crossing of zero-degrees bank required an average of 22.11 seconds (median = 21.68). Thus, as would be expected, regaining flight control preceded completed diagnosis. This was aided by the visible, albeit faint, horizon between the cloud layers. Several pilots exhibited persistence of behavior in that they continued to follow the ADI even after an initial leveling in bank. One, in fact, continued to fly in a wide circle until contacted by ATC." (page 3)
    Issue: failure assessment may be difficult (Issue #25) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Beringer, D.B. (1997). Automation Effects in General Aviation: Pilot Responses to Autopilot Failures and Alarms. In R.S. Jensen & L. Rakovan (Eds.), Proceedings of the 9th International Symposium on Aviation Psychology. Columbus, OH: The Ohio State University. See Resource details

  35.  
  36. Evidence Type: Excerpt from Experiment
    Evidence: "When asked to report on the difficulty and ease of diagnosing and recovering from autopilot failures experienced during their experimental session, our subjects unanimously agreed that runaway pitch trim was the most difficult from which to recover. The most difficult failure to diagnose was spilt across three: ADI, pitch sensor, and runaway pitch trim, with each failure receiving 27% of the votes. Pitch sensor was voted the easiest to diagnose by 46% of the subjects, with runaway pitch trim being cited by 36%. Pitch sensor was voted easiest to correct by 56% of the subjects." (page 78)
    Issue: failure assessment may be difficult (Issue #25) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: autoflight: autopilot
    Source: Beringer, D.B., & Harris, H.C., Jr. (1999). Automation in general aviation: Two studies of pilot responses to autopilot malfunctions. See Resource details

  37.  
  38. Evidence Type: Excerpt from Experiment
    Evidence: "Soft pitch (pitch sensor). The pitch-sensor failure caused a slow deviation from level pitch while the ADI continued to show correct pitch indications, simulating loss of sensor data to the autopilot. First response to this failure ranged from 330 msec to 73.7 seconds (mean = 16.62; median = 12.51). AP disconnect times ranged from 5.9 1 to 73.7 seconds (mean =24.8; median = 15.4). Although 60% of the pilots disconnected in less than 20 seconds, 33% fell between 30 and 60 seconds. This was due both to the comparative subtlety of the failure and to the ability of pilots to manually override the pitch servo without disconnecting." (page 77)
    Issue: failure assessment may be difficult (Issue #25) See Issue details
    Strength: +1
    Aircraft: unspecified
    Equipment: autoflight: autopilot
    Source: Beringer, D.B., & Harris, H.C., Jr. (1999). Automation in general aviation: Two studies of pilot responses to autopilot malfunctions. International Journal of Aviation Psychology, 9(2), 155-174. Lawrence Erlbaum Associates. See Resource details

  39.  
  40. Evidence Type: Excerpt from Experiment
    Evidence: "When asked to report on the difficulty and ease of diagnosing and recovering from autopilot failures experienced during their experimental session, our subjects unanimously agreed that runaway pitch trim was the most difficult from which to recover. The most difficult failure to diagnose was spilt across three: ADI, pitch sensor, and runaway pitch trim, with each failure receiving 27% of the votes. Pitch sensor was voted the easiest to diagnose by 46% of the subjects, with runaway pitch trim being cited by 36%. Pitch sensor was voted easiest to correct by 56% of the subjects." (page 78)
    Issue: failure recovery may be difficult (Issue #23) See Issue details
    Strength: +3
    Aircraft: unspecified
    Equipment: autoflight: autopilot
    Source: Beringer, D.B., & Harris, H.C., Jr. (1999). Automation in general aviation: Two studies of pilot responses to autopilot malfunctions. See Resource details

  41.  
  42. Evidence Type: Excerpt from Experiment
    Evidence: "All pilots understood that they could overpower the autopilot servos manually. A number were aware of the potential interaction between runaway pitch-trim and autopilot pitch attitude (elevator servo) inputs, whereby the autopilotdriven elevator servo masks the initial stage of the pitch trim excursion." (page 78)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -3
    Aircraft: unspecified
    Equipment: autoflight: autopilot
    Source: Beringer, D.B., & Harris, H.C., Jr. (1999). Automation in general aviation: Two studies of pilot responses to autopilot malfunctions. International Journal of Aviation Psychology, 9(2), 155-174. Lawrence Erlbaum Associates. See Resource details

  43.  
  44. Evidence Type: Excerpt from Experiment
    Evidence: "Pilot scan patterns varied as a function of flight control condition. In manual flight, the basic 'T' formed the nucleus of the pilots' scan patterns. This was not true, however, in the coupled flight condition. The decrease in attention to the ADI in coupled flight is similar to that reported by Spady (1977) and is consistent with the pilots' changing role from that of active controller to monitor of system performance." (page 445)
    Issue: scan pattern may change (Issue #38) See Issue details
    Strength: +1
    Aircraft: B767, B757
    Equipment: automation
    Source: Edwards, R.E., Tolin, P., & Jonsen, G.L. (1982). Pilot Visual Behavior as a Function of Navigation and Flight Control Modes in the Boeing 757/767. See Resource details

  45.  
  46. Evidence Type: Excerpt from Experiment
    Evidence: "... no consistent differences in pilot visual behavior were observed between the EICAS-equipped cab and the cab containing conventional engine instruments." (page 445)
    Issue: scan pattern may change (Issue #38) See Issue details
    Strength: -1
    Aircraft: B757, B767
    Equipment: EICAS
    Source: Edwards, R.E., Tolin, P., & Jonsen, G.L. (1982). Pilot Visual Behavior as a Function of Navigation and Flight Control Modes in the Boeing 757/767. In Proceedings of the 26th Annual Meeting of the Human Factors and Ergonomic Society, 441-445. See Resource details

  47.  
  48. Evidence Type: Excerpt from Experiment
    Evidence: "In this case, the pilots entered “R” to direct the aircraft to fly to a fix on the approach named Rozo. While “R” was the designation for Rozo indicated on the approach chart it was not the designation used for that point in the FMS database. The "R" they actually selected was assigned to another point in Columbia named Romeo. This was a central error in this accident that sent the aircraft into a 180 degree bank to the left towards Romeo. It was a simple error for the pilots to make, likely induced by the fact that “R” was the expected designation for Rozo and was presented on the charts as such. A poorly understood FMS naming convention led to the designation of R for Romeo and not Rozo in the FMS database. (Romeo was nearer to the larger airport in Columbia, Bogota, and therefore received the designator R. Thus Rozo was assigned its full name in the database.)" (page 880)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  49.  
  50. Evidence Type: Excerpt from Experiment
    Evidence: "In the Cali accident, the pilots faced the challenge of working with the FMS display which, by design, portrayed information about the location of navigational fixes but not environmental features such as terrain. The pilots entered “Direct CLO” (Direct to the Cali VOR) in response to a miscommunication with air traffic control (ATC) which led them to believe they had a clearance to proceed direct to Cali as opposed to following the usual waypoints on designated airways. Requesting and receiving a direct clearance is not uncommon in radar controlled airspace, which, based on their extensive background flying in the U.S., this aircrew was accustomed to. The action of making a direct entry into the FMS had an unfortunate side effect, however. It caused a new flight path to be presented between the aircraft’s current position and the Cali VOR (labeled CLO) and all intervening waypoints along the original path to disappear. Thus, when the aircrew received a later clearance from ATC to “report Tulua”, they could not find this waypoint (labeled ULQ) on their display or in an FMS-control device. They devoted considerable efforts in a time pressured situation in trying to find ULQ or other points on their display that corresponded to those on the new approach to runway 19. The selected display did not support the global SA needed to detect their location relevant to pertinent landmarks, nor the global SA needed to rapidly change goals (programming in a new flight path)." (page 878)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  51.  
  52. Evidence Type: Excerpt from Experiment
    Evidence: "Last minute runway assignments can create a significant problem for pilots when they necessitate reprogramming the FMS to execute and/or display the new approach... The requirement to reprogram the FMS and cross check the entries at the last minute certainly played a role in this accident." (page 880)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  53.  
  54. Evidence Type: Excerpt from Experiment
    Evidence: "In this case, the pilots entered “R” to direct the aircraft to fly to a fix on the approach named Rozo. While “R” was the designation for Rozo indicated on the approach chart it was not the designation used for that point in the FMS database. The "R" they actually selected was assigned to another point in Columbia named Romeo. This was a central error in this accident that sent the aircraft into a 180 degree bank to the left towards Romeo. It was a simple error for the pilots to make, likely induced by the fact that “R” was the expected designation for Rozo and was presented on the charts as such. A poorly understood FMS naming convention led to the designation of R for Romeo and not Rozo in the FMS database. (Romeo was nearer to the larger airport in Columbia, Bogota, and therefore received the designator R. Thus Rozo was assigned its full name in the database.)" (page 880)
    Issue: database may be erroneous or incomplete (Issue #110) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  55.  
  56. Evidence Type: Excerpt from Experiment
    Evidence: "…theCali accident demonstrated that there can be substantial differences in the points and nomenclature used between the two information sources. As a result, determining a correlation between identical points on the two different navigation sources can be both difficult and time consuming. In this accident, the points on the desired flight path were named CF19 and FF19 in the FMSgenerated data, and D21 and D16 on the map. It takes considerable calculation to determine that they actually represent the same points, and these calculations are often time-consuming." (page 879)
    Issue: database may be erroneous or incomplete (Issue #110) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  57.  
  58. Evidence Type: Excerpt from Experiment
    Evidence: "This example is also indicative of an underlying problem with the FMS display. Pilots essentially generate their own selective display of the external world, based on the commands they enter. With FMS equipped aircraft, it is possible to enter any series of waypoints and the aircraft will fly that path, however, except for flying near adverse weather, it can be very difficult to detect if the created path is potentially unsafe or incorrect…Without verifying the accuracy of the flight path by comparison with navigation charts, pilots are not able to detect these errors from simply examining the displays and such programming errors are actually fairly easy to make." (page 879)
    Issue: information integration may be required (Issue #9) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  59.  
  60. Evidence Type: Excerpt from Experiment
    Evidence: "…In a time critical situation, it appears that the flight crew trusted the automation to carry out its task (fly to the designated point “R”), as it had many times before... In the Cali accident, the fact that the pilots had become loaded with very demanding tasks that required the use of separate, non-integrated sources of information may have contributed to their lack of vigilance in monitoring the automation during the turn." (page 881)
    Issue: information integration may be required (Issue #9) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  61.  
  62. Evidence Type: Excerpt from Experiment
    Evidence: "In the Cali accident, the pilots faced the challenge of working with the FMS display which, by design, portrayed information about the location of navigational fixes but not environmental features such as terrain. The pilots entered “Direct CLO” (Direct to the Cali VOR) in response to a miscommunication with air traffic control (ATC) which led them to believe they had a clearance to proceed direct to Cali as opposed to following the usual waypoints on designated airways. Requesting and receiving a direct clearance is not uncommon in radar controlled airspace, which, based on their extensive background flying in the U.S., this aircrew was accustomed to. The action of making a direct entry into the FMS had an unfortunate side effect, however. It caused a new flight path to be presented between the aircraft’s current position and the Cali VOR (labeled CLO) and all intervening waypoints along the original path to disappear. Thus, when the aircrew received a later clearance from ATC to “report Tulua”, they could not find this waypoint (labeled ULQ) on their display or in an FMS-control device. They devoted considerable efforts in a time pressured situation in trying to find ULQ or other points on their display that corresponded to those on the new approach to runway 19. The selected display did not support the global SA needed to detect their location relevant to pertinent landmarks, nor the global SA needed to rapidly change goals (programming in a new flight path)." (page 878)
    Issue: insufficient information may be displayed (Issue #99) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  63.  
  64. Evidence Type: Excerpt from Experiment
    Evidence: "In this accident, the pilots received a clearance to Runway 19, a runway they were not expecting, and they chose to accept that clearance (possibly in an effort to land as quickly as possible or possibly due to a confirmation bias as they had previously set-up the FMS for Runway 1 per information from the company dispatcher and in accordance with previous experience into the Cali airport.) At that point they needed to find and review the necessary approach charts and perform a number of steps to program the new approach into the FMS. This process was complicated, however, by the fact that an earlier entry to go direct to the Cali VOR had removed the points from the display that were need for creating the proper path. That is, the Tulua VOR (ULQ) was no longer displayed." (page 880)
    Issue: insufficient information may be displayed (Issue #99) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  65.  
  66. Evidence Type: Excerpt from Experiment
    Evidence: "A particular deficiency of the FMS, in terms of its ability to support the SA requirements of the pilot, is its presentation of vertical information. The flight path displayed provides only the programmed lateral path of the aircraft. No direct display of either the vertical path of the aircraft nor its relationship to surrounding terrain is provided...The pilots demonstrated a lack of awareness of the proximity and altitude of surrounding terrain that would have alerted them to the danger of continuing their descent. A direct display of vertical path information and its relation to surrounding terrain is not provided by the displays. This state of affairs allowed the crew to continue their descent without questioning its advisability (at least as far as the cockpit voice recorder reveals). Particularly in light of the fact that the lateral path was so clearly and saliently displayed, the lack of salience of vertical information on the FMS was a significant factor in this accident."
    Issue: insufficient information may be displayed (Issue #99) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  67.  
  68. Evidence Type: Excerpt from Experiment
    Evidence: "Once the pilots did detect the turn, they spent the next several minutes trying to determine the nature of their navigation difficulty and then correct the problem. This incident is typical of the out-of-the-loop performance problem that has been noted to occur with automated systems. Not only did it take the pilots some time to figure out there was a problem, they also were sufficiently out-of-the-loop such that they had significant difficulty in ascertaining just how they ended up in that position (to understand what the current system state actually was) and trying to figure out how to rectify it. The pilots had significant difficulty in trying to correct the state they found themselves in and their confusion was evident." (page 881)
    Issue: pilots may be out of the loop (Issue #2) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  69.  
  70. Evidence Type: Excerpt from Experiment
    Evidence: "…In a time critical situation, it appears that the flight crew trusted the automation to carry out its task (fly to the designated point “R”), as it had many times before... In the Cali accident, the fact that the pilots had become loaded with very demanding tasks that required the use of separate, non-integrated sources of information may have contributed to their lack of vigilance in monitoring the automation during the turn." (page 881)
    Issue: pilots may be overconfident in automation (Issue #131) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  71.  
  72. Evidence Type: Excerpt from Experiment
    Evidence: "the captain appeared engaged in a state we will call automation fixation. People have grown accustomed to technology working in only fixed ways and they may routinely have to try several tactics to get it to do what they want it to do. Therefore they may engage in a persistence behavior (continuing to repeatedly try different things) which is frequently eventually successful. Engaging in this type of automation fixation may have very negative consequences, however, if the circumstances are such that a wiser course of action would be to give up and do the task in another way (e.g. fly the aircraft manually). Even after all of the problems encountered by this crew, the captain remained intent on trying to program the FMS to fly the approach path." (page 881)
    Issue: pilots may be reluctant to assume control (Issue #26) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. See Resource details

  73.  
  74. Evidence Type: Excerpt from Experiment
    Evidence: "In the Cali accident, the pilots faced the challenge of working with the FMS display which, by design, portrayed information about the location of navigational fixes but not environmental features such as terrain. The pilots entered “Direct CLO” (Direct to the Cali VOR) in response to a miscommunication with air traffic control (ATC) which led them to believe they had a clearance to proceed direct to Cali as opposed to following the usual waypoints on designated airways. Requesting and receiving a direct clearance is not uncommon in radar controlled airspace, which, based on their extensive background flying in the U.S., this aircrew was accustomed to. The action of making a direct entry into the FMS had an unfortunate side effect, however. It caused a new flight path to be presented between the aircraft’s current position and the Cali VOR (labeled CLO) and all intervening waypoints along the original path to disappear. Thus, when the aircrew received a later clearance from ATC to “report Tulua”, they could not find this waypoint (labeled ULQ) on their display or in an FMS-control device. They devoted considerable efforts in a time pressured situation in trying to find ULQ or other points on their display that corresponded to those on the new approach to runway 19. The selected display did not support the global SA needed to detect their location relevant to pertinent landmarks, nor the global SA needed to rapidly change goals (programming in a new flight path)." (page 879)
    Issue: situation awareness may be reduced (Issue #114) See Issue details
    Strength: +1
    Aircraft: B757
    Equipment: automation and FMS
    Source: Inagaki, T., Takae, Y., & Moray, N. (1999). Automation and human interface for takeoff safety. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 402-407. Columbus, OH: The Ohio State University. See Resource details

  75.  
  76. Evidence Type: Excerpt from Experiment
    Evidence: "During the debriefing, all the pilots felt the presentation of Flight Path Angle and Vertical Speed Modes could be improved. In free responses, six pilots stated that the mode annunciations should be made more distinct and identifiable. Three pilots stated the selector for these two modes should be physically separated. One suggestion was to use a different color to highlight the 'non-normal' mode, although no opinion was given about what should be the 'normal' mode." (page 18)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +5
    Aircraft: A320
    Equipment: autoflight
    Source: Johnson, E.N. & Pritchett, A.R. (1995). Experimental Study Of Vertical Flight Path Mode Awareness. See Resource details

  77.  
  78. Evidence Type: Excerpt from Experiment
    Evidence: "Conclusions ... The display cues cited by the pilots and the instruments in their scan suggest study of some changes in mode presentation and pilot training. To monitor autopilot conformance, pilots must compare between mode annunciations, commanded values selected on the mode control panel, and the aircraft states shown on their Primary Flight Displays and HUD. This requires the pilot to reference several displays and compare between displays in different formats on different screens, sometimes referencing states that are not distinctly quantified (such as Flight Path Angle)." (page 21)
    Issue: information integration may be required (Issue #9) See Issue details
    Strength: +1
    Aircraft: A320
    Equipment: automation: displays
    Source: Johnson, E.N. & Pritchett, A.R. (1995). Experimental Study Of Vertical Flight Path Mode Awareness. Cambridge, MA: Massachusetts Institute of Technology, Department of Aeronautics and Astronautics. See Resource details

  79.  
  80. Evidence Type: Excerpt from Experiment
    Evidence: "Six pilots [out of a total of 12, 50%] suggested mode annunciation or graphical cues on the HUD, although three [25%] also expressed concerns about cluttering the HUD and information overload." (page 18)
    Issue: insufficient information may be displayed (Issue #99) See Issue details
    Strength: +3
    Aircraft: A320
    Equipment: automation
    Source: Johnson, E.N. & Pritchett, A.R. (1995). Experimental Study Of Vertical Flight Path Mode Awareness. See Resource details

  81.  
  82. Evidence Type: Excerpt from Experiment
    Evidence: "Two pilots [out of a total of 12, 17%] suggested aural alerts for 'stupid' mode selections. One pilot suggested changes in the procedures used for selecting modes, such as calling out the mode and commanded state value, with a response from the pilot-not-flying." (page 18)
    Issue: insufficient information may be displayed (Issue #99) See Issue details
    Strength: +1
    Aircraft: A310-304
    Equipment: autoflight
    Source: Johnson, E.N. & Pritchett, A.R. (1995). Experimental Study Of Vertical Flight Path Mode Awareness. Cambridge, MA: Massachusetts Institute of Technology, Department of Aeronautics and Astronautics. See Resource details

  83.  
  84. Evidence Type: Excerpt from Experiment
    Evidence: "Conclusions... Most pilots showed a lack of awareness of the commanded descent mode and were confused by the resulting aircraft states. All but one of the subjects [11/12 = 92%] allowed the aircraft to deviate significantly from the intended glide path, with ten pilots allowing the aircraft to reach altitudes where ground impact either happened or would be difficult to avoid. This indicates that pilots had a serious lack of autopilot mode and aircraft state awareness when given the displays used in the study." (page 20)
    Issue: mode awareness may be lacking (Issue #95) See Issue details
    Strength: +5
    Aircraft: A320
    Equipment: autoflight
    Source: Johnson, E.N. & Pritchett, A.R. (1995). Experimental Study Of Vertical Flight Path Mode Awareness. See Resource details

  85.  
  86. Evidence Type: Excerpt from Experiment
    Evidence: "Conclusions ... Improvements in training and procedures were also suggested by some of the pilots. For example, pilots frequently called out altitudes, DME distances and vertical speeds, but did not have a standard protocol for cross-checking these values with those selected on the Mode Control Panel." (page 21)
    Issue: training may be inadequate (Issue #133) See Issue details
    Strength: +1
    Aircraft: A320
    Equipment: automation
    Source: Johnson, E.N. & Pritchett, A.R. (1995). Experimental Study Of Vertical Flight Path Mode Awareness. Cambridge, MA: Massachusetts Institute of Technology, Department of Aeronautics and Astronautics. See Resource details

  87.  
  88. Evidence Type: Excerpt from Experiment
    Evidence: A repeated-measures multivariate analysis of variance (MANOVA) was conducted on the detection data with feedback type as the between-subjects factor and two within-subject factors, concurrent task load and simultaneity. A significant effect was seen for feedback type, F(2, 18) = 17.76, p < ,001. Pilots receiving visual-only feedback detected approximately 83% of the unexpected mode transitions that occurred throughout the flight, whereas pilots in the other two conditions (tactile-visual and tactile only) detected close to 100% of all changes in mode status. (page 546)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +4
    Aircraft: unspecified
    Equipment: automation: displays
    Source: Lin, H.X. & Salvendy, G. (2000). Warning effect on human error reduction. See Resource details

  89.  
  90. Evidence Type: Excerpt from Experiment
    Evidence: "The analysis did not yield a main effect for flight phase but did show a significant interaction between feedback condition and phase of flight, F(6, 54) = 2.91, p < .05. As illustrated in Figure 3, both tactile conditions yielded nearperfect detection rates in each phase of flight, whereas performance in the visual-only condition was markedly affected by the concurrent demands in the A/P Dynamic phase. An additional analysis conducted on the two tactile conditions alone confirmed that detection of tactile cues was not affected by concurrent load." (page 547)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation: displays
    Source: Lin, H.X. & Salvendy, G. (2000). Warning effect on human error reduction. International Journal of Cognitive Ergonomics, 4(2), 145-161. See Resource details

  91.  
  92. Evidence Type: Excerpt from Experiment
    Evidence: "A repeated-measures MANOVA on the detection rates for traffic conflicts and engine deviations did not yield a significant main effect for feedback condition. Significant main effects were found. however, for both simultaneity, F( 1, 18) = 5.86, p < .05, and for flight phase, F(3, 54) = 6.36, p < .Ol. Traffic and engine deviations that were presented individually were more likely to be detected than those presented simultaneously with a mode transition (see Figure 5)." (page 548)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation: displays
    Source: Lin, H.X. & Salvendy, G. (2000). Warning effect on human error reduction. See Resource details

  93.  
  94. Evidence Type: Excerpt from Experiment
    Evidence: "Based on previous research and in light of the issues associated with data link above, several experimental hypotheses were generated and tested in our simulation. ... Procedure: Within the data link condition, the pilot not flying will be more likely to use the review message log than the pilot flying following the traditional roles in cockpit procedure. ... The hypothesis that the PNF would use the review menu more than the PF was supported. Based on the total number of times the review menu was accessed, the PNF used the review menu 84% of the time." (page 1010, 1012)
    Issue: pilot control authority may be diffused (Issue #104) See Issue details
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Lozito, S., McGann, A., & Corker, K. (Undated). Data link air traffic control and flight deck environments: Experiment in flight crew performance. See Resource details

  95.  
  96. Evidence Type: Excerpt from Experiment
    Evidence: "RESULTS ... Commission Errors All of the pilots (N = 21) who experienced the false engine fire message did ultimately shut down the engine. This was contrary to responses on the debriefing questionnaire indicating that an EICAS message without other cues would not be sufficient to diagnose 'definitely a fire,' and that it would be safer, in the presence of only an EICAS message, to retard the throttle of the indicated engine and complete the go-around procedure with the engine running rather than to shut down the suspect engine." (page 58)
    Issue: pilots may over-rely on automation (Issue #106) See Issue details
    Strength: +5
    Aircraft: unspecified
    Equipment:
    Source: Mosier, K.L., Skitka, L.J., Heers, S., & Burdick, M. (1997). Automation bias: Decision making and performance in high-tech cockpits. See Resource details

  97.  
  98. Evidence Type: Excerpt from Experiment
    Evidence: "RESULTS Omission Error Events ... Descriptive analyses revealed overall omission rates for flight-related events of approximately 55% ... The altitude load failure and the heading capture failure, the two events arguably most critical to aircraft operation safety, remained undetected by 44% and 48% of the participants respectively. The frequency misload was undetected by 71% of pilot participants. Only three pilots detected all three flight-related events; five pilots failed to detect any of the three flight-related events." (page 57-58)
    Issue: pilots may over-rely on automation (Issue #106) See Issue details
    Strength: +3
    Aircraft: unspecified
    Equipment:
    Source: Mosier, K.L., Skitka, L.J., Heers, S., & Burdick, M. (1997). Automation bias: Decision making and performance in high-tech cockpits. International Journal of Aviation Psychology, 8(1), 47-63. Lawrence Erlbaum Associates. See Resource details

  99.  
  100. Evidence Type: Excerpt from Experiment
    Evidence: We collected detailed performance data on automation use during the scenario and found several areas in which pilots were not proficient. One notable finding is that there was considerable confusion about the altitude intervention button (which provides very poor feedback). No pilot used this button correctly in all cases. (page 5)
    Issue: behavior of automation may not be apparent (Issue #83) See Issue details
    Strength: +5
    Aircraft: B747-400
    Equipment: automation: controls
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. See Resource details

  101.  
  102. Evidence Type: Excerpt from Experiment
    Evidence: The data showed that pilots often “failed” to fixate the FMA within the first 10 seconds (during which time a green box appears to highlight it). The percentages of cases in which pilots failed to fixate in that first 10 seconds were as follows: M = 53%, AE = 45%, AU = 62%. In other cases, pilots fixated near the time or near the FMA. However, if we examine (more liberally) failures to fixate any FMAs in the first 20 seconds after the green box appears, the failure rates are still high: M = 32%, AE = 29%, AU = 40%. Thus, for a considerable percent of cases pilots do not verify the FMA change, and further, Unexpected mode changes are verified less frequently than the Manual or Automatic-Expected mode changes. This failure suggests that the attention-attracting properties of the green box that accompanies every mode change may be an insufficient cue (Nikolic, Orr, & Sarter, 2001). (page 5)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +2
    Aircraft: B747-400
    Equipment: automation & FMS
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. In Proceedings of the 11th International Symposium on Aviation Psychology. Columbus, OH: The Ohio State University.. See Resource details

  103.  
  104. Evidence Type: Excerpt from Experiment
    Evidence: The data showed that pilots often “failed” to fixate the FMA within the first 10 seconds (during which time a green box appears to highlight it). The percentages of cases in which pilots failed to fixate in that first 10 seconds were as follows: M = 53%, AE = 45%, AU = 62%. In other cases, pilots fixated near the time or near the FMA. However, if we examine (more liberally) failures to fixate any FMAs in the first 20 seconds after the green box appears, the failure rates are still high: M = 32%, AE = 29%, AU = 40%. Thus, for a considerable percent of cases pilots do not verify the FMA change, and further, Unexpected mode changes are verified less frequently than the Manual or Automatic-Expected mode changes. This failure suggests that the attention-attracting properties of the green box that accompanies every mode change may be an insufficient cue (Nikolic, Orr, & Sarter, 2001). (page 5)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +1
    Aircraft: B747-400
    Equipment: automation & FMS
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. See Resource details

  105.  
  106. Evidence Type: Excerpt from Experiment
    Evidence: At the conclusion of the simulator session we interviewed each pilot to assess his understanding of various automation concepts. Through these interviews, we found that pilots typically could state correct expectations about common mode behavior. For example, 16 of 20 pilots indicated that they expected to see VNAV PTH as the pitch mode during cruise. However, few pilots applied this knowledge effectively during the simulator session, where they flew cruise in VNAV ALT. Also, although pilots were generally correct in the information they offered, they provided little information on subtler automation features. Detailed knowledge of VNAV SPD and VNAV ALT, in particular, was not offered. Note, however, that we recorded only what pilots volunteered, and it is possible that pilots knew these details but chose not to offer them in this setting. (page 5)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B747-400
    Equipment: FMS VNAV
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. In Proceedings of the 11th International Symposium on Aviation Psychology. Columbus, OH: The Ohio State University.. See Resource details

  107.  
  108. Evidence Type: Excerpt from Experiment
    Evidence: In three separate Events (3, 7, 8), we changed the FMA artificially (and unknown to the pilot; in fact, for these cases the standard green box did not accompany the mode change). Table 3 shows the three cases where a change occurred. The second column indicates how many pilots fixated the relevant FMA while that change was in effect (note that data were not available for all pilots at this level of precision). The last column shows that in only 1 of the 32 total cases did a pilot notice that the FMA was inappropriate. Thus, even when scanning included the FMA, pilots failed to understand the implications of the FMA. (page 5)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +1
    Aircraft: B747-400
    Equipment: automation & FMS
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. See Resource details

  109.  
  110. Evidence Type: Excerpt from Experiment
    Evidence: In the plan monitoring phase, we wished to first examine the effect of highlighting on change detection and did so by comparing performance in detecting changes to hazards that were highlighted to those that were not highlighted as a function of workload. The analyses revealed that changes to highlighted hazards were detected more accurately (F(1, 26) = 27.72, p < .001) and more quickly (F(1, 22) = 4.47, p = .05) than changes to nonhighlighted elements. Additionally, a speed accuracy tradeoff was found for workload, such that changes in the low workload condition were detected 36.0% more accurately (F(1, 26) = 7.68, p = .01), but 6.7 s more slowly (F(1, 22) = 8.88, p = .007) than those in the high workload condition. The automation and workload interaction was not significant for accuracy (F(1, 26) = 1.75, p > .10) or response time (F(1, 22) = .27, p > .10). (page 4/5)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (2003). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. In Proceedings of the 12th International Symposium on Aviation Psychology, 857-62. See Resource details

  111.  
  112. Evidence Type: Excerpt from Experiment
    Evidence: Automated aids were implemented on one half of trials to assist with the plan selection process. A marginally significant main effect was found for automation on plan selection accuracy (F(1, 15) = 3.75, p = 0.07), such that accuracy was 19.1% higher in trials with attention guidance automation (M = 78.1%), relative to the baseline condition (M = 65.6%), though automation had no significant effect on response time (F(1, 83) = 1.25, p > 0.10). The presence of automation also significantly increased confidence by 10.2% (M = 5.4, F(1, 15) = 7.16, p = 0.02), relative to the baseline condition (M = 4.9). For the measures of accuracy, response time, and confidence, no significant interaction was found for plan selection difficulty and automation, F(3, 45) = 1.63, p > 0.10; F(3, 83) = 1.09, p > 0.10; and F(3, 45) = 0.55, p > 0. 10, respectively. (page 31/32)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. See Resource details

  113.  
  114. Evidence Type: Excerpt from Experiment
    Evidence: In comparing the prevalence of plan continuation errors under the automated condition relative to baseline condition, an analysis revealed that a greater percentage of the trials in the automated condition were comprised of plan continuation errors (7 of 16 or 43.7%), than in the baseline condition (3 of 16 or 18.7%). A chi-squared test performed on these data revealed a nonsignificant trend, Χ2(1) = 2.3 3, p = 0. 13, showing that plan continuation errors were more likely under the automated condition than the baseline condition. This finding is analogous to that described above in which accuracy at selecting a flight plan at the second choice point was lower, due to the pilot’s inability to detect an automation failure, with automation than without the decision aid. (page 40)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. Nasa Technical Report AFHD-02-11/NASA-02-8. Moffett Field, CA: NASA Ames Research Center. See Resource details

  115.  
  116. Evidence Type: Excerpt from Experiment
    Evidence: The pairwise t-tests revealed that pilots selected flight plans at the second choice point more quickly (M = 3.16 s, t(4) = 2.19, p = 0.09, marginally significant) as well as somewhat more confidently (M = 6.06) in the automated condition relative to the baseline condition (M = 5.96 s, 5.3 8, t(15) = 2.55, p = 0.02). For the dependent variable of accuracy, however, pilots were less accurate in the automated (M = 43.7%) than in the baseline condition (M = 75.0%), t(15)= 1.78, p = 0.09, marginally significant). Recall that it is at the second choice point in this mediumML→H plan selection condition under the attention guidance automation that a change occurs to a now important and high risk (but non-highlighted) hazard. The lowered accuracy under the automated condition reflects the pilots’ failure to notice the change in the automated condition and therefore their complacency in detecting the consequences of this automation failure. (page 40)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. See Resource details

  117.  
  118. Evidence Type: Excerpt from Experiment
    Evidence: On one half of trials, attention guidance automation was provided to aid the participants in selecting a flight path by highlighting the hazards that presented the most risk to the flight plans. Following the plan selection stage, pilots were asked to monitor the airspace for changes to hazards that could be either highlighted or non-highlighted as a function of their degrees of relevance for the prior choice. The present study sought to assess change detection performance as a function of automation by conducting pairwise t-tests for both accuracy and response time. This analysis included change detection performance for only the automated condition, and did not examine performance in the baseline condition, because in the baseline condition all hazards appeared at the same luminance level. Results revealed that pilots were significantly more accurate in detecting changes to elements that were highlighted (M = 45.1%) than non-highlighted (M = 14.8%), t(15) = 4.78, p < 0.001. No significant difference was found for response time, t(12) = 0.58, p > 0.10. (page 35)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation: displays
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. Nasa Technical Report AFHD-02-11/NASA-02-8. Moffett Field, CA: NASA Ames Research Center. See Resource details

  119.  
  120. Evidence Type: Excerpt from Experiment
    Evidence: To assess the effects of workload and automation on plan selection performance, we examined plan selection accuracy, response time, and choice confidence in three ANOVAs. The main effect of workload was not significant for accuracy (F(1, 27) = 1.21, p > .10), response time (F(1, 26) = 1.41, p > .10), or confidence (F(1, 27) = 1.93, p > .10). The automation main effect was also not significant for accuracy (F(1, 27) = 1.21, p > .10), response time (F(1, 26) = .94, p > .10), but was significant for the confidence dependent variable (F(1, 27) = 8.36, p = .01), suggesting that pilots were more confident in plan selection with the automated aid than without. (page 4)
    Issue: automation use may slow pilot responses (Issue #161) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (2003). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. See Resource details

  121.  
  122. Evidence Type: Excerpt from Experiment
    Evidence: Automated aids were implemented on one half of trials to assist with the plan selection process. A marginally significant main effect was found for automation on plan selection accuracy (F(1, 15) = 3.75, p = 0.07), such that accuracy was 19.1% higher in trials with attention guidance automation (M = 78.1%), relative to the baseline condition (M = 65.6%), though automation had no significant effect on response time (F(1, 83) = 1.25, p > 0.10). The presence of automation also significantly increased confidence by 10.2% (M = 5.4, F(1, 15) = 7.16, p = 0.02), relative to the baseline condition (M = 4.9). For the measures of accuracy, response time, and confidence, no significant interaction was found for plan selection difficulty and automation, F(3, 45) = 1.63, p > 0.10; F(3, 83) = 1.09, p > 0.10; and F(3, 45) = 0.55, p > 0. 10, respectively. (page 32)
    Issue: automation use may slow pilot responses (Issue #161) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. Nasa Technical Report AFHD-02-11/NASA-02-8. Moffett Field, CA: NASA Ames Research Center. See Resource details

  123.  
  124. Evidence Type: Excerpt from Experiment
    Evidence: On one half of trials, attention guidance automation was provided to aid the participants in selecting a flight path by highlighting the hazards that presented the most risk to the flight plans. Following the plan selection stage, pilots were asked to monitor the airspace for changes to hazards that could be either highlighted or non-highlighted as a function of their degrees of relevance for the prior choice. The present study sought to assess change detection performance as a function of automation by conducting pairwise t-tests for both accuracy and response time. This analysis included change detection performance for only the automated condition, and did not examine performance in the baseline condition, because in the baseline condition all hazards appeared at the same luminance level. Results revealed that pilots were significantly more accurate in detecting changes to elements that were highlighted (M = 45.1%) than non-highlighted (M = 14.8%), t(15) = 4.78, p < 0.001. No significant difference was found for response time, t(12) = 0.58, p > 0.10. (page 35)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation: displays
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. See Resource details

  125.  
  126. Evidence Type: Excerpt from Experiment
    Evidence: In the plan monitoring phase, we wished to first examine the effect of highlighting on change detection and did so by comparing performance in detecting changes to hazards that were highlighted to those that were not highlighted as a function of workload. The analyses revealed that changes to highlighted hazards were detected more accurately (F(1, 26) = 27.72, p < .001) and more quickly (F(1, 22) = 4.47, p = .05) than changes to nonhighlighted elements. Additionally, a speed accuracy tradeoff was found for workload, such that changes in the low workload condition were detected 36.0% more accurately (F(1, 26) = 7.68, p = .01), but 6.7 s more slowly (F(1, 22) = 8.88, p = .007) than those in the high workload condition. The automation and workload interaction was not significant for accuracy (F(1, 26) = 1.75, p > .10) or response time (F(1, 22) = .27, p > .10). (page 4/5)
    Issue: displays (visual and aural) may be poorly designed (Issue #92) See Issue details
    Strength: +2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (2003). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. In Proceedings of the 12th International Symposium on Aviation Psychology, 857-62. See Resource details

  127.  
  128. Evidence Type: Excerpt from Experiment
    Evidence: To assess the effects of workload and automation on plan selection performance, we examined plan selection accuracy, response time, and choice confidence in three ANOVAs. The main effect of workload was not significant for accuracy (F(1, 27) = 1.21, p > .10), response time (F(1, 26) = 1.41, p > .10), or confidence (F(1, 27) = 1.93, p > .10). The automation main effect was also not significant for accuracy (F(1, 27) = 1.21, p > .10), response time (F(1, 26) = .94, p > .10), but was significant for the confidence dependent variable (F(1, 27) = 8.36, p = .01), suggesting that pilots were more confident in plan selection with the automated aid than without. (page 4/5)
    Issue: pilots may lack confidence in automation (Issue #46) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (2003). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. See Resource details

  129.  
  130. Evidence Type: Excerpt from Experiment
    Evidence: The pairwise t-tests revealed that pilots selected flight plans at the second choice point more quickly (M = 3.16 s, t(4) = 2.19, p = 0.09, marginally significant) as well as somewhat more confidently (M = 6.06) in the automated condition relative to the baseline condition (M = 5.96 s, 5.3 8, t(15) = 2.55, p = 0.02). For the dependent variable of accuracy, however, pilots were less accurate in the automated (M = 43.7%) than in the baseline condition (M = 75.0%), t(15)= 1.78, p = 0.09, marginally significant). Recall that it is at the second choice point in this mediumML→H plan selection condition under the attention guidance automation that a change occurs to a now important and high risk (but non-highlighted) hazard. The lowered accuracy under the automated condition reflects the pilots’ failure to notice the change in the automated condition and therefore their complacency in detecting the consequences of this automation failure. (page 40)
    Issue: pilots may lack confidence in automation (Issue #46) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. Nasa Technical Report AFHD-02-11/NASA-02-8. Moffett Field, CA: NASA Ames Research Center. See Resource details

  131.  
  132. Evidence Type: Excerpt from Experiment
    Evidence: Automated aids were implemented on one half of trials to assist with the plan selection process. A marginally significant main effect was found for automation on plan selection accuracy (F(1, 15) = 3.75, p = 0.07), such that accuracy was 19.1% higher in trials with attention guidance automation (M = 78.1%), relative to the baseline condition (M = 65.6%), though automation had no significant effect on response time (F(1, 83) = 1.25, p > 0.10). The presence of automation also significantly increased confidence by 10.2% (M = 5.4, F(1, 15) = 7.16, p = 0.02), relative to the baseline condition (M = 4.9). For the measures of accuracy, response time, and confidence, no significant interaction was found for plan selection difficulty and automation, F(3, 45) = 1.63, p > 0.10; F(3, 83) = 1.09, p > 0.10; and F(3, 45) = 0.55, p > 0. 10, respectively. (page 32)
    Issue: pilots may lack confidence in automation (Issue #46) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (August 2002). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. See Resource details

  133.  
  134. Evidence Type: Excerpt from Experiment
    Evidence: When pilots were presented with an automated aid, both accuracy (t(15) = 1.94, p = .07, marginally significant) and confidence (t(15) = 2.67, p = .02, significant) were higher with automation than without the aid, though these differences were only observed for the high workload condition, as shown in Table 1. (page 4/5)
    Issue: pilots may lack confidence in automation (Issue #46) See Issue details
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Muthard, E.K. & Wickens, C.D. (2003). Factors That Mediate Flight Plan Monitoring and Errors in Plan Revision: An Examination of Planning Under Automated Conditions. In Proceedings of the 12th International Symposium on Aviation Psychology, 857-62. See Resource details

  135.  
  136. Evidence Type: Excerpt from Experiment
    Evidence: "Results of this study do not support statements which speculate a decreasing trend of communication with increasing levels of automation (Poducal, 1987; Speyer and Fort, 1987; Segal, 1989; and Maurino, 1991)." (page 218)
    Issue: inter-pilot communication may be reduced (Issue #139) See Issue details
    Strength: -5
    Aircraft: B737
    Equipment: automation
    Source: Petridis, R.S., Lyall, E.A., & Robideau, R.L. (1995). The effects of flight deck automation on verbal flight-relevant communication. See Resource details

  137.  
  138. Evidence Type: Excerpt from Experiment
    Evidence: In the section titled "Other Pilot Comments" the authors noted that 4 pilots [33.3%] had "concerns about cluttering the primary displays, including the HUD, and about overwhelming the pilot with information". (page 6)
    Issue: information overload may exist (Issue #14) See Issue details
    Strength: +2
    Aircraft: A320
    Equipment: automation: displays
    Source: Pritchett, A.R. & Johnson, E.N. (1994). Vertical flight path mode awareness experimental study. Paper presented at the Training for Automation Workshop, NASA Ames Research Center, 24-25 Aug 1994. See Resource details

  139.  
  140. Evidence Type: Excerpt from Experiment
    Evidence: Cockpit Workload Reporters cited cockpit workload on SIDs and STARS as a factor in 44 percent of reports. The most commonly noted workload issues are shown in Table 2. @I “I tried unseuccessfully to enter the restriction in the FMS. After three attempts, the Captain tried unsuccessfully and tried to explain why it wouldn’t take. Meanwhile, no descent was started.. . We are flying an airplane, not a computer. My focus on the FMS got in the way of doing a very simple descent profile. I will be focusing on flying first, programming second. " (# 259889) (page 14)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +2
    Aircraft: various
    Equipment: FMS & ATC
    Source: Riley, V., Lyall, E., & Wiener, E. (1993). Analytic Methods for Flight-Deck Automation Design and Evaluation, Phase Two Report: Pilot Use of Automation. See Resource details

  141.  
  142. Evidence Type: Excerpt from Experiment
    Evidence: The following is part of Table 2 Table 2 - Cockpit Workload Issues FMS Programming (automation issues) Citations 18 Percent 24% (page 14)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +1
    Aircraft: various
    Equipment: automation & FMS
    Source: Riley, V., Lyall, E., & Wiener, E. (1993). Analytic Methods for Flight-Deck Automation Design and Evaluation, Phase Two Report: Pilot Use of Automation. FAA Contract Number DTFA01-91-C-0039. See Resource details

  143.  
  144. Evidence Type: Excerpt from Experiment
    Evidence: "There were slightly more (61 percent) advanced cockpit (EFIS and/or NAV control) than traditional cockpit aircraft in the data set...It was expected that advanced cock-pit aircraft would be more likely to be involved in crossing restriction alti-tude deviations due to the greater complexity in programming descents and descent crossing fixes. While we did see this pattern, the difference in numbers between advanced and tradi-tional cockpit aircraft was not large." (page 13)
    Issue: data entry and programming may be difficult and time consuming (Issue #112) See Issue details
    Strength: +3
    Aircraft: various
    Equipment: FMS & ATC
    Source: Riley, V., Lyall, E., & Wiener, E. (1993). Analytic Methods for Flight-Deck Automation Design and Evaluation, Phase Two Report: Pilot Use of Automation. See Resource details

  145.  
  146. Evidence Type: Excerpt from Experiment
    Evidence: "The pilots in Experiment Three ended up using the automation an average of 34% more that the students over the entire timeline, and roughly a third of the pilots did not turn the automation off during the failure periods. The dynamic characteristics of the pilots' automation use profile were very similar to those of the students' profile, showing no effect from workload but significant effects from uncertainty and automation reliability. Pilot automation strategies showed somewhat less of a clear pattern based on stated influences than the students' did. Again, no significant correlation was found between manual performance and manual gambles, and there were no significant differences between the two groups in Complacency-Potential Scale scores or measures of risk taking. These conclusions suggest that pilot experience with automation biases them in favor of automatic control, in contrast with the students who showed a bias in favor of manual control." (page 115)
    Issue: pilots may over-rely on automation (Issue #106) See Issue details
    Strength: +1
    Aircraft: B737-300
    Equipment: automation
    Source: Riley, V.A. (1994). Human use of automation. Unpublished doctoral dissertation, University of Minnesota Department of Psychology. See Resource details

  147.  
  148. Evidence Type: Excerpt from Experiment
    Evidence: "In comparing workload between old and new aircraft types, it is worth noting the substantial anecdotal evidence from first officers reconverting to the 737[-200] after being promoted to the left seat. (One-on-one interviews with 12 pilots provided more detailed evidence.) They report having more difficulty with instrument scan, speed control and situational awareness, and that their overall workload is much greater." (page 3)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: B767
    Equipment: automation
    Source: Roscoe, A.H. (April, 1992). Workload in the Glass Cockpit. See Resource details

  149.  
  150. Evidence Type: Excerpt from Experiment
    Evidence: "Results from a total of 73 flight sectors strongly supported numerous anecdotal reports from company pilots that levels of workload in the 767 are almost always noticeably lower than in the 737[-200]." (page 2)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: B767
    Equipment: automation
    Source: Roscoe, A.H. (April, 1992). Workload in the Glass Cockpit. Flight Safety Digest, 1-8. See Resource details

  151.  
  152. Evidence Type: Excerpt from Experiment
    Evidence: "In comparing workload between old and new aircraft types, it is worth noting the substantial anecdotal evidence from first officers reconverting to the 737 after being promoted to the left seat. (One-on-one interviews with 12 pilots provided more detailed evidence.) They report having more difficulty with instrument scan, speed control and situational awareness, and that their overall workload is much greater." (page 3)
    Issue: scan pattern may change (Issue #38) See Issue details
    Strength: +1
    Aircraft: B767
    Equipment: automation
    Source: Roscoe, A.H. (April, 1992). Workload in the Glass Cockpit. See Resource details

  153.  
  154. Evidence Type: Excerpt from Experiment
    Evidence: "In comparing workload between old and new aircraft types, it is worth noting the substantial anecdotal evidence from first officers reconverting to the 737 after being promoted to the left seat. (One-on-one interviews with 12 pilots provided more detailed evidence.) They report having more difficulty with instrument scan, speed control and situational awareness, and that their overall workload is much greater." (page 3)
    Issue: situation awareness may be reduced (Issue #114) See Issue details
    Strength: +1
    Aircraft: B767
    Equipment: automation
    Source: Roscoe, A.H. (April, 1992). Workload in the Glass Cockpit. Flight Safety Digest, 1-8. See Resource details

  155.  
  156. Evidence Type: Excerpt from Experiment
    Evidence: "After G/S capture, a G/S signal loss was simulated at approximately 3,000 ft. ... Although detection time was not measured for this failure, it was observed that it took some pilots a rather long time (in some cases, several minutes) to even realize the problem although they were looking directly at the ADI (with the G/S indications and FD bars disappearing) during this phase of flight." (page 17)
    Issue: failure assessment may be difficult (Issue #25) See Issue details
    Strength: +1
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  157.  
  158. Evidence Type: Excerpt from Experiment
    Evidence: "Further problems with mode awareness were observed in the context of those scenario events that made it difficult but not impossible to anticipate system behavior. In the case of the go-around below 100ft AGL, all pilots failed to anticipate and realize that the autothrust system did not arm when they selected TOGA power. In this case, the pilots had an expectation of system behavior in response to their input. As mentioned by pilots in the debriefing, they all expected the autothrust system to arm because it does so in all other cases where TOGA power is applied." (page 61)
    Issue: mode awareness may be lacking (Issue #95) See Issue details
    Strength: +5
    Aircraft: A320
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1995). Strong, Silent, and Out-of-the-loop: Properties of Advanced (Cockpit) Automation and Their Impact on Human-Automation Interaction. CSEL Report 95-TR-01. See Resource details

  159.  
  160. Evidence Type: Excerpt from Experiment
    Evidence: "Another interesting result refers to failures to engage or re-engage a mode after entering (new) target values into either the MCP or the CDU. This omission occurred at least once during the scenario for 5 of the 6 transitioning pilots (the total number of omissions for this group was 9). Only two of the 14 [14 %] experienced pilots forgot to engage an appropriate mode, and this occurred only once for each of them. The problem occurred four times in regard to the LNAV mode, six times with respect to the VNAV mode and once concerning the LVL CHG mode." [7 of 20 pilots = 35% made an omission] (page 18-20)
    Issue: mode selection may be incorrect (Issue #145) See Issue details
    Strength: +2
    Aircraft: B737-300
    Equipment: FMS & autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  161.  
  162. Evidence Type: Excerpt from Experiment
    Evidence: "A fairly large number of pilots revealed a lack of system awareness in the context of the following scenario events and probes: a) the NDB faliure, b) the runway change, c) the expedited climb, and d) the go-around situation below 100 ft AGL. Eleven pilots [out of a total of eighteen, 61%] never realized the loss of the NDB signal which normally provides lateral guidance to the automation." (page 60)
    Issue: situation awareness may be reduced (Issue #114) See Issue details
    Strength: +3
    Aircraft: A320
    Equipment: automation
    Source: Sarter, N.B. & Woods, D.D. (1995). Strong, Silent, and Out-of-the-loop: Properties of Advanced (Cockpit) Automation and Their Impact on Human-Automation Interaction. CSEL Report 95-TR-01. See Resource details

  163.  
  164. Evidence Type: Excerpt from Experiment
    Evidence: "When asked to disengage the APPR mode after localizer and glideslope had been captured, only 3 pilots (15%) could recall the three ways of accomplishing this ... Seven pilots (35%) did not know of any procedure for disengaging the APPR mode." (page 16)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight: autopilot
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  165.  
  166. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked about the consequences of using an excessive vertical rate of climb in the V/S mode, none of the transitioning pilots could provide the correct answer, as compared to only 5 (35%) of the experienced participants [i.e., (6 + 9)/20 = 75% didn't know]." (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  167.  
  168. Evidence Type: Excerpt from Experiment
    Evidence: Pilots were asked for their expectations concerning ADI mode indications throughout the takeoff roll ... N1 ... and THR HOLD ... Five of the pilots (25%) expected to see both these indications. Twelve subjects (60%) only mentioned either THR HOLD (15% of the pilots) or N1 (45% of the pilots) as an indications during takeoff. Another 3 pilots (15%) could not predict any of the mode indications. (page 15)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  169.  
  170. Evidence Type: Excerpt from Experiment
    Evidence: Immediately before receiving their takeoff clearance, pilots were asked what procedure they would use to abort the takeoff at 40 kts. Although it was emphasized that the takeoff had to be aborted at 40 kts -- before Throttle Hold (THR HLD) is reached at 64 kts, when the pilot can manually position the throttles -- 16 pilots (80%) described the procedure as "Throttles back, reversers, and manual brakes," They did not mention that the autothrottles would have to be disconnected to prevent the throttles from coming back up again after manual intervention. (page 15)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  171.  
  172. Evidence Type: Excerpt from Experiment
    Evidence: "After G/S capture, a G/S signal loss was simulated at approximately 3,000 ft ... pilots were asked about the consequences of this event, and 54% of the pilots provided the correct answer. When asked whether a G/S failure at a lower altitude (<1,500 ft) would have different effects, only 15% of the pilots knew the answer. [85% did not know the answer.] Twenty-three percent of the participants did not know the answer to either question." (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  173.  
  174. Evidence Type: Excerpt from Experiment
    Evidence: "The GA mode becomes available when descending below 2,000 ft radio altitude with autothrottles armed. Out of 20 pilots, only 5 [25%] recalled the altitude at which this occurs. Eight pilots (40%) knew that the availability of the mode depends on reaching a certain altitude, but they did not remember the actual height." (page 16)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  175.  
  176. Evidence Type: Excerpt from Experiment
    Evidence: "5 of the 6 [83%] pilots without line experience could not describe how to program an intermediate descent on the VNAV Cruise page for avoiding traffic, whereas none of the 14 experienced pilots had any problem with this task." (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +4
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  177.  
  178. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked to intercept the LAX 248 [degree] radial, all 6 of the transition pilots had difficulties carrying out the task using LNAV, as compared to only 7 of the 14 experienced pilots." (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +3
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  179.  
  180. Evidence Type: Excerpt from Experiment
    Evidence: "[a] problem related to mode engagement was the attempt to activate a mode without the prerequistes for this activation being met: Three (50%) of the transitioning and one of the 14 experienced pilots tried to engage VORLOC without being in the manual radio mode as required. Three (50%) of the transitioning and 5 of the 14 experienced pilots engaged the APPR mode without lowering the MCP altitude first, and they were surprised to find that the aircraft did not start the descent." [Total number of pilots to make mistakes = 8 to 12 (66% to 88%)] (page 20)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +3
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  181.  
  182. Evidence Type: Excerpt from Experiment
    Evidence: "Nine out of 20 pilots knew how the FMS maintains target speed during a VNAV Path descent. Eight pilots (40%) were aware of the speed control mode during a VNAV Speed descent. With respect to the end-of-descent point of a path descent versus a speed descent, the results were similar: Twelve pilots (60%) were aware of the end of descent during a VNAV Path descent, and 9 pilots (45%) knew at what point the VNAV Speed descent would end." (page 16-17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +3
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  183.  
  184. Evidence Type: Excerpt from Experiment
    Evidence: "... the overall number of 13 altitude violations is an alarming result. Six pilots [33%] failed to detect and recover in time from the loss of previously entered altitude constraints as a consequence of the runway change in the MCDU. ... The problem in these seems to be that the pilot provides an instruction to the automation without realizing the additional unintended implications of his input." (page 61-62)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +2
    Aircraft: A320
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1995). Strong, Silent, and Out-of-the-loop: Properties of Advanced (Cockpit) Automation and Their Impact on Human-Automation Interaction. See Resource details

  185.  
  186. Evidence Type: Excerpt from Experiment
    Evidence: "... the overall number of 13 altitude violations is an alarming result. ... Seven pilots [39%] violated an assigned altitude when they reverted to an inappropriate mode from the EXPEDITE CLIMB mode. The problem in these seems to be that the pilot provides an instruction to the automation without realizing the additional unintended implications of his input." (page 61-62)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +2
    Aircraft: A320
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1995). Strong, Silent, and Out-of-the-loop: Properties of Advanced (Cockpit) Automation and Their Impact on Human-Automation Interaction. CSEL Report 95-TR-01. See Resource details

  187.  
  188. Evidence Type: Excerpt from Experiment
    Evidence: "Another interesting result refers to failures to engage or reengage a mode after entering new target values into the MCP or the CDU. This omission occurred at least once during the scenario for 5 of the 6 transitioning pilots (total number of omissions = 9). Only 2 of the 14 experienced pilots forgot to engage an appropriate mode, and this occurred only once for each of them." (page 18-20)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +2
    Aircraft: B737-300
    Equipment: FMS & autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  189.  
  190. Evidence Type: Excerpt from Experiment
    Evidence: "Another problem related to mode engagement was the attempt to activate a mode without the prerequisites for this activation being met. Fifty percent [3] of the transitioning pilots and 1 of the 14 experienced pilots tried to engage VORLOC without being in the manual radio mode as required. [(3+1)/20 = 4/20 = 20% failed]" (page 20)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +1
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  191.  
  192. Evidence Type: Excerpt from Experiment
    Evidence: "This study ... confirms that most of the difficulties in pilot-automation interaction are related to a lack of mode awareness and to gaps in pilots' mental models of the functional structure of the automation." (page 21)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: +1
    Aircraft: B737-300
    Equipment: automation
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  193.  
  194. Evidence Type: Excerpt from Experiment
    Evidence: "The GA mode becomes available when descending below 2,000 ft radio altitude with autothrottles armed. Out of 20 pilots, only 5 [25%] recalled the altitude at which this occurs. Eight pilots (40%) knew that the availability of the mode depends on reaching a certain altitude, but they did not remember the actual height." (page 16)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -1
    Aircraft: B737-300
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  195.  
  196. Evidence Type: Excerpt from Experiment
    Evidence: "After G/S capture, a G/S signal loss was simulated at approximately 3,000 ft ... pilots were asked about the consequences of this event, and 54% of the pilots provided the correct answer. When asked whether a G/S failure at a lower altitude (<1,500 ft) would have different effects, only 15% of the pilots knew the answer. Twenty-three percent of the participants did not know the answer to either question." (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -1
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  197.  
  198. Evidence Type: Excerpt from Experiment
    Evidence: Pilots were asked for their expectations concerning ADI mode indications throughout the takeoff roll ... N1 ... and THR HOLD ... Five of the pilots (25%) expected to see both theses indications. Twelve subjects (60%) only mentioned either THR HOLD (15% of the pilots) or N1 (45% of the pilots) as an indications during takeoff. Another 3 pilots (15%) could not predict any of the mode indications. (page 15)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -2
    Aircraft: B737-300
    Equipment: autopilot: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  199.  
  200. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked to intercept the LAX 248 [degree] radial, all 6 of the transition pilots had difficulties carrying out the task using LNAV, as compared to only 7 of the 14 experienced pilots." That is, 7 out of 20 (35%) did not have difficulties. (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -2
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  201.  
  202. Evidence Type: Excerpt from Experiment
    Evidence: "Nine [45%] out of 20 pilots knew how the FMS maintains target speed during a VNAV Path descent. Eight pilots (40%) were aware of the speed control mode during a VNAV Speed descent. With respect to the end-of-descent point of a path descent versus a speed descent, the results were similar: Twelve pilots (60%) were aware of the end of descent during a VNAV Path descent, and 9 pilots (45%) knew at what point the VNAV Speed descent would end." (page 16-17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -2
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  203.  
  204. Evidence Type: Excerpt from Experiment
    Evidence: "Another interesting result refers to failures to engage or reengage a mode after entering new target values into the MCP or the CDU. This omission occurred at least once during the scenario for 5 of the 6 transitioning pilots (total number of omissions = 9). Only 2 of the 14 experienced pilots forgot to engage an appropriate mode, and this occurred only once for each of them." That is, 13 of 20 (65%) did not forget. (page 18-20)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -3
    Aircraft: B737-300
    Equipment: FMS & autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  205.  
  206. Evidence Type: Excerpt from Experiment
    Evidence: "When asked to disengage the APPR mode after localizer and glideslope had been captured, only 3 pilots (15%) could recall the three ways of accomplishing this ... Seven pilots (35%) did not know of any procedure for disengaging the APPR mode." This means that 50% of the pilots were able to recall at least one way to disengage the glideslope. (page 16)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -3
    Aircraft: B737-300
    Equipment: autoflight: autopilot
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  207.  
  208. Evidence Type: Excerpt from Experiment
    Evidence: "Another problem related to mode engagement was the attempt to activate a mode without the prerequisites for this activation being met. Fifty percent of the transitioning pilots and 1 of the 14 experienced pilots tried to engage VORLOC without being in the manual radio mode as required." That is, 16 of 20 (80%) did it correctly. (page 20)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. See Resource details

  209.  
  210. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked about the consequences of using an excessive vertical rate of climb in the V/S mode, none of the transitioning pilots could provide the correct answer, as compared to only 5 (35%) of the experienced participants [i.e., (6 + 9)/20 = 75% didn't know]." (page 17)
    Issue: understanding of automation may be inadequate (Issue #105) See Issue details
    Strength: -4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  211.  
  212. Evidence Type: Excerpt from Experiment
    Evidence: "The results supported the conventional wisdom that automation has reduced physical workload more than it has mental workload. Inspection of the portion of the curve that is above 3.0 (“higher than in conventional cockpits”) shows that Physical workload was rated as being higher than in the past for preflight (4.04) and taxi before takeoff (3.14). The biggest reduction in physical demands can be seen in the cruise phase (1.69) presumably reflecting the widespread use of flight control and navigation automation. Mental workload was rated as being higher than in the past for preflight (4.03), taxi before takeoff (3.22), and approach for landing (3.24)." (page 110)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +4
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  213.  
  214. Evidence Type: Excerpt from Experiment
    Evidence: "The results supported the conventional wisdom that automation has reduced physical workload more than it has mental workload. Inspection of the portion of the curve that is above 3.0 (“higher than in conventional cockpits”) shows that Physical workload was rated as being higher than in the past for preflight (4.04) and taxi before takeoff (3.14). The biggest reduction in physical demands can be seen in the cruise phase (1.69) presumably reflecting the widespread use of flight control and navigation automation. Mental workload was rated as being higher than in the past for preflight (4.03), taxi before takeoff (3.22), and approach for landing (3.24)." (page 110)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +4
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  215.  
  216. Evidence Type: Excerpt from Experiment
    Evidence: "The results supported the conventional wisdom that automation has reduced physical workload more than it has mental workload. Inspection of the portion of the curve that is above 3.0 (“higher than in conventional cockpits”) shows that Physical workload was rated as being higher than in the past for preflight (4.04) and taxi before takeoff (3.14). The biggest reduction in physical demands can be seen in the cruise phase (1.69) presumably reflecting the widespread use of flight control and navigation automation. Mental workload was rated as being higher than in the past for preflight (4.03), taxi before takeoff (3.22), and approach for landing (3.24)." (page 110)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +2
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  217.  
  218. Evidence Type: Excerpt from Experiment
    Evidence: "The results supported the conventional wisdom that automation has reduced physical workload more than it has mental workload. Inspection of the portion of the curve that is above 3.0 (“higher than in conventional cockpits”) shows that Physical workload was rated as being higher than in the past for preflight (4.04) and taxi before takeoff (3.14). The biggest reduction in physical demands can be seen in the cruise phase (1.69) presumably reflecting the widespread use of flight control and navigation automation. Mental workload was rated as being higher than in the past for preflight (4.03), taxi before takeoff (3.22), and approach for landing (3.24)." (page 110)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +2
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  219.  
  220. Evidence Type: Excerpt from Experiment
    Evidence: "The results supported the conventional wisdom that automation has reduced physical workload more than it has mental workload. Inspection of the portion of the curve that is above 3.0 (“higher than in conventional cockpits”) shows that Physical workload was rated as being higher than in the past for preflight (4.04) and taxi before takeoff (3.14). The biggest reduction in physical demands can be seen in the cruise phase (1.69) presumably reflecting the widespread use of flight control and navigation automation. Mental workload was rated as being higher than in the past for preflight (4.03), taxi before takeoff (3.22), and approach for landing (3.24)." (page 110)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: +2
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  221.  
  222. Evidence Type: Excerpt from Experiment
    Evidence: "The results supported the conventional wisdom that automation has reduced physical workload more than it has mental workload. Inspection of the portion of the curve that is above 3.0 (“higher than in conventional cockpits”) shows that Physical workload was rated as being higher than in the past for preflight (4.04) and taxi before takeoff (3.14). The biggest reduction in physical demands can be seen in the cruise phase (1.69) presumably reflecting the widespread use of flight control and navigation automation. Mental workload was rated as being higher than in the past for preflight (4.03), taxi before takeoff (3.22), and approach for landing (3.24)." (page 110)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -2
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  223.  
  224. Evidence Type: Excerpt from Experiment
    Evidence: "The results of averaging the ratings across participants and components showed that, overall, participants found the automation components to be unobtrusive (4.02), predictable (3.96), extremely helpful for reducing workload (3.81). and they were inclined to use them whenever appropriate (4.39). On the other hand, they were close to the midpoint when it came to the feeling that they were controlling the flight rather than managing the automation (3.28, in which 5 indicates high controlling), and the feeling that they were focusing on the flight rather than on the automation (3.64, in which 5 indicates attention to flight)." (page 115)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -3
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  225.  
  226. Evidence Type: Excerpt from Experiment
    Evidence: "Concern ratings showed a clear differentiation of levels (see Table 3). Overall concern grew from 2.35 to 3.29 as automation increased, which was a significant difference, F(2, 220) = 96.001, p < .OOOl . A Tukey test showed that each of the levels differed significantly from the others. These results were highly stable, with concern increasing regularly across levels for a majority of items (see superscripted items in Table 3). Major concerns at the fully automated level were increased head-downtime (4.05) complacency (3.95), and degradation of pilot skills (3.90)." (page 111)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: +4
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  227.  
  228. Evidence Type: Excerpt from Experiment
    Evidence: "To assess whether the dimensions were related, correlations were calculated between every possible pair of dimensions, using the average score across participants for each of the 26 components listed in Table 4 as the data entered into the correlation (n = 26). ...This same cluster of correlated dimensions also related to workload (Dimension F). Parts rated highly distracting (Dimension C) were rated as workload intensive, R = .8933, p < .001, as were parts requiring a high degree of attention to the automation (Dimension A), R = .4034, p < .05. Difficulty in predicting the automation’s behavior (Dimension E) also contributed to high workload, R = .6966, p < .05." (page 116)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  229.  
  230. Evidence Type: Excerpt from Experiment
    Evidence: "The one remaining significant correlation concerned understanding of the big picture (Dimension B). In this case, components that contributed to a deeper understanding of the situation (Dimension B) also were rated as demanding attention to the automation (Dimension A), R = -.3955, p < .05." (page 116)
    Issue: automation may demand attention (Issue #102) See Issue details
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  231.  
  232. Evidence Type: Excerpt from Experiment
    Evidence: "To assess whether the dimensions were related, correlations were calculated between every possible pair of dimensions, using the average score across participants for each of the 26 components listed in Table 4 as the data entered into the correlation (n = 26). ...This same cluster of correlated dimensions also related to workload (Dimension F). Parts rated highly distracting (Dimension C) were rated as workload intensive, R = .8933, p < .001, as were parts requiring a high degree of attention to the automation (Dimension A), R = .4034, p < .05. Difficulty in predicting the automation’s behavior (Dimension E) also contributed to high workload, R = .6966, p < .05." (page 116)
    Issue: behavior of automation may not be apparent (Issue #83) See Issue details
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  233.  
  234. Evidence Type: Excerpt from Experiment
    Evidence: Concern ratings showed a clear differentiation of levels (see Table 3). Overall concern grew from 2.35 to 3.29 as automation increased, which was a significant difference, F(2, 220) = 96.001, p < .OOOl . A Tukey test showed that each of the levels differed significantly from the others. These results were highly stable, with concern increasing regularly across levels for a majority of items (see superscripted items in Table 3). Major concerns at the fully automated level were increased head-downtime (4.05) complacency (3.95), and degradation of pilot skills (3.90) (page 111)
    Issue: manual skills may be lost (Issue #65) See Issue details
    Strength: +3
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  235.  
  236. Evidence Type: Excerpt from Experiment
    Evidence: "The results of averaging the ratings across participants and components showed that, overall, participants found the automation components to be unobtrusive (4.02), predictable (3.96), extremely helpful for reducing workload (3.81). and they were inclined to use them whenever appropriate (4.39). On the other hand, they were close to the midpoint when it came to the feeling that they were controlling the flight rather than managing the automation (3.28, in which 5 indicates high controlling), and the feeling that they were focusing on the flight rather than on the automation (3.64, in which 5 indicates attention to flight)." (page 115)
    Issue: pilot control authority may be diffused (Issue #104) See Issue details
    Strength: -3
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  237.  
  238. Evidence Type: Excerpt from Experiment
    Evidence: Concern ratings showed a clear differentiation of levels (see Table 3). Overall concern grew from 2.35 to 3.29 as automation increased, which was a significant difference, F(2, 220) = 96.001, p < .OOOl . A Tukey test showed that each of the levels differed significantly from the others. These results were highly stable, with concern increasing regularly across levels for a majority of items (see superscripted items in Table 3). Major concerns at the fully automated level were increased head-downtime (4.05) complacency (3.95), and degradation of pilot skills (3.90). (page 111)
    Issue: pilots may be overconfident in automation (Issue #131) See Issue details
    Strength: +3
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  239.  
  240. Evidence Type: Excerpt from Experiment
    Evidence: The results of averaging the ratings across participants and components showed that, overall, participants found the automation components to be unobtrusive (4.02), predictable (3.96), extremely helpful for reducing workload (3.81). and they were inclined to use them whenever appropriate (4.39). On the other hand, they were close to the midpoint when it came to the feeling that they were controlling the flight rather than managing the automation (3.28, in which 5 indicates high controlling), and the feeling that they were focusing on the flight rather than on the automation (3.64, in which 5 indicates attention to flight). (page 115)
    Issue: pilot's role may be changed (Issue #144) See Issue details
    Strength: +3
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. See Resource details

  241.  
  242. Evidence Type: Excerpt from Experiment
    Evidence: "The results of averaging the ratings across participants and components showed that, overall, participants found the automation components to be unobtrusive (4.02), predictable (3.96), extremely helpful for reducing workload (3.81). and they were inclined to use them whenever appropriate (4.39). On the other hand, they were close to the midpoint when it came to the feeling that they were controlling the flight rather than managing the automation (3.28, in which 5 indicates high controlling), and the feeling that they were focusing on the flight rather than on the automation (3.64, in which 5 indicates attention to flight)." (page 115)
    Issue: pilot's role may be changed (Issue #144) See Issue details
    Strength: -2
    Aircraft: various
    Equipment: automation
    Source: Skitka, L.J., Mosier, K.L., Burdick, M., & Rosenblatt, B. (2000). Automation bias and errors: Are crews better than individuals?. International Journal of Aviation Psychology, 10(1), 85-97. Lawrence Erlbaum Associates. See Resource details

  243.  
  244. Evidence Type: Excerpt from Experiment
    Evidence: "The results of the examination of workload and automation for the A320 certification data suggested that for the range of automation experienced, an inverse relationship exists. Thus, as automation increases, workload experienced by the crew decreases. Unfortunately, the flying situations could not support firm conclusions with respect to automation." (page (3))
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: A320
    Equipment: automation
    Source: Speyer, J.J. & Blomberg, R.D. (1989). Workload and automation. See Resource details

  245.  
  246. Evidence Type: Excerpt from Experiment
    Evidence: "... statistical results from this formal test strongly militated for the fly-by-wire/sidestick arrangement planned for the A320: ... - pilot control inputs were reduced by 50% or more (as seen from reversal rates), lowering pilot taskload and hence enabling him to attend other supervisory duties." (page (4))
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -3
    Aircraft: A320
    Equipment: flight control
    Source: Speyer, J.J., Blomberg, R.D., & Fouillot, J.P. (1990). Evaluation the Impact of New Technology Cockpits: Onwards from A300FF, A310, A320 to A330, A340. In Proceedings of the International Conference on Human Machine Interaction and Artificial Intelligence in Aeronautics and Space. See Resource details

  247.  
  248. Evidence Type: Excerpt from Experiment
    Evidence: "It appears from ... [the] graphic plots that the results of each aircraft under certification were generally indicating decreased taskload burden for each crewmember when compared to their referenced aircraft. Burden figures for CM2 are always much higher than for CM1 as the former s carrying out the bulk of the system management work. With regard to the weighted average taskload the individual crewmember figures for the aircraft under certification were generally equivalent to their reference aircraft. More important, however, was the fact that they stay well inside the satisfactory range of the static taskload scale. It is concluded that there are less tasks on the new aircraft and that they are easy to execute. Several other ways exist to graphically represent the results of the Static Taskload Analysis one of which being Normalized Principal Components Analysis of the taskload matrices. ... The objective of normalized principal components analysis is to provide a synthetic representation of the information contained in a matrix of p continuous variables and n observations. ... In this particular way of representation we used procedure matrices whose observation points corresponded with the burden data for normal, abnormal and emergency procedures of both aircraft to be compared. The variable corresponded with the 6 elementary activities in a task. Differentiation of the two aircraft to be compared (the DC-9 and the A300 FF) was done by attributing different codes to the projected observation points. ... One can get an idea of the homogeneity of procedures or of the homogeneity of action burden data associated with the procedures whether the subclouds are clustered or dispersed. In essence this method indicated that as a whole the elementary activities normal procedures and 10 abnormal/emergency procedures. ... Task analyses of system management activities were performed for each crew member of each aircraft with a task breakdown into basic actions (look, observe, monitor, reach, operate and monitor) on the A300 FF are more homogenously grouped and centered and therefore less demanding that on the DC-9." (page 475, 478)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: A300FF
    Equipment: automation
    Source: Speyer, J.J., Fort, A., Fouillot, J.P., & Blomberg, R.D. (1987). Assessing pilot workload for minimum crew certification. See Resource details

  249.  
  250. Evidence Type: Excerpt from Experiment
    Evidence: "The performance benefits of the NAV condition were clearly documented in this study. The FMC appears capable of commanding the AP to navigate the aircraft in the horizontal plane with great precision and repeatability. This frees the pilots to attend to other tasks or simply reduces their workload and makes them more available to respond to unexpected occurrences." (page 11.14)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: A310
    Equipment: FMS & autoflight: autopilot
    Source: Speyer, J.J., Monteil, C., Blomberg, R.D., & Fouillot, J.P. (1990). Impact of New Technology on Operational Interface: From Design Aims to Flight Evaluation and Measurement. Advisory Group for Aerospace Research and Development No. 301, Vol. 1. See Resource details

  251.  
  252. Evidence Type: Excerpt from Experiment
    Evidence: "... the performance gains observed for both the EFIS and FMS were not associated with any increase in the workload perceived by the pilots in the experiments. In fact, they rated (on a 10-point numeric interruption scale) flying with the EFIS as a lower workload situation than flying with conventional instruments. Likewise, use of the FMS was associated with lower rated workload than trials flown without it. Although neither of these latter differences was statistically significant, the results provided the clear implication that pilot workload would be positively influenced by the introduction of these new, electronic flight instruments." (page 11.15)
    Issue: automation may adversely affect pilot workload (Issue #79) See Issue details
    Strength: -1
    Aircraft: A310
    Equipment: EFIS & FMS
    Source: Speyer, J.J., Monteil, C., Blomberg, R.D., & Fouillot, J.P. (1990). Impact of New Technology on Operational Interface: From Design Aims to Flight Evaluation and Measurement. See Resource details

  253.  
  254. Evidence Type: Excerpt from Experiment
    Evidence: "It is also important to realize that the performance benefits of the NAV condition were achieved without noticeably altering the 'style' in which the aircraft flew the circuit. The tracks produced by the FMS appeared 'normal', i.e., not unlike the intended track or the tracks produced when the pilots flew in the STANDARD condition. There was no apparent cause for concern that flight tracks flown with the FMS in command would differ materially from those flown by aircraft not equipped with an [in sic] FMS." (page 11.14)
    Issue: automation may use different control strategies than pilots (Issue #122) See Issue details
    Strength: -1
    Aircraft: A310
    Equipment: FMS & autopilot
    Source: Speyer, J.J., Monteil, C., Blomberg, R.D., & Fouillot, J.P. (1990). Impact of New Technology on Operational Interface: From Design Aims to Flight Evaluation and Measurement. Advisory Group for Aerospace Research and Development No. 301, Vol. 1. See Resource details

  255.  
  256. Evidence Type: Excerpt from Experiment
    Evidence: "It is also important to realize that the performance benefits of the NAV condition were achieved without noticeably altering the 'style' in which the aircraft flew the circuit. The tracks produced by the FMS appeared 'normal', i.e., not unlike the intended track or the tracks produced when the pilots flew in the STANDARD condition. There was no apparent cause for concern that flight tracks flown with the FMS in command would differ materially from those flown by aircraft not equipped with an [in sic] FMS. Hence, it could be concluded that the Airbus A310 and similarly equipped aircraft should blend smoothly and easily into the existing ATC environment regardless of the mode in which they are flown." (page 11.14)
    Issue: flightdeck automation may be incompatible with ATC system (Issue #82) See Issue details
    Strength: -1
    Aircraft: A310
    Equipment: FMS & autopilot & ATC
    Source: Speyer, J.J., Monteil, C., Blomberg, R.D., & Fouillot, J.P. (1990). Impact of New Technology on Operational Interface: From Design Aims to Flight Evaluation and Measurement. See Resource details
Flight Deck Automation Issues Website  
© 1997-2013 Research Integrations, Inc.