The ironies of automation

An increasingly automated world

“Technology… is a queer thing. It brings you great gifts with one hand, and it stabs you in the back with the other”.

Carrie Snow

Automation can be defined as the use of machines and technology to operate or control a process or system without continuous input from an operator, reducing human intervention to a minimum.

Increasingly these days we see or interact with automated systems. A machine makes our morning coffee, our fruit or cereal is automatically picked, sorted, and bagged by machines, our morning newspaper is delivered on a tablet computer, our floors are cleaned by a robot, we have smart phones and TVs and groceries can be ordered automatically online by a smart refrigerator.

The invention and development of technology in personal, business and social application, has changed our lives both for the better and the worse. Some positive changes include:

  • Education – online schools and universities, as well as ease of access to virtual libraries, has allowed people from remote locations or disadvantaged communities to get an education as long as they have access to a computer and the internet.
  • The age of ‘now’ communication – in the past a letter could take weeks to get to a far-flung destination. Today we have mobile, internet, computer and social media, video conferencing tools and mobile apps to communicate instantly with anyone around the world at any time.

However, technology has also brought some negatives, including:

  • Addiction – many people are addicted to the internet or cannot stop themselves habitually scrolling through their phone. This does not encourage creativity or the development of social skills. Some experts believe it is having a negative impact on brain development.
  • Health and fitness – sitting down next to a computer all day long has created a largely sedentary society.
  • Critical thinking skills – why think when you can use a search engine or Wikipedia! Everyone wants to read the easy explanation, which risks the development of critical thinking skills.
  • Dependence – when the technology doesn’t work, we often can’t fix the problem and function without it. Compare the difficultly of repairing a fault in a modern electric car compared to a 1967 HQ Holden!

The pace of technological change

Consider how fast technology has developed. Gadgets we thought a few years ago were science fiction are now a reality, thanks to the greatest predictors of modern day 21st century living; the creators of the original Star Trek TV series! The show first aired in September 1966 and offered a fascinating look at what space exploration might look like in the future. A surprising number of before-their-time technological gadgets from the show are now in widespread use.

The most obvious influence for designers was the flip-phone communicators used by the crew of the Enterprise, which inspired the mobile phone. Other familiar gadgets include tablet computers, voice interface computers, Bluetooth headsets, portable memory (from floppy disks to USB sticks), automatic doors, big screen displays and teleconferencing (today’s WhatsApp and Skype).

 

Figure 1. Star Trek original TV series 1966

A few technologies were never realised, such as the ‘beam me up’ transporter, able to dematerialise and rematerialise people. Perhaps this is a good thing, because it would most likely spell the end of present day air transport systems. Why fly when you can dematerialise?

Contrasting design philosophies

There are two distinct approaches to the design of automation:

  1. Technology-centred; and
  1. Human-centred.

Technology-centred automation seeks to overcome the limitations of human performance by replacing human functioning with machine functioning. The overriding design philosophy is to use automation wherever possible to reduce operator workload and eliminate errors. Designers seek to exploit the accuracy and efficiency of automation to achieve economies such as efficiency, user comfort and reduced training costs.

By contrast, human-centred automation seeks to enhance the capabilities of, and compensate for, the limitations of human performance. The philosophy is not to replace human functioning, but rather to enhance human effectiveness by optimising workload and supporting the operator in managing complex systems and making effective and timely decisions.

There are concerns that technology-centred automation in some industries like aviation has taken pilots ‘out of the loop’, to the extent that they may not be able to adequately perform their monitoring and supervisory roles.

The automation paradox

While designers have often focused on reducing the physical workload of operators, they have not adequately considered the effect this may have on increasing the operator’s cognitive or mental workload. Increasingly complex automated systems require more cognitive processing and a greater understanding of their design and functions. This has led to the following problems:

  • Over-reliance – As the level of automation increases, by definition, the need for manual control of certain processes reduces. We may be hesitant to take over manual control if we are not confident in our ability to do so, or feel we are no longer proficient in such tasks. There is a similar concern about the risks of over-reliance on automation in areas from self-driving vehicles to spell-checkers.
  • Automation-induced complacencyoperators may be hesitant to intervene at the first signs of trouble as we have a high level of trust in the system to correct itself, and often leave it until it is too late to intervene.
  • Startle and surprisethis involves unexpected situations that disrupts our cognitive processing and can have a negative impact on our decision-making and problem-solving abilities. That is, it can take time to recover our senses just at a time when we are most likely to have to react quickly and decisively to recover an abnormal situation.

Increasing Automation: Too much of a good thing?

On 1 June 2009, an Air France Airbus A330 en-route from Rio de Janeiro to Paris crashed into the Atlantic Ocean, killing all 228 on board. The aircraft had entered an aerodynamic stall after the autopilot disengaged because of inconsistencies between airspeed measurements, likely caused by ice crystals in the aircraft’s pitot tubes.

The final report by the French investigators concluded that the crew had failed to follow appropriate procedures for loss of airspeed information. Without reliable information about the aircraft’s angle of attack, the crew had made inappropriate control inputs.

The pilot flying had failed to recognise that the aircraft was stalled, despite a stall warning. Once the angle of attack became extreme, the system rejected the data as invalid and stopped the warning, with the perverse result that each time the pilot lowered the nose, the warning started again.

Figure 2. Wreckage recovered from the Air France aircraft off the coast of Brazil

Ironies of automation

While a favoured strategy by engineers is to allocate the machine every function that can be automated, as a way of reducing the risk of error, this is often dangerous in practice. First, not everything can be automated, and leaving unrelated bits and pieces of functions to people leads to boredom, inattention, de-motivation and overall poor performance. Second, machines are not perfectly reliable, and the more complex they are, the more they fail. This usually means employing higher skilled, better trained and more expensive operators, who, with not enough to do, suffer from even higher levels of boredom. See the below Panel, Ironies of automation.

 

Figure 3. The ironies of automation

Want to know more?

For more in depth information about human factors solutions or practical human factors training for your workplace, contact Leading Edge Safety Systems. We are a group of highly qualified and experienced human factors experts, with second to none experience in a range of industries with a proven track record of providing practical solutions to addressing key safety, risk and human factors challenges in the workplace.

The ironies of automation

Leave a Reply

Your email address will not be published. Required fields are marked *