If we deploy automation without thinking strategically about intelligence, too, isn’t AI likely to backfire on us?

Airplane manufacturer, Boeing, made headlines in 2019 for all the wrong reasons. Its 737 Max aircraft was indefinitely grounded after two fatal crashes in the space of just six months had claimed the lives of 346 people. Investigation into the accidents revealed that updates to an automated system – the Maneuvering Characteristics Augmentation System, known as MACS – had failed to integrate one of two intelligent sensors, meaning the system lacked a critical security backstop. As the aircrafts switched into autopilot mode shortly after takeoff, the error sent them both into fatal nosedives within minutes.

These tragedies highlight an issue with automation that needs more focused attention says Goizueta’s Ruomeng Cui, assistant professor of information systems and operation management. And it’s this: if we deploy automation without thinking strategically about intelligence too, is AI likely to backfire on us?

Cui is an expert in operations strategies in digital retail and platform markets. To better understand the challenges surrounding automation and intelligence in operational processes, she teamed with Shichen Zhang of Tianjin University and Rutgers’ Meng Li to explore how AI brings value in the procurement space.

With Deloitte reporting that almost 45% of Chief Procurement Officers globally are now using, piloting, or planning to integrate AI into their operations, these insights should provide interesting food for thought, says Cui.

“AI isn’t just about being quicker, it’s also about being smarter. It can deliver automation but can also deliver predictive intelligence; and while these two dimensions might be correlated, one doesn’t necessarily imply the other – as the Boeing example demonstrates,” says Cui.

From the tech perspective, there’s a lot of buzz about how AI is helping to drive decision-making, she adds. But there is still plenty that we don’t know about the operational dimensions to using artificial intelligence.

“With international procurement, you’re basically talking about big retailers going in and requesting prices for goods or products from wholesale suppliers. And that’s a process that could, in theory, lend itself very well to AI, since it can automate simple (and tedious) tasks over and over again. So there’s a significant potential gain in companies outsourcing this kind of task to the machine.”

But although the potential might be clear, Cui and her colleagues believe that simply automating these processes might not in fact yield optimal results; and could in fact work against buyers by encouraging suppliers to quote higher prices than they might in personalized, human transactions.

A full article detailing Cui’s research is attached, within it – several theories were explored.

Who Comes Out Ahead on Price? Humans or AI Chat Bots?

“We speculated about the possibility of wholesalers discriminating against the AI,” says Cui. “Specifically, we wanted to know if the sellers would quote higher prices to AI bots than they would to human buyers, because at the end of the day these bots are just machines; they don’t bring the authenticity or sincerity of human beings.”

When Machines are Smart, Discounts Rise

“When wholesalers are just asked over and over for their prices, they know that they are dealing with a machine and the intuition is that the machine is not intelligent, that it doesn’t have market expertise, and that it isn’t capable of decision-making. There’s no incentive to build relationships or to engage in any kind of negotiating dynamic here.”

The topic is fascinating, and given the increase of AI in the workplace – a timely one.

And, if you are a journalist looking to cover this research or speak with Professor Ciu about the subjects of telework and productivity, simply click on her icon now to arrange an interview today.