At the conclusion of each calendar year, organizations and observers across all industries try to put forward predictions on what the new year might hold, and we at Thales were no exception. In the latest episode of our Thales Security Sessions podcast episode, entitled Predictions, I spoke with guest David Holmes, CTO for Application Security at Imperva, in detail about this very issue. David delivered some excellent observations on the activities and technologies that currently vie for our attention, including AI, passkeys, APTs, and ransomware. When considering all of these variables, the constant thing that kept coming back to my mind as the host and co-producer of the show is the one subject that sits at the root of all of these changes and challenges: human nature.
Efficiency Through AI
We spoke about AI, of course, and, as David pointed out, inevitably,, the corporate approach to leap in with both feet into the deep end of the AI pool, is often made with cost savings as the primary motivator. Company decision makers all over the world are getting excited over the prospect of just how much money can be saved by delegating tasks that were once done by employees, to a new AI or Generative AI solution.
David quoted an aviation industry study conducted at the start of 2024, which demonstrated how Generative AI could be used to handle phone calls with customers who had straightforward requests such as rescheduling or canceling their flight, changing their assigned seat, or getting credit for travel miles. The study projected that an airline would save $667 million a year by giving this over to AI. The pressure, he said, simply that bottom line pressure of saving two-thirds of a billion dollars is going to be massive and impossible to resist.
A Cautionary Tale
When I see such lofty economic panoramas, painted from the glorious perspective of the Gartner Hype Cycle’s Peak of Inflated Expectations, I think back to the Disney movie Fantasia, and in particular, the part dealing with The Sorcerer’s Apprentice. If you have not seen this chapter of the Fantasia film, it is just a click away on YouTube. In short, an overworked magician’s assistant, tired of schlepping and sweeping, sneakily borrows his boss’s book of magic spells and creates a small army of miniature autonomous brooms to do his work for him. The brooms, however, quickly show they have their own ideas on how to get things done, and a humiliated Mickey must ask his boss, the Sorcerer, to restore calm.
In our real world, we, too, often feel like the overworked apprentice, and when a new technology, seemingly capable of doing all the drudge work comes along, it seems too good to be true. That’s because it probably is.
To be clear, I am a technology enthusiast, not a Luddite. I enjoy seeing developments in technologies and am just as happy as the next person to integrate ChatGPT into my working life. But there’s a strong caveat that all humans must consider when considering bringing novelty into an existing workflow. Most of us have already heard of hallucinations, the polite term for inaccuracies and outright untruths that Generative AI will create on the strength of your request, but less dramatic but equally as impactful, are the non-successes of other innovative technologies that come face-to-face with unchanging human nature.
Lessons From the Past
Think about how computer software gave rise to viruses and ransomware. These were not errors in coding; they were developed by humankind’s predatory nature to use innovation to cause chaos or steal money. Think about passwords and how much people don’t want to change their passwords or use better technology like passkeys because of the inconvenience of change. Think about social media – a tool designed to bring the world together, tainted by toxicity and bullying from faceless trolls, and driven by algorithms. Think about PowerPoint. What’s wrong with PowerPoint? Nothing in general, but in most cases, it has famously not improved the productivity or usefulness of most meetings. The same can be said about video chat tools like Zoom or Microsoft Teams. Great technology, but in many cases, it has led to more boring meetings, just now with bad lighting and blurred backgrounds.
Humans have a hard time pushing themselves into the future. Change and progress are ruled by two factors. The first is the desire to make every recent technology do what old processes used to do. The first television shows were Vaudeville-type skits. The first web pages were simply book indexes, and again, the first decade of video chat is an online replication of a meeting room.
The Human Touch
When technologies like Generative AI show promise by appealing to the most pressing commercial priority of all – making more at less cost, the same potential for running awry remains. The human touch remains vital to the quality of the output. AI may be superior in analyzing visual anomalies in X-rays and CAT scans, for example, but it takes human intuition to interpret the results correctly. A chatbot might be great in welcoming an online shopper to an e-commerce site, but the moment it fails to interpret a customer’s question, that customer will leave and find a different vendor. ChatGPT can write lots of material, but it comes out bland and shapeless, like elevator music.
Worse, as David and I discussed during the episode, a facility – let’s say the world’s largest storage site of plutonium – or a simple water treatment plant – can have all the safety procedures and technologies available to humankind, but when its administrators forget to enforce the safety checks, yet file the paperwork that says that they did them, or when all the safety engineers are given the same password in order to check on their plants remotely rather than doing all that costly driving, the house of cards collapses.
It is important to recognize that humans who, for the moment, are still in control of the technologies they have invented are still human, and their desire to get ahead by saving money, cutting corners, or simply reproducing the activities of the past in a new format, will be the architects of each technology’s success or failure. Unfortunately, human nature is much, much older than any technology, and it still dominates our actions and thoughts. Most year-end predictions have a 50-50 chance of proving themselves correct. Still, I would suggest that counting on human nature to create a mess or at least perpetuate mediocrity should be a vital ingredient in any company’s plans for the new year. Optimism is wonderful, but pragmatism will take you much further.
Steve is a specialist in organizational psychology, focusing on the interaction of people, technology and change. He holds degrees in journalism and psychology, and is pursuing a PhD in Psychology, focusing on brain/technology interaction.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.