From Microsoft’s infamous AI-powered tweet bot, Tay, which within 24 hours of its release started to spew racist responses, to Amazon’s deeply biased machine-learning recruitment tool, examples of artificial intelligence going ‘rogue’ are easy to find.
Concerns over opaque black-box algorithms to questions regarding the ethical use of personal data and responsibilities related to security and privacy has made AI a hotbed of modern ethical dilemmas.
These dilemmas must be addressed by the swathes of public and private organisations now relying on AI to power innovation. However, despite the proliferation of AI in the enterprise, many organisations still lack strong AI governance crucial to ensuring the integrity and security of data-led systems.
In fact, the latest O’Reilly research shows that over half of AI products in production at global organisations still do not have a governance plan overseeing how projects are created, measured and observed.
Deeply concerning is that privacy and security – issues that may directly impact individuals – were among the risks least cited by organisations when questioned on how they evaluate the risks for AI applications. AI-empowered organisations report that ‘unexpected outcomes’ are the most significant risk facing AI projects, followed closely by model interpretability and model degradation, representing business issues. Interpretability, privacy, fairness, and safety all ranked below business risks.
There may be AI applications where privacy and fairness are not issues (for example, an embedded system that decides whether the dishes in your dishwasher are clean). However, companies with AI practices must prioritise the human impact of AI as both an ethical imperative and a core business priority.
As UKRI (UK Research and Innovation) highlights, ‘responsible use of AI is proving to be a competitive differentiator and key success factor for the adoption of AI technologies. However, cultural challenges, and particularly the lack of trust, are still deemed to be the main obstacles preventing broader and faster adoption of AI.’
Lack of governance is not just an ethical concern. Security is also a massive issue, with AI subject to many unique risks: data poisoning, malicious inputs that generate false predictions, and reverse engineering models to expose private information, to name a few. However, security remains close to the bottom of the list of perceived AI risks.
With cybercriminals and bad actors surging ahead in their adoption of sophisticated technology, cybersecurity cannot take a back seat in the race to realise AI’s promise. It is a vital strand of much-needed AI governance. Governance must rise up the matrix of risk factors for AI projects, becoming a cornerstone of any development and deployment programme.
AI governance in a nutshell
With that in mind, what exactly is AI governance? According to Deloitte, it encompasses a ‘wide spectrum of capabilities focused on driving the responsible use of AI. It combines traditional governance constructs (policy, accountability, etc.) with differential ones such as ethics review, bias testing, and surveillance. The definition comes down to an operational view of AI and has three components: data, technique/algorithm, and business context.’
In summary, ‘achieving widespread use of AI requires effective governance of AI through active management of AI risks and implementation of enabling standards and routines.’
Without formalising AI governance, organisations are less likely to know when models are becoming stale, results are biased, or when data is improperly collected. Companies developing AI systems without stringent governance to tackle these issues are risking their businesses. They leave the way open for AI to effectively take control, with unpredictable results that could cause irreparable damage to reputation and large legal judgments.
The least of these risks is that legislation will impose governance, and those who have not been practising AI governance will need to catch up. In today’s rapidly shifting regulatory landscape, playing catch up is a risk to reputation and business resilience.
What has created the AI governance gap?
The reasons for AI governance failure are complex and interconnected. However, one thing is clear – accelerated AI development and adoption has not been matched by a surge in education and awareness of its risks. What this means is that AI is suffering a people problem.
For example, the most significant bottlenecks to AI adoption are a lack of skilled people. Our research demonstrates significant skills gaps in key technological areas, including ML modelling and data science, data engineering, and the maintenance of business use cases. The AI skills gap is well documented, with much government discussion and policy to drive data skills through focused tertiary education and up/reskilling.
However, technological skills are not enough to bridge the gap between innovation and governance. It is neither advisable nor fair to leave governance to technical talent alone. Undoubtedly those with the skills to develop AI must also be equipped with the knowledge and values to make decisions and problem solve within the broader context in which they operate. However, AI governance is truly a team effort and represents the values of an organisation brought to life.
That means no organisation can be complacent when embedding ethics and security within AI projects from the outset. That means everyone across the organisation, from CEO to data analyst, CIO to project manager, must engage in AI governance. They must align on why these issues matter and how the organisation’s values play out through AI implementations.
Such a strategy starts with empowerment through education, awareness and role-specific training. When it comes to AI, vigilance is a holistic skill that all must master. Frameworks, principles and policies provide the basis for sound innovation but mean nothing without engaged, educated and empowered humans to bring them to life.
Rachel Roumeliotis, Vice President of Data and AI at O’Reilly
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.