How We Can Get Beyond The Buzzwords To Understand What’s Real, And What Works In AI

The State of AI 2019 is comprehensive study which shows that corporate adoption of Artificial Intelligence has tripled in the last 12 months, with one in seven large companies now adopting some form of AI.  

Expert Comment below:

Matt Walmsley, EMEA Director at Vectra: 

With 40% of Europe’s so called “AI companies” being exposed as not actually using AI in their offerings* it’s understandable that there’s scepticism around the liberal use of AI in many companies’ claims. So how can we get beyond the buzzwords to understand what’s real, and what works in AI? Here’s a few tips from Vectra who were founded in 2012 with a singular focus on using AI, way before it was in vogue, to solve some incredibly tough cybersecurity challenges. 

When looking at a company, think of questions like: What’s the depth and breadth of their development capability? How many have a Machine Learning (ML) or Data Science background, what’s their academic history? For example, do they have post-doctorates who are developing the very edges of knowledge in their field and are hard to find and hire, but equally can provide a wealth of knowledge and the capability to really innovate. Jump on LinkedIn and get searching their employees! If there’s just one developer, with no demonstrable ML experience then at best they’re likely to have bolted on an open source ML library to some existing code and are just “playing” with AI.  

Equally, try and see if there are subject matter experts in the problem domain that you’re looking to solve. In many cases of applied AI, it’s not simply a case of capturing big data and throwing mathematics at it. For example, in cybersecurity we need expert security researchers to hypothesise and validate attacker technical behaviours, something a data scientist alone wouldn’t have any insight in. Only when collaborating with the security researcher can the data scientist develop an effective cybersecurity attacker detection algorithm. 

Look for evidence of a commitment to long term innovation and demonstrable results. Awards and industry recognition have a spectrum of creditability and relevance. Are they named and respected by credible industry analysts?  Have they got patents pending, or even better awarded for the AI capabilities they’ve created? That shows commitment in building value through innovation (and protecting it). 

It’s called data science for a reason, it’s not just about building amazingly intricate algorithms! Aside from selecting algorithmic approaches the Data Scientist also has to manage the curation of data, feature selection and extraction, and training. Ask your want-to-be AI vendor about the data they use to build, train, test and operate their algorithms. Seek to understand their data’s provenance, quality, management and security, and how and where it is used. 

Particularly for younger businesses critique the executive team, and their backers. Do they have relevant experience in the sector, or at least tangential to it? What’s their track record and reputation, where else have they invested, where have they been successful? Look for teams and investors that are not just looking at AI as an easy funding source, that overly hype themselves, or are simply chasing a perceived market opportunity. 

Finally, remember that as a group of technologies AI is just the tool, not the goal or reason.  It’s the “How” not the “Why?”.  Does the AI tool you’re considering create efficiencies or new insights in a truly autonomous way, or is more a “mechanical Turk” with veneer of automation which is really underpinned by back door processes, sometimes even human? Ask your vendor if they need remote access to the AI tool to deliver on their promise or service, and then make sure you understand exactly what that access is used for. Look for evidence of happy long-term customers that prove the AI tool’s claimed efficacy and value. Vendor case studies are nice, but independent verified reviews and user communities provide even more powerful sources of feedback. Testing a prospective AI tool in your own organisation is the gold standard to understand how effective it could be for you. Look for evaluation or proof of concept programs that allow you to get hands on, in your environment, and consider a competitive back off between alternatives.