To fully realise the potential of AI and machine learning (ML), organisations must prioritise critical factors such as model selection, optimisation, monitoring, scalability, and the definition of success metrics.
Incorporating AI and ML into business operations has become a critical priority for organisations aiming to remain competitive in a rapidly evolving landscape. However, for many organisations, harnessing the potential of these technologies in a meaningful way is still an unfulfilled reality. To address this challenge, I’ve examined some of the latest trends in MLops and developed actionable strategies that can help solve common ML engineering challenges.
As you might expect, generative AI models differ significantly from traditional machine learning models in their development, deployment, and operational requirements. This article explores differences in training, delivery pipelines, monitoring, scaling, and measuring model success, with a few key questions organisations should address to guide their AI/ML strategy.
Ultimately, by focusing on solutions, not just models, and by aligning MLops with IT and DevOps systems, organisations can unlock the full potential of their AI initiatives and drive measurable business impacts.
Foundations of MLops: Understanding the core differences
Like many things in life, to successfully integrate and manage AI and ML into business operations, organisations first need to have a clear understanding of the foundations. The first fundamental of MLops today is understanding the differences between generative AI models and traditional ML models.
Cost is another major differentiator. The calculations of generative AI models are more complex, resulting in higher latency, demand for more computer power, and higher operational expenses. Traditional models, on the other hand, often utilise pre-trained architectures or lightweight training processes, making them more affordable for many organisations. Recent JFrog research found that 14% of UK organisations are still avoiding the use of ML models altogether, with cost and complexity likely playing a key role in that hesitation. When determining whether to utilise a generative AI model versus a standard model, organisations must evaluate these criteria and how they apply to their individual use cases.
Model optimisation and monitoring: Tailoring for the use case
Optimising models for specific use cases is crucial. For traditional ML, fine-tuning pre-trained models or training from scratch are common strategies. GenAI introduces additional options, such as retrieval-augmented generation (RAG), allowing private data to provide context and ultimately improve model outputs. Choosing between general-purpose and task-specific models also plays a critical role. Do you need a general-purpose model, or can you use a smaller model trained for your specific use case? General-purpose models are versatile but often less efficient than smaller, specialised models built for specific tasks.
Model monitoring also requires distinctly different approaches for generative AI and traditional models. Traditional models rely on well-defined metrics like accuracy, precision, and an F1 score, which are straightforward to evaluate. In contrast, generative AI models often involve metrics that are a bit more subjective, such as user engagement or relevance. Good metrics for GenAI models are still lacking, and it comes down to the individual use case. Yet nearly half (47%) of UK organisations are skipping runtime security scans, highlighting a critical gap in how models are monitored in production. Assessing a model is very complicated and sometimes requires additional support from business metrics to understand if the model is acting according to plan. In any scenario, businesses must design architectures that can be measured to make sure they deliver the desired output.
Evolving practices in Machine Learning engineering
Traditional machine learning has long relied on open source solutions, from open source architectures like LSTM (long short-term memory) and YOLO (you only look once), to open source libraries like XGBoost and Scikit-learn. These solutions have become the standards for most challenges thanks to being accessible and versatile. Interestingly, the UK shows greater caution here, 38% of organisations restrict public software downloads, 10% above the global average, which may impact their ability to fully leverage these open source tools. For GenAI, however, commercial solutions like OpenAI’s GPT models and Google’s Gemini currently dominate due to high costs and intricate training complexities. Building these models from scratch means massive data requirements, intricate training, and significant costs.
Despite the popularity of commercial generative AI models, open-source alternatives are gaining traction. Models like Llama and Stable Diffusion are closing the performance gap, offering cost-effective solutions for organisations willing to fine-tune or train them using their specific data. However, open-source models can present licensing restrictions and integration challenges to ensuring ongoing compliance and efficiency.
Scaling smart: Building for performance and growth
As more and more companies decide to invest in AI, there are best practices for data management and classification, and architectural approaches that should be considered for scaling ML systems and ensuring high performance.
Leveraging internal data with RAG
Important questions revolve around data: What is my internal data? How can I use it? Can I train based on this data with the correct structure? One powerful strategy for scaling ML systems with GenAI is retrieval-augmented generation. RAG is the ability to use internal data to change the context of a general-purpose model. By embedding and querying internal data, organisations can provide context-specific answers and improve the relevance of GenAI outputs. For instance, uploading product documentation to a vector database allows a model to deliver precise, context-aware responses to user queries.
Designing for scalable MLops
Creating scalable and efficient MLops architectures requires careful attention to components like embeddings, prompts, and vector stores. Fine-tuning models for specific languages, geographies, or use cases ensures tailored performance. An MLops architecture that supports fine-tuning is more complicated, and organisations should prioritise A/B testing across various building blocks to optimise outcomes and refine their solutions.
Defining and measuring success
Aligning model outcomes with business objectives is essential. Metrics like customer satisfaction and click-through rates can measure real-world impact, helping organisations understand whether their models are delivering meaningful results. Human feedback is essential for evaluating generative models and remains the best practice. Human-in-the-loop systems help fine-tune metrics, check performance, and ensure models meet business goals.
In some cases, advanced generative AI tools can assist or replace human reviewers, making the process faster and more efficient. By closing the feedback loop and connecting predictions to user actions, there is an opportunity for continuous improvement and more reliable performance.
From models to solutions: A holistic approach
The success of MLops hinges on building holistic solutions rather than isolated models. Solution architectures should combine a variety of ML approaches, including rule-based systems, embeddings, traditional models, and generative AI, to create robust and adaptable frameworks.
Organisations should ask themselves a few key questions to guide their AI/ML strategies:
- Do we need a general-purpose solution or a specialised model?
- How will we measure success, and which metrics align with our goals?
- What are the trade-offs between commercial and open-source solutions, and how do licensing and integration affect our choices?
At its core, the message is clear: AI and ML are no longer about developing isolated models; they are about building holistic, interconnected solutions. These solutions consist of complex architectures where each component influences the overall user experience and success metrics derived from it. As MLops continues to mature, organisations must prioritise building scalable, metrics-driven systems. With the right blend of tools and strategic approaches, businesses realise the opportunity of AI and machine learning to foster innovation and achieve measurable business outcomes.
With over a decade of experience in data and machine learning, Yuval Fernbach is the co-founder and CTO of Qwak, leading the development of a fully managed ML platform that streamlines the entire machine learning lifecycle. Previously an ML Specialist at Amazon Web Services, Yuval has helped organisations design and deploy impactful machine learning solutions. Driven by a passion for innovation, Yuval specialises in leveraging data and technology to transform businesses.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.