Data scientists play a vital role in operationalising machine learning. Within MLOps, data scientists are responsible for developing, evaluating and amending models as they move into production.

But what happens when it comes to software or platform knowledge, and integrating models into business systems?  

 

A successful MLOps team absolutely needs data scientists – but that isn’t all. If you want to deploy ML models successfully, you’ll need  a cross-functional team of experts who provide a number of important skills. An ideal MLOps team structure should include:  

• Data scientists to create, build and amend the model
• Platform or machine learning engineers to provide an environment to host the model
• Data engineers to create the production data pipelines to retrain the model
• Software engineers to integrate the model into business systems

It’s worth noting that in some smaller organisations, these roles might be part-time and performed by one person, or one person might fulfil more than one role. In larger organisations, there could be multiple people providing each function.

No matter the size of your team, it’s important that everyone has an idea of the responsibilities and requirements of other team members and roles. As your model moves from prototype to production, it’s important everyone understands the concerns of other team members, and the format and type of information that needs to be provided.

Building a cross-functional team means that your MLOps development benefits from a broader skillset. The more your team members understand the skills of the wider team, the more effectively they can work together.

• Engineers should recognise that the most pressing concern for data scientists is prototyping, experimentation and algorithm performance evaluation.
• Data scientists often need to learn more about software development practices, and the separation of environments such as development, staging and production.

Ultimately, the goal of a cross-functional team is to create a clear framework that takes models through the entire development and production process. This framework should be built into the CI/CD framework. Create a simple document and spend a session taking data scientists through the development process that you have chosen. When the team forms, recognise that it is one team and organise yourself accordingly. Backlogs and stand-ups should be owned by and include the whole team.

If you’d like to know more about building effective MLOps team structures and operationalising machine learning, download our recent playbook, which is packed with insights into building successful MLOps projects and getting them into production.

 

If you’re new to the world of MLOps, here’s what you need to know: MLOps (which stands for machine learning operations) is a set of tools and ideas that help data scientists and operations teams to develop, deploy and monitor models in the AI world.

That’s a big deal because organisations that want to deliver AI projects often struggle to get projects off the ground at scale, and to deliver effective return on investment (ROI). Using MLOps helps those organisations to create machine learning models in a manner that is effective, consistent and scalable.  

Over the last decade, machine learning has become a critical resource for many organisations. Using ML models, companies can create models that can analyse vast quantities of structured and unstructured data, making predictions about business outcomes that can be used to inform faster, better decisions. The challenge, increasingly, is how those organisations monitor and manage multiple ML models and iterations.

MLOps brings discipline and structure to AI 

That’s where MLOps comes in. While DevOps focuses on how systems are developed with regard to security, compliance and IT resource management, MLOps focuses on the consistent development of scalable models. Blending machine learning with traditional devops models creates an MLOps process that streamlines and automates the way that intelligent applications are developed, deployed and updated.

Examples of how MLOps is being used include:

• Telecoms – using MLOps systems to manage network operations and customer churn models.
• Marketing – in advertising, MLOps is being used to manage multiple machine learning models in production to present targeted ads to consumers.
• Manufacturing – Using machine learning models to predict asset maintenance needs and identify performance and quality problems.

With MLOps, Data scientists can place models into production, then monitor and record their performance to ensure they’re working well. With MLOps they can also capture information on all ML models in a standard form that allows other teams to use those models or revise them later.

How MLOps can deliver  higher ROI 

This isn’t just about making life easier. We know that 90% of AI projects fail under current development frameworks. MLOps provides a far more reliable, cost-effective framework for development that can deliver successful projects much more quickly. By adopting MLOps, it becomes easier for organisations to make the leap from small-scale development to large-scale production environments. By increasing the speed and success of ML models being deployed, MLOps can improve the ROI of AI projects.

It’s also worth considering that models – by their nature – need to change. Once an ML model is created and deployed, it generally won’t continue operating in the same way forever. Models need to be constantly monitored and checked, to ensure they’re delivering the right insights and business benefits. MLOps helps data scientists to make faster inventions when models need to be revised – such as during a global pandemic or supply chain crisis – with changes deployed at a faster rate.   

If organisations want to adopt MLOps they must first build the relevant skills within data and operations teams. This includes skills such as full lifecycle tracking and a solid AI infrastructure that enables the rapid iteration of new ML models. These will need to support both main forms of MLOps – predictive (charting future outcomes based on past results) and prescriptive (making recommendations for future decisions).

Need more guidance? 

The key thing to understand about MLOps is that it can’t guarantee success, but it will lower the cost of experimentation and failure.

Ensuring you get the best results from MLOps isn’t always easy, and our MLOps Playbook is a good place to start for guidance on how to maximise the ROI and performance of models in your organisation. The playbook outlines the basic principles of effective MLOps, including creating solid data foundations, creating an environment where data scientists can create and the pitfalls to avoid when creating MLOps practices.

 

MLOps is still a fairly new concept in many organisations, but according to Cognilytica, the global MLOps market will be worth $4 billion by 2025. The industry is growing by around 50% each year, as organisations look to deliver more value from cutting-edge AI programmes. 

We asked our Equal Experts experts to provide answers to some of the most common questions about MLOps: 

What is MLOps and why do we need it?

MLOps is a culture and set of best practices that teams can follow to productionize machine learning models. We need MLOps because it can replace the old-fashioned approach of building models in a separate data science team, and then throw them over the wall to a software engineer team.

What is the difference between MLOps and DevOps? 

The difference between MLOps and DevOps is that MLOps is more far reaching as a framework. In many ways, MLOps can be seen as an extension of DevOps. Once a team productionizes machine learning models, some special roles are needed like Data Scientist and Machine Learning Engineers. MLOps gives guidance on how to integrate these roles, and how to handle the technology that is needed to develop and evolve the machine learning models.

How do you learn MLOps?

If you want to learn MLOps the good news is that there are plenty of courses and tutorials. Our advice is not to focus too much on the tools, but to learn the technology-independent best practices and patterns.

What does MLOps stand for? 

It stands for Machine Learning Operations.

Why don’t AI projects make it to production? 

Many companies don’t realise that a machine learning model only starts its life in a (Jupyter) notebook. Once you have built a model, it needs to evolve and become a part of your technology landscape. This phase of AI development is the most difficult and probably takes most of the time. If you are struggling this article might help.

When is the best time to employ MLOps? 

The best practices for MLOps can be applied to any project and team that is working with machine learning. There’s no one best time to employ MLOps. 

How do you deploy ML models into production? 

To deploy ML models into production, remember that an ML model should be treated as an artefact that needs to be versioned, governed, and managed. This artefact can be deployed to a system in a variety of ways. A common one is in the form of an API, where a wrapper is placed around it so the model can serve as a microservice.  Now the API can be published on the company network. Other options are to load the model artefact in memory for distributed stream processing, as a procedure in a database or to deploy the model on edge devices in the internet of things.

What are examples of ML models in daily life? 

Nowadays there are too many examples to mention. Some examples of ML models in daily life might include the social media feed you see using a machine learning model to present you content. The spam filter in your email is a machine learning model. The recommendations that a website presents to you. A self-driving car is packed with ML. Basically any system that works with data is likely to have an ML model integrated.

Why do ML models degrade in production?

ML models degrade in production because of a process called data drift. This means that the data that the model is using in production is different from the data that it was trained on. This is common, since in most use-cases system behaviours change over time, for example customer behaviours change frequently according to the season. Therefore it is important to keep retraining your models on recent data.

Are machine learning and AI the same thing? 

No, machine learning and AI aren’t the same thing. Machine learning is a subset of AI where algorithms are developed and trained on historical data, without being explicitly told how, to make predictions about new data. AI is mostly used in a broader sense, indicating systems that perform intelligent tasks in a human-like manner.

If you want to learn more about MLOps, we’ve written a playbook (pdf version), blogs and case studies which can all be found here. 

 

 

Our experience of working on AI and ML projects means that we understand the importance of establishing best practices when using MLOps to test, deploy, manage and monitor ML models in production.

Considering that 87% of data science projects never make it into production, it’s vitally important that AI projects have access to the right data and skills to solve the right problems, using the right processes.

Below, we outline six fundamental principles of MLOps that should be at the heart of your AI strategy.

1 – Build solid data foundations

Your data scientists will need access to a store of good quality, ground-truth (labelled) historical data. ML models are fundamentally dependent on the data that’s used to train them, and data scientists will rely on this data for monitoring and training.

It’s common to create data warehouses, data lakes or lake houses with associated data pipelines to capture this data and make it available to automated processes and data teams. Our data pipeline playbook covers our approach to providing this data. Make sure to focus on data quality, security, and availability.

2 – Provide an environment that allows data scientists to create

Developing ML models is a creative, experimental process. Data scientists need a set of tools to explore data, create models and evaluate their performance. Ideally, this environment should:

• Provide access to required historical data
• Provide tools to view and process the data
• Allow data scientists to add additional data in various formats
• Support collaboration with other scientists via shared storage or feature stores
• Be able to surface models for early feedback before full productionisation

3 – ML services are products

ML services should be treated as products, meaning you should apply the same behaviours and standards used when developing any other software product.

For example, when building ML services you should identify and profile the users of a service. Engaging with users early in the development process means you can identify requirements that can be built into development, while later on, users can help to submit bugs and unexpected results to inform improvements in models over time.

Developers can support users by maintaining a clear roadmap of features and improvements with supporting documentation, helping users to migrate to new versions and clearly explaining how versions will be supported, maintained, monitored and (eventually) retired.

4 – Apply continuous delivery of complex ML solutions

ML models must be able to adapt when the data environment, IT infrastructure or business needs change. As with any working software application, ML developers must adopt continuous delivery practices to allow for regular updates of models in production.

We advise that teams should use techniques such as Continuous Integration and Deployment (CI/CD), utilise Infrastructure as Code and work in small batches to have fast, reasonable feedback.

5 – Evaluate and monitor algorithms throughout their lifecycle

It’s essential to understand whether algorithms are performing as expected, so you need to measure the accuracy of algorithms and models. This will add an extra layer of metrics on top of your infrastructure resource measurements such as CPU and RAM per Kubernetes pod. Data scientists are usually best placed to identify the best measure of accuracy in a given scenario, but this must be tracked and evaluated throughout the lifecycle, including during development, at the point of release, and in production.

6 – MLOps is a team effort

What are the key roles within an MLOps team? From our experience we have identified four key roles that must be incorporated into a cross-functional team:

• Platform/ML engineers to provide the hosting environment
• Data engineers to create production data pipelines
• Data scientists to create and amend the model
• Software engineers to integrate the model into business systems

Remember that each part of the team has a different strength – data scientists are typically strong at maths and statistics, while they may not have software development skills. Engineers are often highly-skilled in testing, logging and configuration, while data scientists are focused on algorithm performance and accuracy.

At the outset of your project consider how your team roles can work together using clear, defined processes. What are the responsibilities of each team member, and does everyone recognise the standards and models that are expected?

To learn more about MLOps principles and driving better, more consistent best practices in your MLOps team, download our Operationalising Machine Learning Playbook for free.

When using MLOps, it’s easy to focus on the technical aspects of the project. But as we explain in our recently published Playbook, Operationalising Machine Learning, it’s vital that MLOps is based around user involvement at every stage – including some employees you might not expect. 

Back in 2014, tech giant Amazon built an internal ML University so that its in-house developers could keep their skills up to date. In 2022, developers still use the university – but so do product managers, program managers and a host of other business users from across Amazon.

What Amazon realised was that giving novice users an understanding of basic AI and ML ideas empowered those users to get involved with data teams, and resulted in better projects. Business users at Amazon play a collaborative role in developing a strong business case for ML models, driving solutions that will meet the needs of the business and its customers.  

Without this collaboration, ML teams risk building impressive prototypes that never get business buy-in, or don’t have real world customer impact. That’s something to think about considering that IDC reports that 47% of AI projects never get past the initial experimentation phase, and 28% projects simply fail. 

The importance of user involvement in MLOps 

MLOps can help the success of AI projects by providing a structured framework for moving ML models from development through to production and management. Where and how should data teams start to build user involvement into this process? 

Here are five ways you should involve end users in your MLOps process:  

Step 1: Ask users for input before development starts 

A common pitfall when surfacing a new machine learning score or insight is that end users don’t understand or trust new data points. This can lead to them ignoring the insight – no matter how useful it is.

This can be avoided by involving users at the very start of an ML development. What problem does the user expect the model to solve for them? Use this insight to guide the initial investigation and analysis.   

Step 2: Demonstrate and Iterate 

Once development starts, make a point of demonstrating model results to users as part of the iterative model development – take users on the journey with you. This is an opportunity to gain early feedback that can help guide development of models that will deliver real benefit to the business and its customers. Data teams should surface ML models for early feedback from users before full productionisation. Tools such as Streamlit and Dash can help to prototype and share models with end users. 

Step 3: Focus on explainability 

As the model nears completion, ensure that you have something that can be explained – this may be the model itself, or how it arrives at a recommendation or insight. 

If you’re building a model that will provide an insight into a credit risk score, you might need to explain what data is being used to drive the insight, and how this insight can be applied within the business user’s regular process of processing a loan application, for example. 

Step 4: Monitor your users’ experience 

Once a model is live, make sure users are involved in testing, and can provide feedback on any bugs or faults they experience. Consider also using telemetrics for monitoring, so that you can monitor performance of the model and be alerted in case of any issues. You should consider sharing these metrics with business users where appropriate. 

These steps will help to build and maintain user trust in the model, and increase the likelihood that the results generated by ML will be adopted as intended. 

Step 5: Adopt continuous improvement 

When an ML model is in production, you will almost certainly continue to improve the service throughout its lifetime. To maintain high levels of user involvement, capture iterations of your service as versions, and help users to migrate to newer versions. 

It’s important to provide good, current user documentation and regularly test how models appear from the user’s perspective. Finally, when you retire a service, have a clear process and ensure that users are supported if a model will no longer be supported. 

Summary 

We believe that ML services should be developed and treated as a product, meaning organisations should apply the same behaviours and standards that would be used when developing any other software product. 

When developing an ML model, it is essential to identify, profile and maintain an active relationship with the end users of your ML service. Work with users to identify requirements that feed into your development backlog, involve users in validating features and improvements, and notify them of updates and outages. In doing so, you will secure buy-in from business users and increase the odds of the AI project delivering real business value. 

In our recent Operationalising ML Playbook we discussed the most common pitfalls during MLOps. One of the most common pitfalls? Failing to implement appropriate secure development at each stage of MLOps. 

Our Secure Development playbook describes the practices we know are important for secure development and operations and these should be applied to your ML development and operations.

In this blog we will explore some of the security risks and issues that are specific to MLOps. Make sure you check them all before publishing your model into production. 

In machine learning, systems use example data to try to learn something – which may be output as a prediction or insight. The examples used to train ML models are known as training datasets, and security issues can be broadly divided into those affecting the model before and during training, and those affecting models that have already been trained. 

Vulnerability to data poisoning or manipulation  

One of the most commonly discussed security issues in MLOps is data poisoning – this is an attack where hackers attempt to corrupt or manipulate the data used for training ML models. This might be by switching expected responses, or adding new responses into a system. The result of data poisoning is that data confidentiality and reliability are both damaged.  

When data for ML models is collected from online sources from sensors or online sources, the risk of data poisoning can be extremely high. Attacks can include label flipping (data is poisoned by changing labels in data) and gradient descent attacks (where the ability of a model to understand how close it is to predicting the correct answer is damaged by either making the model falsely believe it’s found the answer, or by preventing it from finding the answer by constantly changing that answer). 

Exposure of data in the pipeline

You will certainly need to include data pipelines as part of your solution. In some cases they may use personal data in the training. Of course these should be protected to the same standards as you would in any other development. Ensuring the privacy and confidentiality of data in machine learning models is critical to protect against data extraction attacks and function extraction attacks. 

Making the model accessible to the whole internet

Making your model endpoint publicly accessible may expose unintended inferences or prediction metadata that you would rather keep private. Even if your predictions are safe for public exposure, making your endpoint anonymously accessible may present cost management issues. A machine learning model endpoint can be secured using the same mechanisms as any other online service.

Embedding API Keys in mobile apps 

A  mobile application may need specific credentials to  directly access your model endpoint. Embedding these credentials in your app allows them to be extracted by third parties and used for other purposes. Securing your model endpoint behind your app backend can prevent uncontrolled access.

As with most things in development, it only takes one person to neglect MLOps security to compromise the entire project. We advise organisations to create a clear and consistent set of governance rules that protect data confidentiality and reliability at every stage of an ML pipeline. 

Everyone in the team needs to agree on the right way to do things – it only takes one leak or data attack for the overall performance of a model to be compromised. 

 

Despite huge adoption of AI and machine learning (ML), many organisations are still struggling to get ML models into production at scale.

The result is AI projects that stall, don’t deliver ROI for years, and potentially fail altogether. Gartner Group estimates that only half of ML models ever make it out of trials into production. 

Why is this happening? One of the biggest issues is that companies develop successful ML prototype models, but these models aren’t equipped to be deployed at scale into a complex enterprise IT infrastructure. 

All of this slows down AI development. Software company Algorithmia recently reported that most companies spend between one and three months deploying a new ML model, while one in five companies took more than three months. Additionally, 38% of data scientists’ time is typically spent on deployment rather than developing new models. 

Algorithmia found that these delays were often due to unforeseen operational issues. Organisations are deploying models only to find they lack vital functionality, don’t meet governance or security requirements, or need modification to provide appropriate tracking and reporting. 

How MLOps can help 

Enter MLOps. While MLOps leverages DevOps’ focus on compliance, security, and management of IT resources, MLOps add much more emphasis on the consistent development, deployment, and scalability of models. 

Organisations can accelerate AI adoption and solve some of their AI challenges by adopting MLOps. Algorithmia found that where organisations were using MLOps, data scientists were able to reduce the time spent on model deployment by 22%, and the average time taken to put a trained model into production fell by 31%. 

That’s because MLOps provides a standard template for ML model development and deployment, along with a clear history and version control. This means processes don’t need to be reinvented for each new model, and standardised processes can be created to specify how all models should meet key functional requirements, along with privacy, security and governance policies. 

With MLOps, data teams can be confident that new code and models will meet architecture and API requirements for production usage and testing. By removing the need to create essential features or code from scratch, new models are faster to build, test, train and deploy. 

MLOps is being widely used for tasks such as automation of ML pipelines, monitoring, lifecycle management and governance. MLOps can be used to monitor models to sense any fall in performance or data drifts that suggest models might need to be updated or retrained.

Having a consistent view of ML models throughout the lifecycle in turns allows teams to easily see which models are live, which are in development, and which require maintenance or updates. These can be scheduled more easily with a clear overview of the ML landscape. 

Within MLOps, organisations can also build feature stores, where code and data can be re-used from prior work, further speeding up the development and deployment of new models. 

Learn more about MLOps 

Our new playbook, Operationalising Machine Learning, provides guidance on how to create a consistent approach to monitoring and auditing ML models. Creating a single approach to these tasks allows organisations to create dashboards that provide a single view of all models in development and production, with automated alerts in case of issues such as data drift or unexpected performance issues.

If you’re struggling to realise the full potential of machine learning in your organisation, the good news is that you’re not alone. According to industry analysts VentureBeat, 87% of AI projects will never make it into production.

MLOps emerged to address this widespread challenge. By blending AI and DevOps practices, MLOps promised smooth, scalable development of ML applications.

The bad news is that MLOps isn’t an immediate fix for all AI projects. Operationalsing any AI or machine learning solution  will present its own challenges, which must be addressed to realise the potential these technologies offer. Below we’ve outlined five of the biggest MLOps challenges in 2022, and some guidance on solving these issues in your organisation.

You can read about these ideas in more detail in our new MLOps playbook, “Operationalising Machine Learning”, which provides comprehensive guidance for operations and AI teams in adopting best practice around MLOps. 

Challenge 1: Lack of user engagement

Failing to help end users understand how a machine learning model works or what algorithm is providing an insight is a common pitfall. After all, this is a complex subject, requiring time and expertise to understand. If users don’t understand a model, they are less likely to trust it, and to engage with the insights it provides.

Organisations can avoid this problem by engaging with users early in the process, by asking what problem they need the model to solve. Demonstrate and explain model results to users regularly and allow users to provide feedback during iteration of the model. Later in the process, it may be helpful to allow end users to view monitoring/performance data so that you can build trust in new models. If end users trust ML models, they are likely to engage with them, and to feel a sense of ownership and involvement in that process.

Challenge 2: Relying on notebooks

Like many people we have a love/hate relationship with notebooks such as Jupyter. Notebooks can be invaluable when you are creating visualisations and pivoting between modelling approaches.

However, notebooks contain both code and outputs, along with important business and personal data, meaning it’s easy to inadvertently pass data to where it shouldn’t be. Notebooks don’t lend themselves easily to testing, and cells that can run out of order means that different results can be created by the same notebook based on the order that cells are run in.

In most cases, we recommend moving to standard modular code after creating an initial prototype, rather than using notebooks. This results in a model that is more testable and easier to move into production, with the added benefit of speeding up algorithm development.

Challenge 3: Poor security practice 

There are a number of common security pitfalls in MLOps that should be avoided, and it’s important that organisations have appropriate practices in place to ensure secure development protocols.

For example, it’s surprisingly common for model endpoints and data pipelines to be publicly accessible, potentially exposing sensitive metadata to third parties. Endpoints must be secured to the same standard as any development to avoid cost management and security problems caused by uncontrolled access.

Challenge 4:  Using Machine Learning inappropriately

Despite the hype, ML shouldn’t always be the default way to solve a problem. AI and ML are essentially tools that help to understand complex problems like natural language processing and machine vision.

Applying AI to real-world problems that aren’t like this is unnecessary, and leads to too much complexity, unpredictably and increased costs. You could build an AI model to predict whether a number is even or odd – but you shouldn’t.

When addressing a new problem, we advise businesses to try a non-ML solution first. In many cases, a simple, rule-based system will be sufficient.

Challenge 5: Forgetting the downstream application of a new model   

Achieving ROI from machine learning requires the ML model to be integrated into business systems, with due attention to usability, security and performance.

This process becomes even longer if models are not technically compatible with business systems, or do not deliver the expected level of accuracy. These issues must be considered at the start of the ML process, to avoid delays and disappointment.

A common ML model might be used to predict ‘propensity to buy’ – identifying internet users who are likely to buy a product. If this downstream application isn’t considered when the model is built, there is no guarantee that the data output will be in a form that can be used by the business API. A great way to avoid this is by creating a walking skeleton or steel thread (see our Playbook for advice on how to do this).

Find out more about these challenges and more in our new Operationalising Machine Learning Playbook, which is available to read here.

Building a predictive model to forecast the future from historical data is standard practice for today’s businesses. But deploying, scaling and managing these models is far from simple.

Each ML solution depends on an algorithm (code) and a set of data used to develop and train the algorithm. For this reason, building ML solutions is different to other types of software development. 

Enter MLOps, or machine learning operations, a set of processes that help organisations to develop, deploy and monitor ML models at scale by applying best practices to infrastructure, code and data. 

MLOps is a relatively new idea but one that has been adopted by many organisations – the market for MLOps solutions is expected to reach $4 billion by 2025. At Equal Experts, we have been involved in developing and deploying AI and ML for a number of applications including to: 

• Assess cyber risk 
• Evaluate financial risk 
• Improve search recommendations for retail websites 
• Improve logistics and supply chains 

Key Terms used in MLOps

If you’re new to MLOps there are several important terms to be aware of:

• Machine learning (ML) – a subset of AI that involves training algorithms with data rather than developing hand-crafted algorithms. A machine learning solution uses  a data set to train an algorithm, typically training a classifier that says what type of thing  this data is (e.g. this picture is of a dog ); a regressor, which estimates  a value (e.g. the price of this house is £400,000.) or an unsupervised  model, such as generative ones  which can be used to write novel text (such as song lyrics).  
• Model – In machine learning a model is the result of training an algorithm with data, which maps a defined set of  inputs to outputs.  
• Algorithm – we use this term more or less interchangeably with model. (There are some subtle differences, but they’re not important and using the term ‘algorithm’ prevents confusion with the standard software engineering use of the term ‘data model’ – which is a definition of the data entities, fields, relationships etc  for a given domain, that is used to define database structures among other things.)
• Ground-truth data – a machine-learning solution usually needs a data set that contains the input data (e.g. pictures) along with the associated answers (e.g. this picture is of a dog, this one is of a cat)  – this is the  ‘ground-truth’.
• Labelled data – means the same as ground-truth data. 

How does MLOps work? 

We talk about MLOps as a set of processes that help data scientists to develop consistent, scalable ML models, and monitor their performance. To create and use these algorithms, you will usually follow these steps: 

Initial development of the algorithm – Developing a model is the first step in machine learning. Data scientists will identify or create ‘ground truth’ data sets and explore them. They will build and evaluate prototypes of the models, trying out different core algorithms and data transformations  until they arrive at  one which meets the business need.

Integrate/deploy the model – once the model has been built, it must be integrated into the business. This can be done in various ways depending on the consuming service. In modern architecture, models are commonly implemented as a standalone microservice and models are deployed by copying an approved version of the model into an operational environment. 

Monitor performance – All ML models need to be monitored to ensure they’re running and meeting demand, but also that the results of the model are accurate and reliable.

Update model – over time, models must be retrained to reflect new data, or improvements to the model. In this case, it’s important to maintain version control and to direct downstream services to the new model.  

Operationalising Machine Learning 

Our MLOps playbook brings together our experiences working with algorithm developers to build ML solutions. It provides a comprehensive overview of what you need to consider when providing the architecture, tools and infrastructure to support data scientists and to integrate their outputs into the business.

Download the playbook for expert guidance on how your organisation can attain  the promised business value from algorithms by providing engineering to support algorithm development, and by integrating ML more effectively into your business processes. You’ll find helpful advice on how to:

• Collect data that drives machine learning, and make that available to data scientists 
• Integrate algorithms into your everyday business 
• Configuration control, deploy and monitor deployed algorithms 
• Test and monitor the algorithms  

View our online version or download a pdf here.