How to Migrate an Algorithm to an ML Model

By starting here with an existing algorithm, we bypass the need to start with a business objective as gate criteria. At the end of this blog, we discuss what to do if no legacy algorithm exists. In that case, the importance of having a business objective and acceptance criteria cannot be overstated.

Start With Data

ML models need lots of data, and the cost of ML projects is driven by data availability. If you already have an algorithm, then you should be able to produce lots of data very quickly. Create a test driver that sends exhaustive amounts of input data to the algorithm and capture the results. This becomes the “supervised training data” for your new ML model.

Train Your ML Model

Training an ML model nowadays is surprisingly simple. You don’t have to write any complex code, you don’t need to run a compiler or a builder, and you don’t need to hire a data scientist. There are a variety of low-code and no-code options for implementing ML today, both commercial and open-source. Here is a resource that describes some of the models that are commonly available. Alternatively, your cloud provider offers APIs with pre-trained models for a variety of use cases and plenty of training and documentation on how to use them.

Validate the Model

How do you know if it worked? You typically hold back 20 percent of your training data to use as a “validation set.” After you train the model with 80 percent of the data, then you send the “validation set” through the newly trained model as a “quiz” to see how well the model performs. You already have the correct answer, so you can automatically check whether the newly trained model got it right. Use these results to calculate the accuracy. If the model has decent accuracy, then you have a starting point to start replacing your algorithm.

The point here is that you don’t have to spend 1,000s of hours of QA time trying to validate that the ML model is working. You don’t need a QA person trying to design test cases for black box, glass box, edge case, boundary case, happy case, etc. Just evaluating the “Data In — Data Out” aspect of the model is sufficient.

Deploying the ML Model

Initially, you will run the newly trained model in parallel with your production algorithm. You will also audit some subset of the results to determine the “correct” answer. (You can audit manually. Or, if your algorithm is good enough, you can use the algorithm output directly as the “correct” answer.) Any of your audited data – especially the records where the model failed to find the correct answer – should be saved as additional “supervised training data” for the next time you train a model.

Wait – what next time? Yes, you are going to want to train another model. ML is an iterative process, not a “one-and-done.” Initially you will be auditing new data and training new models every day, or multiple times per day. And each time you train a new model, you will have more training data, which represents more use cases. So your model should get smarter and more accurate over time. Once you reach your target accuracy, then you can decrease your frequency of auditing and retraining to once per week, or once per month; just enough to maintain the desired accuracy. But for most use cases, you will never reach a point where you can just stop training new models, because you will always need to account for new data values or new patterns.

Note: Training models is not necessarily deterministic. (That is, two models trained with the same dataset will not necessarily yield identical results.) So you always need to validate your new model’s accuracy, even if it was trained the same (or better) data as before.

Eventually, when your model starts to outperform your algorithm (in terms of accuracy), then you can retire the algorithm, and just run the production data through the ML model.