Why machine learning projects fail

In the search engine, start typing “artificial intelligence will change” and you will see suggested links, such as “world”, “everything in your life” and “business face in the next decade”. Look a little further and it will become clear that AI projects and machine learning not only make progress, but are inseparable from their success. According to an Accenture study, 85% of capital industry executives say they will not achieve their growth goals if they do not expand AI.

At the same time, MIT Sloan’s research shows that the gap between organizations that value data science successfully and those that strive for it is widening. As we know, data science and machine learning are the engine behind AI applications because it is through data processing that AI learns how to interpret our world and respond as we want. If AI is to have a real impact on companies and their customers, companies need to take a new approach to machine learning. As the MIT Technology Review concludes: “The way we teach, AI is fundamentally flawed.”

Many articles in publications such as Towards Data Science and Open Data Science (see here and here) try to distinguish exactly why machine learning projects with good tooth combs and technical jargon help fail. These articles are great if you’re an information scientist, but not so useful if you’re a company trying to figure out why a conversation assistant or personalization campaign that you’ve spent thousands of dollars on has never worked.

The reality is that your machine learning project probably didn’t fail because you disrupted your approach to the data version or model layout. Most engineering training projects fail simply because companies did not have the right resources, experience, or strategy from the beginning. McKinsey’s 2021 report on the artificial situation confirms this, reporting that companies that see the greatest impact from AI adoption are following both core and AI best practices and spending more efficiently and effectively on AI than their peers.

Five common AI errors in enterprises

Through our work on ML projects for some of the world’s largest companies, Applause has identified an example of common errors that reduce efficiency, increase costs, delay deadlines – and ultimately lead to the failure of machine learning projects.

Common Error 1: Incorrect estimation of resources required to study ML algorithms

The first reason for the failure of machine learning projects is that companies are not ready to see them and are not equipped. According to a measurement study, 8 out of 10 companies find machine learning projects more difficult than expected because they ignore work that is properly incorporated into training models. This is why so many data science projects are being put into production; without a clear understanding of the resources and experience required, companies will either face insurmountable obstacles or burn out of their budgets due to inefficiency. One of the things they misjudge the most is the attempt to get the right training data – which leads us to the second most common error.

Common Mistake 2: Relying on data brokers to provide the same training data

Companies do not struggle to get education. Finally, there are numerous data vendors that sell educational artifacts in large quantities at low prices. The reason machine learning projects fail is because companies are struggling to make ends meet high quality educational information.

When purchasing the same information from vendors, companies do not get enough information for the needs of their machine learning project. To understand why, consider the example of an online exercise class provider that makes a personal digital trainer (PT). In order for a PT to be able to recognize a poor form and recommend improvement, it must be trained with information that goes beyond images of individuals in different exercise positions. It should also know how to recognize individuals at different levels of fatigue and sweating, wearing different clothes and having different fitness levels and experiences.

There are many other challenges with pre-packaged training packages, including:

  • There is no guarantee that the data will show a balance of age, gender, race, accents, etc., which is necessary to reduce bias.

  • The data is either not explained at all or is not interpreted in a way that makes sense for the algorithm

  • The data have not been tested for compliance with data standards required by global AI regulations, such as the draft European Artificial Intelligence Act (EU AIA).

  • Companies cannot be assured that proper data confidentiality and security measures have been observed, nor can they be guided by data protection.

In order to carry out truly successful machine learning projects, companies need to consider training data as something they need. sort outrather than source.

Common Mistake 3: Ignoring the extent to which AI development requires constant repetition

Purchasing information from vendors not only has a negative impact on the quality of training, but also makes the rest of the AI ​​learning process endlessly more difficult.

The study of ML algorithms is not a one-time process. As the study continues, developers will need to constantly request changes to the data collected as the needs of the data model become clearer. The reason is that learning the AI ​​algorithm is like trying to shop and cook at the same time: you may think you have all the necessary ingredients, but when you start cooking, you realize that you forgot the ingredient, one has to replace or replace. the balance of ingredients isn’t right – and you’ll need to change your recipe accordingly.

In machine learning, it is difficult to know exactly what information you need until you begin the algorithm learning process. You may realize that the training package is not large enough or that there are problems with data collection. Many data brokers have a strict reform policy – or offer no ability to change orders – leaving AI developers with information they can’t use and have no choice but to buy another set of training that meets their new requirements. This is a common barrier for many companies, which raises prices, shortens deadlines, and reduces efficiency. Ultimately, this is the main reason why machine learning projects fail.

Common Error 4: Failure to enter a QA test

Companies across all industries often do not integrate QA testing at all stages of the product development process. It is falsely considered an add-on, as a procedure to double-check that the product is working properly, as opposed to a tool that can be used to optimize the product by the iterative method.

One of the reasons for the failure of machine learning projects is that this approach to QA testing is not acceptable given the realities of AI development. Unlike traditional software development, you cannot isolate errors with a simple software update; but errors detected during the QA test phase can only be corrected by re-performing the entire process. If your AI is not working as expected, it is likely that there was a problem with the learning data or that the learning data distorted the model in the wrong direction. In any case, it means a return to the first stage and the processing of new educational artifacts.

Companies that do not integrate validation of results at all stages of the AI ​​development process do more for themselves. Instead of learning the algorithm with a huge set and then testing the AI, companies need to re-learn and test. Taking a quick and “mature” approach to testing will help reduce unnecessary costs, speed up schedules, and allocate resources more efficiently.

Common Mistake 5: Lack of a frequent review plan

The last reason machine learning projects fail is that companies celebrate their success too early.

AI projects never really get done. Even if the AI ​​experience fully meets the exact expectations and performance, it is still only learned about the data that reflects the society in its current state. The algorithm learns to make decisions based on ideas, dialogues, and images that are already changing. Think about natural language processing programs (NLP): they only know how to communicate because they were once trained in real conversations with people. Given that about 5,400 new words are created in English alone each year, NLP applications are declining very quickly.

If AI practices are to be useful to clients, they need to be retrained on a regular basis as social relationships, advances in technology, and terminology change.

How to ensure successful machine learning projects

What companies need is a software approach to AI development. Instead of thinking of each individual stage of the process as separate projects, companies should consider integrating them as part of a comprehensive program. AI development is a recurring and agile process in which teams must work in tandem, not a silo, where all are managed by the program manager with responsibility for the success of the program.

To learn more about how your company can apply an application approach to creating an AI experience that is truly beneficial to your customers, download our white paper: Building a Global Data Collection Program and ML / AI Quality.


Build a global data collection and quality program in AI / ML

AI development requires a special program. In this article, we’ll look at where current approaches to AI development go wrong and show why a software approach is the answer.

Want to see more of this?

Leave a Comment