ABOUT

If you have not lived in a cave during the last decade, you have probably heard the concept of a machine learning system at least once in a while. Whether it is auto-translation, auto-completion, face or voice recognition, recommendation systems or autonomous driving, AI-based systems can be found in almost every aspect of our daily lives.

Although the development of learning systems has become common among companies and a number of methodologies have been developed around them, there is still a lot of confusion around the deployment of those systems in a production environment - whose responsibility it is and most importantly who monitors those models once they are deployed.

In this lecture we will talk about the data science project cycle which holds five main stages - defining your project objectives, collecting and cleaning your data, training and testing a predictive model, deploying it in a production environment and monitoring its actions and decisions.

We will then concentrate on the last forgotten stage which is critical for DevSecOps teams and see why monitoring those systems is crucial for organizations using real-life examples from recent years of AI-based systems that went crazy when they were deployed without any supervision. One famous example was Tay, Microsoft’s Twitter chatbot, which was shut down only 16 hours after an organized trolling causing it to post offensive tweets.

We will examine different methods for measuring the success of different machine learning problems and focus on the clustering cyber-attacks problem, introducing the full architecture of monitoring a successful solution and its benefits.

By the end of this talk, you will understand why monitoring AI-based system matters and is an integral part of DevSecOps, the risks in leaving those systems unsupervised and the importance of data scientists and DevSecOps working closer together to ensure that those systems are fulfilling their true business purpose.