How to Add Machine Learning Based A/B testing Techniques to Your Reliability Engineering Toolkit

How to Add Machine Learning Based A/B testing Techniques to Your Reliability Engineering Toolkit by Dr. Ananth Narayanan $300 per person Group discounts available 3 hours 03122025

Course Abstract: Reliability practitioners predominantly use frequentist statistical A/B testing techniques. This webinar will enable them to add a variety of Bayesian Machine Learning (ML) techniques that can significantly enhance their DfX process. As reliability practitioners our job is to make risk-based, data driven decisions. Every day we are expected to answer a variety of questions. Is tool A better than tool B? Is one coating technique more corrosion resistant than the other? Design engineers want us to select the best diffusion barrier and the finance team wants to know if product X has a higher warranty cost than product Y. To answer these questions, we perform several tests and analysis which fall under the broad umbrella called A/B testing. Although the name suggests that this is a binary selection technique (A Vs B), in the real world we may have multiple factors with multiple outcomes that may either be categorical or continuous. Categorical outcomes are discrete and finite. For instance, the outcome of a mission reliability experiment can be a pass or a fail. Predicting the failure mode(s) in a system that has multiple competing failure modes is an example where multiple categorical outcomes are possible. One could also envision cases where the outcome is continuous, for example, time to failure for a product will yield a continuous distribution. Reliability practitioners widely use inferential statistical techniques for A/B testing. For classification problems where the outcomes are categorical, a Chi square test is widely used. For regression A/B testing where the outcomes are continuous, Student’s t-test is used. If the variances are unequal, Welch t-test is used and if the distributions are not Gaussian, other non-parametric tests like Mann-Whitney U test, Kruskal–Wallis test are used. We will briefly review some of these techniques and their applications. These techniques can be extremely powerful because they enable batch reliability testing and are straightforward to implement. But they are not without limitations. We will discuss the shortcomings of these tests using simulated data and justify the need for alternative adaptive techniques. We will then introduce the concept of ML based Bayesian approach to A/B testing. Central to the idea of this technique is the explore-exploit conundrum or the armed-bandit problem. The armed bandit problem is a classical problem in the field of machine learning and decision making. It is a scenario in which a decision maker must choose between multiple actions to maximize a certain objective, while receiving feedback about the quality of each action over time. The goal is to find a balance between exploration (trying out different actions to learn about their quality) and exploitation (focusing on actions that have shown to be successful in the past) to achieve the highest possible reward over time. Before discussing specific techniques, we will have a brief overview of Bayes theorem, conjugate priors, Bernoulli, Gamma, and other relevant distributions. We will review three Bayesian ML A/B testing techniques in detail in this webinar: 1. Epsilon Greedy 2. UCB1 3. Thompson sampling We will discuss the theory and implementation techniques for each of these algorithms and show how reliability practitioners can effectively use them in the DfX process.

We will then discuss real life engineering case studies where these techniques can be implemented. We will also discuss some caveats and disadvantages of these techniques and discuss when and when not to use them. Finally, we will discuss how to overcome tactical hurdles that reliability practitioners may face while embracing these techniques.

Your instructor: Dr. Ananthakrishnan (Ananth) Narayanan is a subject matter expert in semiconductor/hardware reliability engineering. He is currently the Principal for Worldwide Hardware Reliability at Lenovo where he directs all technical aspects of hardware reliability for the server division. Dr. Narayanan has made several noteworthy inventions in the field of hardware reliability, he co-invented a novel thermoelectric platform which is a key enabler for non hermetic packaging in  opto-electronic components. He was also the design for reliability leader for the world’s first ever medical grade solid state refrigeration system. Dr. Narayanan enjoys giving back to the reliability community. He is a voting member and a contributor of the reliability working group of IEEE standards committee, he has held several key leadership positions in various top tier reliability conferences. He is also an author and editor for leading journals in the reliability space. He lives in North Carolina with his wife and two boys.

https:hobbsengr.com/contact/

To purchase this webinar recording link, contact us with the link above, call 303 655-3051 or email [email protected]