How to Add Machine Learning Based A/B testing Techniques to Your Reliability Engineering Toolkit by Dr. Ananth Narayanan
Join us for our next webinar! October 9, 2024 8:30 – 11:30 AM Pacific time (California) Cost $300 per person, Group discounts available! This course will be recorded and available for purchase to view at a time more convenient for you. Contact us for more details.
Register here and pass this on to your co-workers!
Course Abstract: Reliability practitioners predominantly use frequentist statistical A/B testing techniques. This webinar will enable them to add a variety of Bayesian Machine Learning (ML) techniques that can significantly enhance their DfX process. As reliability practitioners our job is to make risk-based, data driven decisions. Every day we are expected to answer a variety of questions. Is tool A better than tool B? Is one coating technique more corrosion resistant than the other? Design engineers want us to select the best diffusion barrier and the finance team wants to know if product X has a higher warranty cost than product Y. To answer these questions, we perform several tests and analysis which fall under the broad umbrella called A/B testing. Although the name suggests that this is a binary selection technique (A Vs B), in the real world we may have multiple factors with multiple outcomes that may either be categorical or continuous. Categorical outcomes are discrete and finite. For instance, the outcome of a mission reliability experiment can be a pass or a fail. Predicting the failure mode(s) in a system that has multiple competing failure modes is an example where multiple categorical outcomes are possible. One could also envision cases where the outcome is continuous, for example, time to failure for a product will yield a continuous distribution. Reliability practitioners widely use inferential statistical techniques for A/B testing. For classification problems where the outcomes are categorical, a Chi square test is widely used. For regression A/B testing where the outcomes are continuous, Student’s t-test is used. If the variances are unequal, Welch t-test is used and if the distributions are not Gaussian, other non-parametric tests like Mann-Whitney U test, Kruskal–Wallis test are used. We will briefly review some of these techniques and their applications. These techniques can be extremely powerful because they enable batch reliability testing and are straightforward to implement. But they are not without limitations. We will discuss the shortcomings of these tests using simulated data and justify the need for alternative adaptive techniques. We will then introduce the concept of ML based Bayesian approach to A/B testing. Central to the idea of this technique is the explore-exploit conundrum or the armed-bandit problem. The armed bandit problem is a classical problem in the field of machine learning and decision making. It is a scenario in which a decision maker must choose between multiple actions to maximize a certain objective, while receiving feedback about the quality of each action over time. The goal is to find a balance between exploration (trying out different actions to learn about their quality) and exploitation (focusing on actions that have shown to be successful in the past) to achieve the highest possible reward over time. Before discussing specific techniques, we will have a brief overview of Bayes theorem, conjugate priors, Bernoulli, Gamma, and other relevant distributions. We will review three Bayesian ML A/B testing techniques in detail in this webinar: 1. Epsilon Greedy 2. UCB1 3. Thompson sampling We will discuss the theory and implementation techniques for each of these algorithms and show how reliability practitioners can effectively use them in the DfX process.
We will then discuss real life engineering case studies where these techniques can be implemented. We will also discuss some caveats and disadvantages of these techniques and discuss when and when not to use them. Finally, we will discuss how to overcome tactical hurdles that reliability practitioners may face while embracing these techniques.
Your instructor: Dr. Ananthakrishnan (Ananth) Narayanan is a subject matter expert in semiconductor/hardware reliability engineering. He is currently the Principal for Worldwide Hardware Reliability at Lenovo where he directs all technical aspects of hardware reliability for the server division. Dr. Narayanan has made several noteworthy inventions in the field of hardware reliability, he co-invented a novel thermoelectric platform which is a key enabler for non hermetic packaging in opto-electronic components. He was also the design for reliability leader for the world’s first ever medical grade solid state refrigeration system. Dr. Narayanan enjoys giving back to the reliability community. He is a voting member and a contributor of the reliability working group of IEEE standards committee, he has held several key leadership positions in various top tier reliability conferences. He is also an author and editor for leading journals in the reliability space. He lives in North Carolina with his wife and two boys.
After registering, you will receive a confirmation email containing information about joining the webinar. NOTE: if you do not receive the confirmation email, please call us at 303-655-3051
COST: $300 per person, $270 per person if you have five or more from the same company
View System Requirements
