Life-critical medical devices require robust safety and efficacy to treat patient populations with
potentially large inter-patient and intra-patient variability. Today, the de facto standard for evaluating medical devices is the randomized clinical trial. However, even after years of device development many clinical trials fail. For example, in the Rhythm ID Goes Head to Head Trial (RIGHT) the risk for inappropriate therapy actually increased relative to control treatments.
With recent advances in physiological modeling and devices incorporating more complex software components, population-level device outcomes can be obtained with large-scale simulations. Consequently, there is a need to explore alternative approaches to evaluate devices within a clinical trial context.

This work presents a framework to utilize computer modeling and simulation to improve the evaluation of medical device software, such as the algorithms in  implantable cardioverter defibrillators (ICD). Within this framework, virtual cohorts are generated and combined with real data to evaluate the efficacy of ICD algorithms while also quantifying the uncertainty due to the simulation. Results predicting the outcome of RIGHT and the improvement in statistical power while reducing the sample size are presented. Next, an approach to improving the performance of the device with Bayesian optimization is presented. Devices can degrade in performance when deployed to populations initially excluded in a clinical trial. For example, ICDs have shown increased rates of inappropriate therapy in patients with congenital heart disease. Bayesian optimization can be used to automate the adjustment of device parameters and fine-tune performance for a given cohort with minimal intervention. Our approach  identifies parameters which improve the performance of the device with outcomes aligned with the Multicenter Automatic Defibrillator Implantation Trial–Reduce Inappropriate Therapy (MADIT-RIT) clinical trial.