Math Modeling Seminar - Modeling movement and input of plastic pollution in Lake Erie

Third-Year Work in Progress Presentation:Modeling movement and input of plastic pollution in Lake Erie Sophie KotokMathematical Modeling Ph.D. Student3rd YearAdvisor: Dr. Nathan CahillAbstract:Inductive transfer learning for deep neural networks assumes that pre-training a model on a source task extracts generalizable knowledge relevant to a different, but related, target task. The most widely used technique for inductive transfer learning is fine-tuning of models that have been pre-trained on very large datasets. In a sense, the pre-trained model provides an initial guess at the parameters of the target model when a lack of sufficient data would otherwise lead to severe overfitting if the parameters were randomly initialized. While this approach has enabled these models to achieve good results on small datasets, overfitting can still be a problem.To avoid loss of generalization, existing alternatives to standard fine-tuning regularize the target model by assuming that there exists an optimal set of parameters for the target task that is “close” to the pre-trained source model, and imposing a penalty in the loss function to enforce this nearness. Given recent results demonstrating that networks exhibit paths of nearly flat loss connecting local minima, it may not be reasonable to assume that a good solution to the target task is close to the particular minimum that was found when the source model was pre-trained. We explore different ways of encoding this relaxed assumption in terms of penalties / regularization strategies.Speaker Bio:Sophie is a third year Mathematical Modeling PhD student. A native of Colorado, she received her B.A. in mathematics from Whitman College in Walla Walla, WA. As an advisee of Dr. Nathan Cahill, her research interests include transfer learning for deep neural networks, lifelong learning, and computer vision. She can often be found roaming the trails of Western NY on foot, regaling or nonplussing fellow ultra-endurance runners with recreational mathematical riddles and tidbits.Intended Audience:All Math Modeling Ph.D. students (of all year levels) are required to attend this seminar.