Research Projects

Research Project 1: LLMs in Multi-Agent Systems

This individual project will study how a multi-agent-based system can leverage large language models (LLMs) to enhance the negotiation process among the resources in the multi-agent system and, when needed, constrain it on human operator processes for generating robust and efficient process plans. Recent advances in LLMs have enabled new ways to interact with automated agents. This will allow for more naturalistic conversations between humans and machines to complete complex planning tasks together. The IRES scholar will investigate frameworks for developing advanced multi-agent systems, that can be used as a key tool for agent negotiations leveraging LLMs. The research outcomes can yield a shift towards better integrated and more flexible manufacturing systems, beyond state-of-the-art interaction methods, which are essential for modern, sustainable, and human-centered manufacturing solutions.

2025 IRES Scholar: Liam Ryan. UW Mentors: Danielsson and Bennulf, RIT Mentors: Alm and Sahin

Research Project 2: Deep Learning for Machine Recovery in Plug and Produce Production

In this project, the student will investigate and compare state-of-the-art deep learning architectures to capture the techniques and actions employed by skilled maintenance engineers when investigating and addressing unexpected stops in AI-controlled Plug and Produce. An important research question is what data from the control system, from existing machine-embedded and externally added sensors, including sensors worn by operators in the shape of gloves and glasses, are most valuable for generating robust and interpretable ML models that can later be queried by operators in need for guidance.

Research Project 3: Design of Traceability Signatures for AI-Supported Product Quality Assurance 

The quality of produced items partially depends on their production history–which different machines in an automated production line have affected the item, and the traceability of this history in the production process. The IRES scholar will design and compare ML-supported quality inspection methods that integrate AI in a generalizable traceability signature process. UW’s Production Technology Center has a sensor infrastructure that will be used for this project. This will include studying how traceable historical features correlate with product quality issues, and how an energy-efficient ML system can filter configurations to provide early warnings or, alternatively, use interpretable graph-based optimization to redesign a production line flexibly and efficiently.

2025 IRES Scholar: Calvin Nau. UW Mentor: Sikström, RIT Mentor: Shi.

Research Project 4: Real-Time ML-based Deviation Control

In additive manufacturing, production deviations in real-time remain an open research issue that has yet to be successfully automated. The IRES scholar will extend research at UW on anomaly detection using a hybrid ML approach with both reinforcement and supervised learning, and combine historical and real time data. Furthermore, the project will leverage federated ML to enhance the collaborative aspects of the anomaly detection. Federated ML enables integrating multiple local sources to collectively improve performance, without sharing sensitive data, in a joint approach to address production deviations.

2025 IRES Scholar: David Millard. UW Mentor: Ali, RIT Mentors: Baheri, Shi, and Alm

Research Project 5: AI in Human Resource Management

AI offers new opportunities to manage employees and strengthen organizational performance. Employee occupational safety and well-being in AI-automated environments is an essential social sustainability factor in industry. The IRES scholar will study ML methods to forecast and prevent safety incidents (avoiding accidents, minimizing mental health-related sick-leaves), including in human re-skilling, using data sourced from privacy-preserving infrastructure (shop floor, office environments) that comply with regulation, and its integration with an LLM-based architecture able to learn continuously that generates cues for safety-promoting workplace practices.

Research Project 7: Intention Detection of Operators Active near and inside Robot Cells in Industrial Production Environments

Traditional speed and separation monitoring (SSM) approaches have limitations, reacting to present conditions without recording prior states or anticipating future states. Consequently, even human-centered automation requires human adaptation to machines by avoiding triggering unnecessary safety responses. Collaborative robots should behave flexibly, maintaining work efficiency by altering behavior in a proactive and context-aware manner. At the same time, flexible robot behaviors must not compromise operator safety, imposing a risk on the use of uninterpretable models to make safety decisions. Interpretable-by-design models (ad-hoc) may degrade model accuracy, whereas retroactive interpretation of model behaviors (post-hoc) may degrade interpretation accuracy. In this work, we investigate the expansion of classical SSM to maintain a state history and use a machine learning (ML) model of operator behavior to anticipate future states. Based on these predictions, the robot alters behaviors or adjusts SSM constraints to minimize downtime. We investigate the interpretability of our predictive models through existing ML interpretability techniques to quantify the costs imposed by the use of ad-hoc versus post-hoc methods. The project explores two questions: RQ1: How can we improve the efficiency of classical SSM implementations by altering planning and adapting parameterization based on a model of anticipated operator behavior? RQ2: What are the efficiency and performance costs imposed by the use of interpretable anticipatory models versus post-hoc interpretability analysis of anticipatory models?

2026 IRES Scholar: Will Coffey. UW Mentors Danielsson and Bennulf, RIT Mentors: Sahin and Alm

Research Project 8: Visual Perception and Attention in AR

This research project investigates how guided visual attention affects memory recall in immersive XR environments. Established findings demonstrate that subtle gaze direction can improve spatial memory recall, motivating the exploration of whether attention guidance mechanisms systematically influence participants’ memory for visual properties such as object color, shape location, and temporal sequencing. Color appearance phenomena (simultaneous contrast, crispening, the Helmholtz-Kohlrausch effect, the Stevens effect, the Bartleson-Breneman effect, etc.) will be used to guide attention. Participants will experience controlled virtual scenarios using Unreal Engine where visual cues direct their attention to specific objects or regions. Their ability to accurately recall spatial arrangements, temporal order of presented stimuli, and the object's color will be tested. Our primary research questions examine: (1) how attention guidance via color appearance phenomena impacts task performance, (2) whether attention guidance systematically influences color memory recall, (3) how guided versus unguided attention affects spatial and temporal memory accuracy. This research could help design XR interfaces that naturally guide users to critical information while maintaining naturalness and avoiding distraction, including in Industry 5.0 contexts.

2025 IRES Scholar: Emily Rooney. UW Mentors: Pederson and Sjölie, RIT Mentors: Murdoch, Thorstenson, and Bailey.

Research Project 9: Visual Perception and Attention in AR

We propose a framework for accelerating few-shot robotic grasp learning on novel assembly-line objects by integrating large vision-language models (LVLMs), graph-based object similarity modeling, and reinforcement learning with LVLM-based oracle feedback. First, we will investigate whether LVLMs can be used to generate synthetic grasping annotations, task descriptions, and training examples from existing grasping datasets, enabling improved performance in low-data grasping scenarios without requiring new human-subject data collection. Second, we will characterize objects using distinctive geometric, semantic, material, and affordance-related features, then evaluate whether graph-based clustering can group objects according to similar optimal grasp locations, grasp types, and manipulation strategies. These clusters will support rapid identification of plausible grasping approaches for previously unseen parts and enable transfer learning from related objects. Third, we will develop a reinforcement learning approach that incorporates LVLMs as uncertainty-aware oracles, allowing the system to request additional synthetic labels, grasp evaluations, or corrective signals when confidence falls below a defined threshold. Initial experiments will use established grasping datasets before extending to additional benchmark or testbed-derived object data. By combining LVLM-generated supervision, similarity-based transfer, and oracle-guided reinforcement learning, this work aims to reduce the data requirements of robotic grasp training while improving adaptability to novel industrial components.  

2026 IRES Scholar: Bryce Gernon. UW Mentors: Ramasamy and Sjölie, RIT Mentors: Sahin and Alm

Research Project 10: Beam-Shaping Reinforcement Learning for Melt-Pool Distribution Matching in Laser Welding

In laser beam welding, weld quality depends strongly on melt-pool geometry, which determines weld penetration, fusion, and porosity. Reproducing a target pool is key to optimizing a weld, but is extremely challenging, as along a seam, heat accumulates, so beam shapes and powers that reproduce the target melt-pool at one point no longer reproduce the melt-pool downstream. That is, a fixed setting is unlikely to replicate the target melt-pool geometry. We recast this welding problem as a constrained Markov Decision Process along the seam, where a learned policy observes the current melt pool and then chooses beam shape, power, and speed at the next step restricted to a physically safe process window, with deposited energy determining the next melt-pool state. We score reproduction through the Wasserstein distance between the produced and target melt-pool fields, which therefore rewards distribution matching over pointwise or scalar error. Prior work has either shaped the beam with a model-based controller toward a scalar target, or applied reinforcement learning to laser power alone; our method instead learns a policy that pursues a distributional objective and whose actions include the beam shape. The policy is trained in a differentiable thermal surrogate and validated on a deformable-mirror welding platform, with ablations isolating each contribution: the optimal-transport objective, the beam-shape action, and the learned policy. We also study an extension in the multi-laser regime where thermally coupled sources share a common thermal field formulated as a cooperative multi-agent problem.

2026 IRES Scholar: Chirayu Salgarkar. UW Mentors: Sikström and Mi, RIT Mentors: Baheri, Bailey, and Shi