Imaging Science Seminar: Manipulating Object Appearance with Augmented Reality

Manipulating Object Appearance with Augmented Reality

Dr. Michael Murdoch
Assistant Professor
Integrated Sciences Academy, RIT

Abstract:
Recent and ongoing work has addressed the possibility of manipulating the appearance of real objects with optical overlays, using an optical see-through (OST) augmented reality (AR) system. The OST-AR system uses a large beamsplitter to overlay a displayed image onto the viewer’s visual field, physically adding light to the scene in a manner conceptually similar to both head-mounted AR systems and projection-mapping systems. Understanding the perception of manipulated materials and objects is critical to AR applications such as medicine, education, and entertainment. This presentation will share recent methods and results leading towards color appearance models describing AR situations. The perceived appearance of the optical mix of real background objects and AR-displayed virtual foreground colors diverges in some cases from an expected, physical summation. Experiments with 2D color patches show a consistent “discounting” of the background object, suggesting that viewers understand the overlay and partially ignore its effect [Hassani & Murdoch, Color Research & Application, 2019]. Additional experiments show that background discounting seems to be affected by the complexity of the virtual foreground object. Color matches made between foreground colors on different backgrounds in these experiments can be predicted using a modified CAM16 color appearance model that accounts for background discounting in linear XYZ tristimulus space [Hassani PhD dissertation, RIT, 2019]. Figure: Examples of appearance manipulation using different AR overlays on a 3D gray-painted cube. The interaction of the effect of the complexity of the real background object with the precision of the alignment of virtual foreground overlays was explored in a preliminary experiment with real 3D cubes [Leary & Murdoch, SAP 2018]. This work is currently being extended, with the hypothesis that the foreground overlay can be discounted to various extents due to visual cues including alignment and contrast. These observations are connected with older work on veiling luminance and lightness constancy [Fry & Alpern, Optometry and Vision Science, 1954], [Gilchrist & Jacobsen, Journal of Experimental Psychology, 1983]. Going forward, the apparent color, texture, and shape can be manipulated with AR overlays using a combination of 3D graphics and color processing. Models of color and material appearance are needed to predictably recreate the observed effects.

Speaker Bio:
Michael J. Murdoch is an Assistant Professor in RIT's Munsell Color Science Laboratory, where he teaches psychophysical methods, MATLAB programming, color imaging, and lighting perception topics. His current research includes the effects of temporally dynamic LED lighting on visual adaptation and color appearance and visual adaptation in Augmented Reality (AR). Mike has academic and industrial research experience with color system modeling and human-centered design for solid state lighting, LCD, and OLED display systems. He holds a BS in chemical engineering from Cornell, MS in computer science from RIT, and PhD in human-technology interaction from Eindhoven University of Technology in The Netherlands.

Intended Audience:
All are welcome. No background knowledge required.