Site-wide links

Imaging

RIT is a world leader in imaging, from scientists who are creating technologies that will revolutionize the use of imaging applications to artists who constantly advance the world of creative arts.

Image Segmentation Revolutionized and Commericalized

Image Segmentation Revolutionized and Commericalized

Image segmentation reveals information hidden to the human eye, enhancing the ability to extract and analyze meaningful information. RIT researchers are advancing the technology to support the business needs of industry partners, including Hewlett-Packard (HP), Ortho Clinical Diagnostics, a Johnson & Johnson company, and Lenel.

RIT and HP

For nearly a decade, RIT and HP have engaged in sponsored research activities that support HP's product development goals. Recently, Dr. Eli Saber, an associate professor in the department of electrical engineering, has been leading an effort in collaboration with HP that focuses on developing and advancing novel methodologies for image segmentation. "By leveraging Saber's experience and expertise, we are able to address the needs in our color and imaging work," says Dr. Ken Lindblom, distinguished technologist at HP. "RIT is able to provide a unique skill set that augments what exists in our business division."

The interest in digital media has grown to new heights with the rapid technological advancements being made in the ability to capture and share digital images. This has necessitated the exploration of methods to interpret, organize, enhance, classify, and extract information from those images. Image segmentation is one approach that provides the foundation to make these functionalities more effective and expeditious.

Over the past two decades, many methodologies have been developed to tackle the complex problem of image segmentation, and the progression of these methodologies has resulted in the achievement of meaningful segmentations. However, the development of an effective simulated environment with realtime capabilities to perform this imaging task has proven to be extremely challenging. Saber, who is also an extended faculty member at the Chester F. Carlson Center for Imaging Science, along with Sreenath Rao Vantaram and Mustafa Jaber, imaging science doctoral students, and Mark Shaw and Ranjit Bhaskar, color and imaging R&D experts at HP, have developed a novel methodology that balances superior segmentation quality with speed. The patent-pending technique has been found extremely useful for diverse commercial and research initiatives related to the printing, medical, and remote sensing fields.

Segmentation at Lamberton Conservatory

Segmentation at Lamberton Conservatory:

Dr. Eli Saber, pictured in the desert setting at the Lamberton Conservatory, creates a particularly challenging environment for image segmentation due to the natural Advancing Image Segmentation: Dr. Eli Saber, scenery and complex textures and shapes.

Novel Approach to Image Segmentation

The approach, Multiresolution Adaptive and Progressive Gradient-based color image SEGmentation (MAPGSEG), integrates color, texture, and gradient information in a multiresolution framework, and is based on the principle that the segmentation results of images at low resolution can be used to efficiently segment their corresponding high-resolution counterparts. The algorithm, which is implemented entirely in a MATLAB—an interactive numerical computing environment—is comprised of six modules.

The first module performs dyadic wavelet decomposition to obtain highquality approximations of the input RGB (red, green, blue color space) image at different resolution levels. The number of decomposition levels is automatically determined based on the smallest workable image dimension specified by a user or constrained by an application.

The second module converts the input image from RGB to the CIE L*a*b*, a device independent color space developed in 1976 by the Commission Internationale de l'Eclairage to achieve approximate perceptual consistency of colors. Consequently, the CIE L*a*b* color space facilitates improved color differentiation and separation of luminance-chrominance information, which enables the efficient handling of images with illumination variations. The color conversion scheme is followed by the computation of gradient information used to automatically and adaptively generate the thresholds required to process regions at different resolutions.

The third module performs a computationally efficient progressive and dynamic multiresolution region-growth procedure to create an initial estimate of regions in the image. In the fourth module, the MAPGSEG algorithm executes a texture characterization process, which differentiates textures at various resolutions. This module is an important aspect of the algorithm, as texture frequently manifests itself as multiple color shades or intensity variations due to material properties like density, gradient, and coarseness, impairing the process of image segmentation.

Next, the initially identified regions are integrated with the acquired texture model using a statistical region-merging procedure to obtain an interim segmentation. In the final phase, the acquired interim result is up-scaled and regions of high confidence are determined. These regions are passed on as a priori information to the next stage of the algorithm to process the remaining unsegmented areas with minimal computational costs. The algorithm progresses from one resolution level to another of the image pyramid until a segmentation result is achieved at the highest/original resolution and is semantically consistent with the input image.

Multiresolution Adaptive and Progressive Gradient-based Color Image SEGmentation

Multiresolution Adaptive and Progressive Gradient-based Color Image SEGmentation:

The novel approach is comprised of six modules processed at different image resolutions: (1) Input RGB image (2) convert image into CIE L*a*b* color space (3) compute gradient/edge information (4) initial estimate of various regions (5) texture characterization (6) final segmentation output where every region has a distinct color.

Click on the image to see the details of the approach.

Convincing Results

The team has evaluated the MAPGSEG technology using a prominent segmentation evaluation metric called the Normalized Probabilistic Rand (NPR) index on several hundred images made publicly available by the University of California at Berkeley. The results demonstrate their approach outperforms contemporary segmentation benchmarks with superior quality.

Handling Structural Variations: Church Example

Handling Structural Variations:

Gradient variations in natural scene images, as seen in the images of a church, often cause images to be over-segmented. Click on the image to see the image segmentation comparison.

An example shown above illustrates a reasonably complex image of a church, which consists of stark illumination variations and a considerable number of structural details. By comparison, previous methods over-segment the image in the sky and dome regions due to illumination disparities. The structural details also cause these approaches to over-segment the facade of the church. The segmentation map that uses the novel MAPGSEG methodology illustrates the efficiency of the algorithm in handling issues of illumination. The images that proceed demonstrate the region growth strategy that merges similar color-texture regions independent of their spatial locations, resulting in an efficient and accurate image segmentation outcome.

Detecting Details: Formula 1 Racecourse Example

Detecting Details:

An image from a Formula 1 raceway creates a complex scene where image content is likely to be occluded and contains fine details, such as text. Click on the image to see the image segmentation comparison.

The image of a Formula 1 racecourse presents a few challenges, such as the occlusion of image content by foreground objects as well as the presence of fine detail. Again, alternate methods over-segment the image, where MAPGSEG's region merging procedure successfully separates occluded regions and extracts fine details with great competence.

The segmentation task becomes extremely challenging when regions with dissimilar textures have great color similarity, such as the image of the cheetah. In this scenario, the texture descriptor is able to segment the skin tone that complements the background. The results shown emphasize the significance of the texture characterization module in the MAPGSEG approach.

"This research has catapulted us forward in our understanding of image segmentation," says Shaw. "We have an algorithm that has progressively gone faster, which we will be able use as a building block in our business."

Distinguishing Textures: Cheetah Example

Distinguishing Textures:

The image of a cheetah demonstrates the challenge of distinguishing textures when the dissimilar textures have great color similarity. Click on the image to see the image segmentation comparison.

Fostering Innovation

The research team has expanded on this work through the RIT Corporate R&D program. In partnership with Ortho Clinical Diagnostics, a Johnson & Johnson company, and Lenel, a UTC Fire and Security company, master's degree students are developing imaging algorithms specific to the industry partner's business needs.

The Corporate R&D program was pioneered by RIT's President Bill Destler to foster research partnerships among industry and universities. The agreement allows companies to retain the rights to any intellectual property that may be generated during the research projects, while RIT retains the right to publish and conduct further research that builds off of the work.

"The benefits are mutual," says Saber. "We are able to conduct applied research that has a direct implication to their business needs; meanwhile, our students gain tremendous confidence and become more than adequately prepared for a career in industry."

"Going forward, we view university partnerships as complementary to our internal development processes. Sometimes we have an idea for the future, but are unable to focus on it right away. That's the beauty of working with a university; we can tune the research to our specific needs and it becomes very beneficial to the business," explains Lindblom.

"While the private sector should not dictate the intellectual directions of the university, if universities and colleges are to become the economic engines they aspire to be, then the research and development activities need to be focused, at least in part, on projects that have the potential to lead to new products and services," adds Destler.

Image segmentation of a computed tomographic

Segmentation for Medicine:

Image segmentation of a computed tomographic (CT) image identifies each element within the body—be it an organ, tumor, or abnormality—as a different region.

More information is available online at http://people.rit.edu/esseee/ and http://www.rit.edu/research/corporate

Multiresolution Adaptive and Progressive Gradient-based Color Image SEGmentation:

The novel approach is comprised of six modules processed at different image resolutions: (1) Input RGB image (2) convert image into CIE L*a*b* color space (3) compute gradient/edge information (4) initial estimate of various regions (5) texture characterization (6) final segmentation output where every region has a distinct color.

Handling Structural Variations:

Gradient variations in natural scene images, as seen in the image of a church, often cause images to be over-segmented. A previous method (b) over-segments the sky and structural details of the church due to differences in illumination. The novel MAPGSEG algorithm (c) efficiently handles the illumination issues, resulting in an accurate segmentation.

Detecting Details:

An image from a Formula 1 raceway (a) creates a complex scene where image content is likely to be occluded and contains fine details, such as text. A previous method (b) over-segments areas of the raceway occluded by the vehicles in the scene. MAPGSEG's region merging procedure separates occluded regions and extracts fine details with great competence, as shown in (c).

Distinguishing Textures:

The image of a cheetah demonstrates the challenge of distinguishing textures when the dissimilar textures have great color similarity. Previous methods (b) were unable to distinguish the cheetah camouflaged by the sandy background; the results shown in (c) demonstrate the significance of MAPGSEG's texture characterization module.