Eye in the Sky

Remote Sensing with Unmanned Aerial Vehicles

Published Feb. 28, 2017

The technology research firm Gartner says that, barring regulatory hurdles, the United States unmanned aerial vehicles (UAV) business could be worth $7 billion in a decade. What’s driving the growth? The Internet of Things—the idea of making physical things “smart enough” to provide intelligence reliably and cost-effectively.

The following feature story is one of five signature interdisciplinary research areas RIT is current exploring through advanced research. Each of these research areas is reflective of a growing significance in society, and all are endeavors where RIT can make high-impact contributions in the fields.

Commercial industries around the world are starting to understand the power of this concept, and are looking to UAVs as one way of getting there. RIT, long known for its expertise in both aerial and satellite imaging, is also one of the world’s leading centers for research on UAVs.

As these two areas align, a new disruptive commercial model emerges. Low-cost UAVs collect information in new ways, and deliver it to software platforms capable of analyzing images and video reliably and in real time. The result produces insights that will dramatically change farming, insurance inspection, defense, law enforcement, disaster recovery, and many other industries.

Photo of DJI S900 n-copter

Pegasus Environmental: RIT’s partner provides its DJI S900 n-copter to collect multispectral image data for water quality analysis research.

But before that vision becomes reality, there are many technical and practical hurdles to overcome, including imaging science, engineering, and public policy, said David Messinger, director of RIT’s Chester F. Carlson Center for Imaging Science. That’s the reason RIT chose remote sensing with unmanned aerial vehicles as one of five strategic initiatives to receive $1 million for research. The interdisciplinary team, including imaging science, engineering, public policy, and computer engineering technology, will research ways to refine and integrate technology in ways that serve many different commercial and government needs.

When UAVs collect data, for example, there needs to be consistency in how the camera takes a picture, Messinger said. “Current instruments are not always stable, so we’re looking to build a system that uses inexpensive cameras to collect data in a precise way.”

Precision is critical, he said. Especially since many applications require “change detection.” Today, humans are paid to watch hours of surveillance video looking for meaningful change between one image and another. Programming a computer to do this is a difficult problem to solve. Farmers practicing precision agriculture, for example, want to collect real-time data and use predictive analytics to make smarter decisions.

“But they don’t want to stare at video to determine where a leak in their irrigation system is,” Messinger said. “They want reliable usable information every day.”

From Bug Infestation to Hail Storms

Today, large-scale farms look to three types of aerial imagery for precision farming. Satellite imagery, used since the 1970s, is relatively inexpensive but provides a resolution of only five to 20 meters. Aircraft pictures, a much more costly approach, capture images from 5 to 10 cm in resolution. Both of these options are limited in how often data sets are collected, and many farm management decisions require information on a daily or weekly basis. Other limitations also exist; for example, satellites can’t see through clouds.

UAVs, on the other hand, can capture data from images with a 2.5 cm resolution, providing a level of detail that is vastly different from the other two options. But to work, “the science behind it needs to be rethought—and that’s not always happening,” said Carl Salvaggio, a professor at RIT in the Chester F. Carlson Center of Imaging Science and member of the Digital Imaging and Remote Sensing (DIRS) Laboratory. “A lot of people are claiming these camera systems can do things that they can’t.”

Rethinking the science, according to Salvaggio, is what his students are working on in their research. Take, for example, shadows. Detecting them is difficult for a computer program to sort out. But it’s important if the farmer is going to get accurate crop information. Unlike using satellite imagery or aircraft pictures, an image taken from a closer point of view includes shadows, and these can be confused with signs of disease or infestation.

Photo of a fixed-wing aircraft

PrecisionHawk Lancaster: PrecisionHawk of Toronto, Ontario, has provided RIT with its Lancaster fixed- wing aircraft capable of lifting imaging sensors to altitudes up to 400 feet for up to 45 minutes.

PrecisionHawk, a terrestrial data acquisition and analysis company, is collaborating with RIT to share technology improvements and test remote sensing applications using PrecisionHawk unmanned aerial vehicles. Data generated from applications such as precision agriculture will be shared between PrecisionHawk and RIT for future development. PrecisionHawk recently provided RIT researchers with its Lancaster UAV platform. This fixed-wing aircraft with a five-foot wingspan is capable of accepting interchangeable, modular imaging systems weighing up to 2.2 lbs. that collect hyperspectral, light detection and ranging, and high-resolution color imagery. This system can stay aloft for 45 minutes at altitudes up to 400 feet, flying at speeds up to 25 mph, providing large coverage to the RIT team.

Another ongoing research project relates to assessing water quality, Salvaggio said. A UAV equipped with the appropriate spectral sensing camera can be used to detect and map harmful algal blooms in lakes.

This new, finer point of view has endless business applications, including damage assessment for insurance companies. Instead of sending assessors onto a rooftop, these companies are looking to send up a UAV with camera systems that can reliably determine whether a roof was damaged by hail.

“Damage from hail has small characteristics that are different from other types of damage,” Messinger said. “Looking for those small imperfections and separating them out is a hard problem that we are working on.”

Photos of roof damage

Rooftop Damage: RIT researchers are looking to automatically quantify the amount of damage to rooftops caused by hail (top) and wind (bottom) to help property casualty insurance inspectors carry out their inspections in a faster and more accurate fashion to settle claims for their clients.

RIT has become the first academic technology member of the Property Drone Consortium, a group of major property insurance industry members and affiliated companies exploring technological improvements for their industry. RIT hopes to leverage its expertise in remote sensing with new opportunities and applications assisting consortium members Allstate, American Family Insurance, Erie Insurance, Auto-Owners Insurance, EagleView Technologies, Pilot Catastrophe Services Inc., Insurance Institute for Business and Home Safety, and National Roofing Contractors Association to explore use of UAVs and remote sensing technology for faster, more cost-effective property damage detection and claim adjustments. The goal is to provide new data, never before achievable in an automated fashion, to settle claims rapidly and accurately in post-disaster scenarios.

Working Together to Expand Research

Professor Daniel Kaputa with RIT’s College of Applied Science and Technology is an expert in controlling how things fly. He says another key hurdle to advancing the aerial imagery research at RIT is creating a custom low-cost drone development platform. With a price point of around $200, the platform will be used to test out algorithms such as real-time video processing and collision avoidance. “These platforms would have their own sensors, motors, and cameras that are tailored to our research,” he said. “We can prove out our algorithms at a low cost, and then scale up.” It would also open up many opportunities to explore indoor applications.

When drones fly outside, they have access to GPS navigation, which provides information within a couple of feet of the UAV’s location. But inside, that navigation needs to be accurate to within an inch. “Imagine you own a retail store and want to do an inventory of books,” he said. “You could use a drone to fly up and down every row and take images of what is in stock.”

Aerial photo of large field of vegetation

Turf Health: A vegetation stress orthomosaic formed from over 300 individual images collected over a turf field in Pittsford, N.Y. Dark green colors represent healthy turf with yellow areas indicating vegetation exhibiting stressed conditions.

Creating a prototype system that’s inexpensive will allow us to “attack any problem we want to,” Salvaggio said. “Drones are just a beautiful way to get us where we want to go.”

But along with progress comes considerable public policy challenges related to safety and privacy, said Josephine Wolff, assistant professor in RIT’s public policy department and a member of the extended faculty of the computing security department. “Answering the technical questions well requires that we have some idea of how we want to answer the social and regulatory questions and, similarly, making policy recommendations requires that we have some knowledge of the technical capabilities and implications, so it’s crucial to address questions like these as part of an interdisciplinary team.”

Some of those questions include how UAVs can be incorporated into airspace without posing serious risks to the safety of other aircraft. And what kinds of privacy-preserving algorithms and technologies can be used to shield individuals from unintended surveillance?

“It’s somewhat unusual to see the policy piece being addressed in tandem with—and by the same team as—the technical challenges since engineers tend to be wary of lawyers and policymakers (and vice-versa),” Wolff said. “To have a research initiative like this one, that incorporates policy into a larger technical UAV research agenda, you really need scientists and engineers who are willing to acknowledge the importance of getting the policy landscape right and who aren’t just writing off the regulatory piece as an unavoidable nuisance.”