Some applications that have traditionally relied on single wavelength filters and multiple sensors or multispectral imagers are, with the use of dual band filters employed together with other recent technical advances, now becoming far more affordable. With the resulting reduction in cost barriers, some of these technologies are far more accessible and the potential to benefit a far larger number of individuals and organizations is now greater than ever. Applications include Security/surveillance imaging, 3D point cloud imaging, gaming, automotive, NDVI imaging and numerous other applications.

Surveillance

For applications that use a color camera during the day and discrete near infrared (NIR) illumination at night, a dual-bandpass filter can be an ideal solution. Use of these filters ensures that clear, accurate images are acquired under most lighting conditions – eliminating the need for a costlier filter switching mechanism or, in some cases, two separate cameras.

Dual-band filters used with color cameras in security, license plate recognition and other day/night surveillance applications pass visible light plus a specific narrow wavelength range somewhere in the NIR. This NIR band usually corresponds to the emission wavelength of an IR light source. Blocking longer IR wavelengths along with a large portion of the lower NIR spectrum allows for more accurate color presentation during the day, while retaining the ability to illuminate at night using an illuminator with commensurate LEDs, laser diode or filtered lamps.

Figure 1: Comparison of three images taken in full sun with a Day/Night converted color camera. Color rendition using the DB940 Dual Bandpass filter (1c) is clearly superior.

The most common wavelength for IR lighting used in these applications is 850nm. High brightness 850nm LEDs typically have greater output than other types of infrared LEDs. They are semi-covert, readily available and more affordably priced. Also, most color CCTV cameras will have reasonably good response at 850nm once any NIR-blocking filter inside the camera has been removed.

The use of a visible-850nm dual-bandpass filter in day/night color cameras has become fairly common. A frequent complaint about these filters is that color during daylight hours is far less than perfect (Figure 1b). Some objects, particularly green plants, reflect the majority of 850nm light that reaches them. Thus, their appearance can be washed out and disconcerting.

This can also be particularly problematic in strictly monochrome, NIR applications when a NIR longpass filter is used over the lens. Once the IR-blocking filter is removed from a color camera, the white balance algorithm will in particular be affected by NIR light coming through the sensor’s red and blue channels. This is why some cameras can exhibit a strong red or magenta-hued image under such conditions (Figure 1a).

When this occurs, one other option is to use 930 to 950nm illumination and a corresponding visible-940nm dual-bandpass filter. At 940nm, relative NIR sensitivity in the camera’s green, blue and red channels is not as strong, while moisture in the atmosphere absorbs more light (Figure 2), resulting in more natural color rendition (Figure 1c).

However, this reduced sensitivity that CMOS/CCD sensors have to 930-950nm light can obviously also be an issue. Compounding the problem, relative to 850nm LED light sources, similarly-configured 940nm LED lighting is often of lower intensity. Therefore in order to obtain a sufficiently bright image, higher power LEDs and/or narrower beam angles may have to be considered.

On the other hand, 940nm LEDs have the advantage of being truly covert. Unlike 850nm LEDs, there is usually no faint, visible red glow. To insure that the use of 940nm lighting is practical, it can also be important to use an NIR-corrected day/night lens, one with anti-reflection coatings designed to reach out beyond 900nm. Of course a lens with a larger aperture will also allow more light to enter – assuming its format size is not larger than that of the sensor. If possible, longer camera exposure times at lower frame rates will also help.

By optimizing all of these variables, 940nm illumination, together with an appropriate dual-band filter, usually results in a viable option. With the constant improvement in LED power and brightness that is taking place over time, this option now deserves more consideration than it has received in the past.

Figure 2: Canon 4500 Bayer filter array spectral distribution (blue, green and red channels) with IR blocking filter removed vs. MidOpt DB940 filter transmission and relative intensity of sunlight. Note the reduced spectral intensity and sensitivity at 940nm.

3D point cloud imaging

3D point cloud imaging has found many uses in virtual reality, augmented reality, automotive, security and numerous other applications. Traditionally this has been done using one NIR camera and one or more RGB cameras to capture color information. Structured NIR light is projected onto a scene, allowing extraction of volumetric, precise distance, biometric, and other types of information.

To add the images from the two cameras together, it is necessary to texture the flat RGB image over the 3D NIR image. Not only does this require costly hardware and CPU power, but the color image will be somewhat inaccurate due to parallax error and, when capturing streaming data, synchronization issues will occur. Importantly, there is also significant computational lag.

With dual band visible-plus-NIR filtering, companies such as 2r1y (San Francisco, CA. USA; www.2r1y.com) can use a single color CMOS sensor to capture both the NIR and RGB information simultaneously in real time. This means that, when created, each pixel of the point cloud has a correct XYZ and RGB value. As in color security or surveillance applications, the problem of correcting for over exposure is avoided and RGB color is also returned to an acceptable level.

Over a range of several meters, at a speed of 60 frames per second or faster for a one-megapixel image, the introduction of dual-band filtering has resulted in significant improvement in speed and image quality. With the development of new technologies, such as image sensors based on quantum dots that have excellent sensitivity at 940nm, significant further improvements might be anticipated.

NDVI imaging

Another set of applications where the use of dual-bandpass filters together with color cameras is now receiving more attention is in process control, mapping and vegetative health monitoring in agricultural and forestry applications. Since the 1960s, this type of monitoring has largely been carried out at high altitude and great cost using two or more precisely aligned CCD sensors carried by aircraft or satellites, or two co-registered flights using one camera with different filtering.

The majority benefiting from this information have therefore been universities conducting research, large corporations, or organizations managing vast tracts of land, and by the time this information is disseminated, for many applications it is already outdated. If this information were truly affordable, more timely, and could be applied on a smaller scale, organizations of more modest size or individual farmers might be able to make good use of it.

To differentiate healthy vegetation from background information and vegetation which is stressed, an indicator known as the normalized difference vegetation index (NDVI) is typically applied to each image pixel, although other algorithms are sometimes used as well. NDVI numbers have been arrived at by comparing differences in reflectance in the near infrared and visible portions of the spectra according to the following equation:

NDVI = (NIR – VIS) ÷ (NIR + VIS)

Traditionally only red light has been used to represent activity in the visible portion of the spectrum. The logic behind this is that healthy, growing green vegetation must produce needed energy through photosynthesis. When plants are actively photosynthesizing, they reflect or scatter NIR light. Absorption of these longer wavelengths would result in overheating and tissue damage. The visible portion of the spectrum is absorbed, although a little more green light is reflected away relative to red and blue light (Figure 3). This accounts for the greenish coloring that we see in most healthy foliage.

Figure 3: Typical reflectance curves for healthy vegetation, stressed vegetation and soil.

However, when plants are sick or dying, photosynthesis slows down or stops completely. IR wavelengths start to be absorbed, while more visible light – particularly red light-starts to be reflected. This is exemplified by the graph showing spectral reflectance of a healthy green lawn vs. un-watered, dried out grass (Figure 4). Also, in the case of high altitude imaging, atmospheric scattering of blue and green wavelengths has favored the use of red and NIR light.

Figure 4: % Spectral reflectance of green lawn grass vs. dry grass

When using two cameras or sensors along with appropriate filtering, getting accurate red and NIR data is straightforward. However, in an attempt to reduce the weight, size, power and cost of systems used to gather NDVI information, and with the recent proliferation of relatively inexpensive unmanned aerial vehicles (UAVs), interest has grown significantly in the use of lightweight, inexpensive single camera systems.

In response to this, multispectral and hyperspectral imagers have recently become smaller and more affordable. Unfortunately, their costs still start in the several thousands of dollars, making them cost prohibitive for many if not most potential end users. Therefore, attempts have been made to use low cost, commercially available cameras, dual-band filters and newly-available software for post processing of images for this application.

This technology is far more accessible, however, consumer cameras are mainly designed for taking pictures of cats and typically have poor color band separation and limited dynamic range. It is for this reason that appropriate dual-band filtering can offer significant advantages.

Many initial studies using these types of cameras have somewhat successfully used traditional absorptive filters for monitoring plant health or vegetation cover, such as wratten-type red (a red/NIR longpass filter) or Schott BG3 (a blue/NIR dual band colored glass filter) filter material, together with converted infrared or mixed-spectrum digital cameras.

The sensors in most color cameras include an integral Bayer color filter array, which consists of separate blue, red and green filtering over normally standard silicon-response image pixels. However, each of these RGB filters also transmit significant amounts of NIR light, the exact amounts differing with each camera or sensor design used. Thus all color cameras are also provided with an NIR blocking filter that is placed over the image sensor.

Because it is necessary to remove this NIR blocking filter for true NDVI imaging, the results obtained with different cameras can then vary greatly. For example, the red Bayer longpass filter will usually pass both red and NIR light equally well, so it becomes difficult to separate the two wavelength ranges and accurately calculate NDVI.

Additionally, most of these cameras employ RGB-optimized auto exposure and auto white-balancing features that are not user controllable, and images are captured in jpeg format rather than as raw images. Once the NIR blocking filter is removed and infrared light is allowed to reach the sensor, the camera doesn’t realize what has happened and the resulting remixing of the color balance and overlapping in sensitivity of the channels, particularly because of the NIR light present in each channel, often inhibits conversion of the results into meaningful NDVI values.

There are important advantages to using cameras that allow for manual white balancing and exposure control. Using different color indices and supporting filtering can also yield promising results. While use of a dual-band filter allows for greater band separation and overcomes some of the dynamic range issues, this single-camera configuration may not show differences in plant health as accurately as combined-dual-camera systems collecting red and near-infrared data.

This can be blamed in part on most of the visible/NIR dual band filters used in the past. Instead, narrower, better-defined filters with sharper cut-on and cut-off slopes have yielded much improved results (Figure 5). After post processing, the contrast in these images can approach those obtained with a multi-sensor camera or two-camera single band filter images (Figure 6). It can be argued that the results are more than acceptable for many applications and end users.

Figure 5: Aerial images of an orchard taken with a single sensor color camera and (5b and 5c) MidOpt DB450/808 dual band filter added. (5c) has been processed with Fiji (Image J).

Thus performance of single multispectral camera systems that cost more than $3,500 per system can now be rivaled by single sensor cameras costing $400 per camera. This is a tremendous cost difference and, when mounted on drones or kites, this type of NDVI capability now becomes affordable for individual farmers or smaller end users, many located in third- world countries.

Simple and affordable

Results obtained using dual-bandpass filters integrated into lower-cost digital color cameras, together with other recent advances, emphasize their value in contributing to simplified and more affordable solutions in various imaging applications.

For day/night security and surveillance, the advent of low-cost yet powerful 930-950nm infrared illumination and appropriate, higher resolution sensors and confocal lensing make the use of visible-940nm dual band filters a more viable and visually appealing option. And if organizations and individuals with limited budgets are able to make use of information gathered from small UAVs equipped with low cost digital cameras, what more might they be able to do in terms of crop or forestry management?

Figure 6: Comparison of NDVI false color aerial images of an orchard taken with (6a) a converted single sensor color camera and (6b) a multi-sensor camera. Both have been processed with Pix4DMapper Pro. There is little discernable difference.

Many companies are now offering reasonably priced products and services employing dual-band filtering, so the answers to such questions should start to become apparent. Beyond aerial applications, in the near future such cameras can be mounted on farm equipment to provide control to systems that intelligently and economically apply chemicals to crops where they are most needed, identify specific areas stressed due to a lack or overabundance of water, or that are used in harvesting or culling specific individual trees or plants.

Simplifying the timely collection of information with increasingly lower cost vision systems could hold as much if not more overall value as new advancements leading to the creation of more sophisticated, technically advanced instrumentation for image processing.

This article was featured in Vision Systems Design.