Dual bandpass filters used on the modified DJI Mavic 2 Pro Full-Spectrum Hasselblad can be purchased here: Both Types of Filters for sale here:
Since 2016 I have been researching and practicing using drones for vegetation monitoring. In essence the entire project can be summed up with a simple piece of blue filter gel:
The devil of course lies in details and, at the risk of mixing metaphors, he only appears with results if you get particularly crafty!
There are in fact many different types of filter to choose from (with exciting names such as Lee 181, Rosco 382, Wratten 25A and so on). Whatever the case we want a filter to block the distracting visible wavelengths of light to get the key wavelength band that indicates how healthy vegetation looks from a distance.
Regardless of how a plant looks to the visible eye, in the near infrared (NIR for short) if a plant does not reflect strongly in this band then it is stressed, dying or dead (or not a plant at all, i.e. soil).
When plants are actively photosynthesizing, they reflect or scatter near-IR light, thanks largely due to the spongy mesophyll portion in plant cells. Plant cells evolved NIR reflection largely to avoid overheating and tissue damage so a high albedo in the NIR is very helpful for plant health in general and so is a key indicator of health.
The blue and red visible portion of the spectrum is absorbed by Chlorophyll A and B; however, green light is reflected away, relative to blue and red light. Thus most plants will appear green.
To bring the devil back into the discussion we could play his advocate and ask: "Wait, why are you droning on about the Near-Infrared? Couldn't we just look for how green a landscape is. Plants are green, duh!"
Most healthy plants are indeed green (but not all), but their foliage or indicators of their health, i.e. flowers are sometimes not actually strongly green, or the intensity of green is different between equally healthy plants of different species.
Moreover a lot of things are green in an environment. They may not appear strongly green on their own but that's usually because of colors mixing. Soil, Earth and Sand can look brown, but that's actually mostly a mixture of green and red. Open brown Earth does not emit much near infrared however, it absorbs it (and can convert it to longer wavelength thermal infrared but that's another story).
Hence with near infrared we can distinguish between healthy plants, stressed or dying plants and essentially bare open Earth devoid of plants.
And quite distinguishable it is too. In some cases the plants are glowing in comparison to the surrounding features in both close-up and landscape NIR images:
This brings us to the issue of cameras. New camera models offer greater sensitivity to the near infrared with bigger and more sensitive CMOS pixels and so forth. Combine this with the emerging technology of drones on the market and we have a great recipe for creating images that rival satellites and even surpass them in some cases.
Think of moments of mixed cloud cover where satellites fail to get NIR reflected from the surface. Moreover in the case of satellite imaging overall atmospheric scattering of blue and green wavelengths has restricted the scanning wavelengths usually to red and near-IR light.
When using two cameras or sensors and appropriate optical filtering, getting visible and near-IR data is straightforward. However, to reduce the weight and cost of systems used to perform NDVI mapping, interest has grown in single camera designs using machine vision algorithms.
Consumer cameras are mainly designed to have limited dynamic range across the potential sensitiy of the CCDS they use. However these cameras can be highly affordable, and with the help of inexpensive dual band filtering, single camera NDVI imaging is of relatively low cost compared to a proprietary hyper-spectral camera while at the same time matching its performance.
As discussed in a previous article, consumer cameras use an integral color filter array, usually a Bayer filter, consisting of separate blue, red and green filtering, over standard silicon-response pixels/sensors. Each of these RGB filters also transmit significant amounts of IR light. For this reason, all color cameras are also equipped with an IR blocking "hot mirror" filter.
With that in mind, last year I had modified a DJI Mavic Pro to record NIR images using custom optics, made out of that very important blue gel filter, and with some analysis I was able to create some maps containing a vegetation index colour reference.
This was a first step in a direction that led me to conclude that to go further meant I needed a bigger sensor with the option of converting the camera into a NIR camera by removing the so-called hot mirror filter integrated above the CMOS chip. The original Mavic Pro camera was too small and finicky for me to do this without breaking some of the critical components of the camera and gimbal system.
I had intended, in the past and up until recently, to modify cheaper cameras to allow me to see both NIR and Visible images at the same time and had even developed a prototype to fit onto a drone
Using this cheaper dual camera setup allowed me to get the following image:
Although I got a clear signal of NIR reflectance from plants vs Blue reflectance from roads and buildings the results were not up to the standard I wanted.
Wanting to move away from the domain of amateur NIR imaging I knew then I had to wait for the arrival of the Mavic 2 Pro to provide a camera big enough and sensitive enough to make this operation worth performing.
And so I did.
Some preliminary testing with the unmodified camera is showing me that it already exceeds the original to the extent that the hot mirror filter removal mods may not be necessary for typical use.
With the hot mirror filter and without any external filters we get a clear birds eye view of the landscape we are interested in studying:
For Near-Infrared monitoring I found that the Rosco #382 filter worked best and it only needed a slight correction to generate this following image.
Settings to ISO and vividness are still necessary but that is to be expected with a camera that still contains a hot mirror filter.
These "Infrablue" filters are dual bandpass filters. They allow the near UV and Near Infrared to pass through while blocking the green, Yellow and Orange part of the visible spectrum.
The next step is analysis of the images. The standard algorithm I developed for the Mavic 1 runs into some problems with the larger Mavic 2 formats. Moreover the camera itself needs completely different gamma values to create the contrast needed to create a NDVI style temperature map. This was overcome relatively quickly and a NDVI analog image was created:
With the method I developed last year I run into a problem. Essentially shadows are erroneously processed as having stronger reflections in the near infrared, as they appear darker in one colour channel of the RAW image filter relate to another, say darker in red than in blue. This creates a large bug in the processing using a non-modified camera.
After performing the hot mirror removal on the Mavic 2 Pro Hasselblad Camera (which I do not recommend performing yourself to avoid warranty issues!) we definitely see a lot more vivid response of vegetation with our multilayer filter:
With some enhancements to the analysis procedure we then get a more clearer colour map which properly registers shadows.
Shadows are annoying and do limit the information we get however a new plan is in the works to remedy this detail with the use of multiple image overlays and point cloud based image reconstruction that I will be proceeding with in the near future.
The beauty of the DJI Mavic 2 Pro camera is that it employs RGB-optimized exposure and white-balancing features that are user controllable. There are important advantages to using cameras that allow for manual white balancing and exposure control. Experimenting with ISO values and the gamma level of the blue vs red channels is key to get good contrast with the false colour ndvi image that is generated.
Using different dual band filtering techniques also holds much potential for exploration and fine-tuning this procedure. A dual band filter can come in two categories, either one with very broad bands with fairly low slopes in the near infrared region, allowing to see a greater portion of the vegetation intensity to be detected. Alternatively a narrower dual band filter can be employed with sharper, more wavelength specific detection of NIR reflectance levels.
Developing this project further will be key to certain expeditions I have planned involving testing of NIR vegetation monitoring in a range of environments. I will be starting initial testing in the West of Ireland very soon. From there this project can be taken further with more extensive field tests with sufficient support.
Infrablue Dual Bandpass Filter Gel
The devil of course lies in details and, at the risk of mixing metaphors, he only appears with results if you get particularly crafty!
There are in fact many different types of filter to choose from (with exciting names such as Lee 181, Rosco 382, Wratten 25A and so on). Whatever the case we want a filter to block the distracting visible wavelengths of light to get the key wavelength band that indicates how healthy vegetation looks from a distance.
Regardless of how a plant looks to the visible eye, in the near infrared (NIR for short) if a plant does not reflect strongly in this band then it is stressed, dying or dead (or not a plant at all, i.e. soil).
When plants are actively photosynthesizing, they reflect or scatter near-IR light, thanks largely due to the spongy mesophyll portion in plant cells. Plant cells evolved NIR reflection largely to avoid overheating and tissue damage so a high albedo in the NIR is very helpful for plant health in general and so is a key indicator of health.
The blue and red visible portion of the spectrum is absorbed by Chlorophyll A and B; however, green light is reflected away, relative to blue and red light. Thus most plants will appear green.
To bring the devil back into the discussion we could play his advocate and ask: "Wait, why are you droning on about the Near-Infrared? Couldn't we just look for how green a landscape is. Plants are green, duh!"
Most healthy plants are indeed green (but not all), but their foliage or indicators of their health, i.e. flowers are sometimes not actually strongly green, or the intensity of green is different between equally healthy plants of different species.
Moreover a lot of things are green in an environment. They may not appear strongly green on their own but that's usually because of colors mixing. Soil, Earth and Sand can look brown, but that's actually mostly a mixture of green and red. Open brown Earth does not emit much near infrared however, it absorbs it (and can convert it to longer wavelength thermal infrared but that's another story).
Hence with near infrared we can distinguish between healthy plants, stressed or dying plants and essentially bare open Earth devoid of plants.
And quite distinguishable it is too. In some cases the plants are glowing in comparison to the surrounding features in both close-up and landscape NIR images:
Close-up NIR Image
Landscape NIR Image
This brings us to the issue of cameras. New camera models offer greater sensitivity to the near infrared with bigger and more sensitive CMOS pixels and so forth. Combine this with the emerging technology of drones on the market and we have a great recipe for creating images that rival satellites and even surpass them in some cases.
Think of moments of mixed cloud cover where satellites fail to get NIR reflected from the surface. Moreover in the case of satellite imaging overall atmospheric scattering of blue and green wavelengths has restricted the scanning wavelengths usually to red and near-IR light.
When using two cameras or sensors and appropriate optical filtering, getting visible and near-IR data is straightforward. However, to reduce the weight and cost of systems used to perform NDVI mapping, interest has grown in single camera designs using machine vision algorithms.
Consumer cameras are mainly designed to have limited dynamic range across the potential sensitiy of the CCDS they use. However these cameras can be highly affordable, and with the help of inexpensive dual band filtering, single camera NDVI imaging is of relatively low cost compared to a proprietary hyper-spectral camera while at the same time matching its performance.
As discussed in a previous article, consumer cameras use an integral color filter array, usually a Bayer filter, consisting of separate blue, red and green filtering, over standard silicon-response pixels/sensors. Each of these RGB filters also transmit significant amounts of IR light. For this reason, all color cameras are also equipped with an IR blocking "hot mirror" filter.
Hot Mirror Filter Removal from a Commercial Camera
With that in mind, last year I had modified a DJI Mavic Pro to record NIR images using custom optics, made out of that very important blue gel filter, and with some analysis I was able to create some maps containing a vegetation index colour reference.
This was a first step in a direction that led me to conclude that to go further meant I needed a bigger sensor with the option of converting the camera into a NIR camera by removing the so-called hot mirror filter integrated above the CMOS chip. The original Mavic Pro camera was too small and finicky for me to do this without breaking some of the critical components of the camera and gimbal system.
I had intended, in the past and up until recently, to modify cheaper cameras to allow me to see both NIR and Visible images at the same time and had even developed a prototype to fit onto a drone
Assembled Dual Camera System, one for visible spectrum imaging, the other for NIR imaging
Using this cheaper dual camera setup allowed me to get the following image:
Although I got a clear signal of NIR reflectance from plants vs Blue reflectance from roads and buildings the results were not up to the standard I wanted.
Wanting to move away from the domain of amateur NIR imaging I knew then I had to wait for the arrival of the Mavic 2 Pro to provide a camera big enough and sensitive enough to make this operation worth performing.
And so I did.
DJI Mavic 2 Pro
Some preliminary testing with the unmodified camera is showing me that it already exceeds the original to the extent that the hot mirror filter removal mods may not be necessary for typical use.
With the hot mirror filter and without any external filters we get a clear birds eye view of the landscape we are interested in studying:
Visible Spectrum Image - taken using hot mirror filtered camera
For Near-Infrared monitoring I found that the Rosco #382 filter worked best and it only needed a slight correction to generate this following image.
Settings to ISO and vividness are still necessary but that is to be expected with a camera that still contains a hot mirror filter.
Infrablue Image - taken using hot mirror filtered camera
These "Infrablue" filters are dual bandpass filters. They allow the near UV and Near Infrared to pass through while blocking the green, Yellow and Orange part of the visible spectrum.
The next step is analysis of the images. The standard algorithm I developed for the Mavic 1 runs into some problems with the larger Mavic 2 formats. Moreover the camera itself needs completely different gamma values to create the contrast needed to create a NDVI style temperature map. This was overcome relatively quickly and a NDVI analog image was created:
NDVI Analogue
With the method I developed last year I run into a problem. Essentially shadows are erroneously processed as having stronger reflections in the near infrared, as they appear darker in one colour channel of the RAW image filter relate to another, say darker in red than in blue. This creates a large bug in the processing using a non-modified camera.
After performing the hot mirror removal on the Mavic 2 Pro Hasselblad Camera (which I do not recommend performing yourself to avoid warranty issues!) we definitely see a lot more vivid response of vegetation with our multilayer filter:
True NIR Image - taken using full spectrum converted camera
With some enhancements to the analysis procedure we then get a more clearer colour map which properly registers shadows.
NDVI Map
Shadows are annoying and do limit the information we get however a new plan is in the works to remedy this detail with the use of multiple image overlays and point cloud based image reconstruction that I will be proceeding with in the near future.
The beauty of the DJI Mavic 2 Pro camera is that it employs RGB-optimized exposure and white-balancing features that are user controllable. There are important advantages to using cameras that allow for manual white balancing and exposure control. Experimenting with ISO values and the gamma level of the blue vs red channels is key to get good contrast with the false colour ndvi image that is generated.
Using different dual band filtering techniques also holds much potential for exploration and fine-tuning this procedure. A dual band filter can come in two categories, either one with very broad bands with fairly low slopes in the near infrared region, allowing to see a greater portion of the vegetation intensity to be detected. Alternatively a narrower dual band filter can be employed with sharper, more wavelength specific detection of NIR reflectance levels.
Developing this project further will be key to certain expeditions I have planned involving testing of NIR vegetation monitoring in a range of environments. I will be starting initial testing in the West of Ireland very soon. From there this project can be taken further with more extensive field tests with sufficient support.
No comments:
Post a Comment
Note: only a member of this blog may post a comment.