Monday, 22 June 2020

Plotting for Scientific and Engineering Applications Using Python



In learning anything to do with programming it often helps to have a motivating factor to invest time and effort into developing a particular programming application.

I have long found Python to be a very valuable tool with regard to spectrum analysis and image processing in developing remote sensing technology using modified drone cameras equipped with customised filters.

The science of acquiring images which contain information, such as near-infrared imagery, involves the development and testing of specialised optical filter technology for use in modified cameras for imaging.

The 2 filters developed for this purpose are essentially dual bandpass filters, made of high quality glass, with different transmission characteristics in the near-UV to blue band and Near-infrared band.

They are circular filters embedded in a custom housing for use with the Mavic 2 Pro Hasselblad Camera.

Filter#1



Filter#2



Measuring the spectra of the optical filters is critical for defining their application in the field.

We can measure the characteristic spectra using a visible wavelength optical spectrum analyser (OSA) and read the data into a plot in Python showing the dual-bandpass nature of the filter at 2 critical wavelength regions. 





We can overlay 2 sets of data from 2 filters and compare the 2 in the same wavelength regions



We can also study the color histogram of the image taken by comparing an image taken without a filter and and image taken using the different filter types.

With No Filter:





With Filter #2:



This information is useful in constructing radiation indices for identification of quantifiable objects in an image, which is key in the practice of quantity surveying using remote sensing techniques.

As we can see using the filter the green channel is greatly diminished and we can leverage the amount of Red detected (which is NIR using our filter) with the Blue using our formula to get the modified NDVI, giving a way to quantify the vegetation density in a photographed and/or mapped region. 




We can also plot the Red channel (Our NIR) vs the Blue Channel (Our Visible Leverage) to get a characteristic soil and wet edge profile, a kind of correlogram, which is very useful as a remote classification tool. 



The data in the scatter plot has a particular pattern however it is really too dense to properly interpret. A solution to this is to create a hexagon bin plot. 



we can tidy up our plotting further by using the gaussian kernal density function. This also helps prepare the plot for further study as we shall see.

To begin we need to flatten the data from a 2D pixel array to 1D with length corresponding to elements (Warning! this can take some time). the fastest way to do this is to use the python library-level function ravel, which reshapes the array and returns its view. Ravel will often be many times faster than the similar function flatten as no computer memory needs to be copied and the original array is not affected. *

With added shading we get an easier graph to view and work with overall:




With the ability to place a contour around certain regions of the plot we can then have the ability to link the spectral content with the spacial content in our image analysis.

First we can define a threshold for our NDVI and plot it to see the NDVI levels marked above this threshold if we wish



Next we can place a contour over the spectral region that the threshold mark corresponds to in the NIR vs Blue spectral graph.



We can interpret this ourselves and classify the spectral analysis using established knowledge of its features. This is the beginning of developing a kind of classification system based on this linked information.

We can also classify using other means such as plotting the pixels located between the dry and wet wedges to indicated where mixed vegetation could be.

Going futher down this path leads to exploring other tools such as clustering algorithms, support vector machines, neural networks and the like as we approach the cusp of the field of machine learning with our spectral information which is a valuable technology in and of itself. This deserves its own series of articles with regard to combining drone-based  remote sensing with machine vision tools, particularly some of those features in use in the QGIS toolboxes, such as Orfeo.

Interesting still is the fact that indices beyond the NDVI can be developed and studied using these scientific plotting techniques using new filter designs to explore new potential environmental markers. An Ultraviolet-based environmental index using UV-remote sensing cameras is already under development for use in land and marine applications, taking advantage of the ability of UV light to penetrate the air-water barrier and return useful reflectance image information which we will be exploring in the near future. 



As always the code is available on the GitHub page and is open for customisation and tinkering for each persons own needs: https://github.com/MuonRay/PythonScientificPlotting



Notes:

* see reference to this at pages 42-43 @ Numerical Python: A Practical Techniques Approach for Industry By Robert Johansson

Saturday, 13 June 2020

Drone NDVI Mapping with QGIS and Python Analysis Code




Code Link: https://github.com/MuonRay/QGISPython/blob/master/NDVIQGISWithMapLegend.py

In this video I showcase the creation and use of Near-Infrared (NIR) TIF orthomosaic datasets made using UAV (drone) photography and photogrammetry which can be analysed in QGIS using a Normalised Differential Vegetation Index (NDVI) processing in the Python console in QGIS. This is comparable to the analysis done using satellite imagery, such as Sentinel-2, however using a 4K NIR converted camera on a drone flying at a maximum height of 70 meters means that we can get up to 1cm pixel resolution on the ground, allowing for very accurate remote sensing of areas of interest.

QGIS itself a free geomatics software package with a lot of functionality with regards to creating custom code recipes for analysis of datasets acquired using Earth-monitoring satellites and/or drones. There are a lot of interesting add-ons for QGIS, going from simple codes that allow for a cursory editing of a dataset to improve contrast or in more complex applications such as the Orfeo toolbox for machine learning. QGIS, being opensource means that there is large online community of professionals who use its features in research and industry alike and regularly update the different applications of this impressive piece of software.

Scripts that run in QGIS are written in Python code with a particular syntax native to QGIS that allows it to call its image processing libraries, which work on TIFs with greater ease than Python standalone scripts alone would and do not create lossy conversions as experienced with Python standalone coding libraries when processing TIF data. There is also options to save the processed images with a defined dpi ratio to preserve the image quality when saving to PNG or JPEG files for viewing completed maps outside of the program.

I would highly recommend using QGIS in conjunction with drone imaging and I am eager to explore some of the more in-depth of its applications further, in particular with respect to the classification and potential of the Orfeo machine learning toolbox.

Tuesday, 9 June 2020

Fusing 3D Modelling with NDVI in Python + VisualSFM + Meshlab



Here is an exercise in 3D image processing I performed using Near-infrared images processed into colormap NDVI, allowing me to create a 3D model of plants for use in 3D plant monitoring/health classification.

Near-infrared (NIR) reflectance images as described before can carry information about the health of plants, with healthy plant tissue reflecting more strongly in NIR as well as Green.



The NDVI formula leverages the NIR channel with the visibly reflected light, in an NIR converted camera the Red channel becomes NIR and Blue and Green become the visible. The dual-bandpass filter chosen will separate the different color channels, in the case of the filter I use a separation between the blue and NIR regions of the spectrum.Thus In my Python code used to generate the NDVI, the blue channel is leveraged against NIR for more precise close range NDVI.


This was created using (1) custom Python code to process the NIR reflectance images into a graded NDVI temperature scale images and using a combination of (2) VisualSFM for point cloud and polygon generation and (3) Meshlab to tidy up and display the polygon file.


An RGB reconstruction was also performed on a collection of standard images (captured using a phone camera) of the collection of plants scanned for comparison.

The NDVI 3D model was not a perfect reconstruction however it was cleaner in general than the RGB model and considerably faster to process in VisualSFM after the Python code had processed the input RAW (DNG format) images. It is relatively easy to see the distinction between the healthy vegetation and the background environment, the wooden decking, the plants were placed upon in the NDVI 3D model. This has lead me to think that this technique could be further developed in machine vision of plants in an environment in 3D, especially if the 3D model can be converted into a movie using a program like CMPMVS which could then be plugged into a platform such as PyTorch or Tensorflow for use in plant health classification in 3D.In any case this was an interesting way to demonstrate the use of NDVI and NIR imaging as applied in novel applications in the field of 3D photogrammetry and modelling with the intent of creating datasets for future explorations in machine vision research.Code Available Here: https://github.com/MuonRay/PythonNDVI/blob/master/ndvibatch.py


Wednesday, 27 May 2020

Generating NDVI Drone Panoramas in Python Code




 I share the development of a project that uses near infrared drone imaging to create NDVI panorama mosaics in Python with all coding available on the GitHub Repository here:
https://github.com/MuonRay/PythonNDVI/blob/master/PanoramaNDVIRawInput.py

Panorama Stitching, as used in aerial/satellite imaging and space exploration, uses a point matching algorithm (i.e. SIFT) on images taken from a camera then applies a homography transform creating a mosaic composite image. 

Here Using a 4K NIR Drone Camera we can create HDR NDVI Panoramas using DNG files for input (a JPEG Version is also available).  




We have talked about NDVI and how to generate standalone NDVI Drone images in previous articles such as here: http://muonray.blogspot.com/2019/07/ndvi-vegetation-mapping-project-with.html and here: http://muonray.blogspot.com/2019/10/specialist-ndvi-filter-developments-for.html

DNG is the RAW Format used in DJI's Mavic 2 Pro drone and is a data rich file format that requires Python's rawpy library to decode into a form that can be worked with using OpenCV for general image editing generally and panorama stitching specifically in Python.

It is hoped that this coding project will be a less computing intense solution to generating NDVI diagnostics of the environment without the need for expensive or computationally intensive processing such as creating 3D orthomosaics or photogrammetry files which can be relatively more daunting to produce and work with as compared to a panorama. 



Essentially any images acquired using a Near-Infrared (NIR) converted camera can be used to generate a modified Normalized Differential Vegetation Index (NDVI). The coding contains standalone colorbar legend and is a batch processing version that works with all DNG files in the working directory. ENDVI and SAVI Indexes also available and with greyscale options. in the larger Python NDVI GitHub project folder. I also encourage open modification and tinkering of this projects code for improving this field of exploration and environmental examination.


I have also included a selection of custom developed python codes for use in various drone imaging applications, such as batch conversion of DNG (RAW) drone images to JPEG or PNG, use of the rawpy library features of demosaicing, gamma factor correction and use of skimage library to demonstrate histogram equalization in colour images to create better contrast and depth. This repository also increases coding developed for use in generating panorama composite images both in JPG and DNG format, a very useful technique in high definition aerial imagery. These codes are open for use in educational and demonstration uses and for non-profit organisations.

See here:https://github.com/MuonRay/Drone-Image-Editing-in-Python-Coding-Repository

Wednesday, 29 April 2020

Using Adobe Bridge with Camera Raw for Drone Panorama Editing Applications





Here is a short video showing how I use Camera Raw in Adobe Bridge to generate Drone Panorama images quickly and easily with or without opening Photoshop. The advantages processing in Adobe Bridge offers is that it can improve the speed and efficiency of stitching together large images in panorama without slowing down your computer significantly.

Photoshop, like all programs, uses up some of your computer's resources while it's open. Even if you're not working in Photoshop at the time, as long as it's open in the background, it's still using up resources. If you're working on a slower computer to begin with, having programs open in the background that you're not using can slow you down even more.

Camera Raw offers such a complete image editing environment that it's entirely possible to do everything you need to do with your photo in Camera Raw without ever needing to open it in Photoshop for further editing, including cropping, aligning, contrast and color channel editing. Camera Raw is perfectly capable of running in Bridge itself, or another way to put it, Camera Raw can be "hosted" by Bridge, just like it can be hosted by Photoshop.

Another benefit to running Camera Raw from Bridge, and one that has an impact on your workflow, is that when you're finished processing one image in Camera Raw and click the Done button to close out of it, you're instantly returned to Bridge, ready to select and open the next image or sets of images for further processing.



Thursday, 26 March 2020

Coronavirus Emergency In Ireland - Why The Government Must Mandate a 2 Week Total Shutdown



This is an urgent message, so lets not waste time with introductions.

The Government of Ireland must mandate a Complete Shutdown of all non-essential and non-medical business immediately. 

This means closing all non-essential manufacturing, factories, construction sites, call centers and non-essential warehouse work (excepting medical device, food services for humans and animals, pharmaceuticals and other essentials) - Private and Multinational firms that are still continuing business as usual in Ireland MUST BE CLOSED - They must not be allowed to continue work as they dictate until they chose to close. WE ARE IN A NATIONAL CRISIS AND THEY MUST BE MANDATED TO CLOSE. 

I RECOMMEND THAT AS MANY PEOPLE AS POSSIBLE USE THIS POST AS A PLATFORM TO NAME AND SHAME PRIVATE COMPANIES IN THE REPUBLIC OF IRELAND THAT ARE REFUSING TO CLOSE AND PUTTING THE LIVES OF OUR CITIZENRY AT RISK AND ARE MORTGAGING THE FUTURE OF LIFE AS WE KNOW IT FOR SHORT-TERM PROFIT.

The numbers of people out on the streets, parks, beaches and other public environs is also alarming. In the UK it is now illegal to be out during a nationwide lockdown while in Ireland we have still not made measures to ENFORCE SOCIAL DISTANCING THAT CAN SAVE HUNDREDS IF NOT THOUSANDS OF LIVES. 




Despite many euphemisms and mitigating measures the government has taken they have been half-measures at best and will do nothing to FLATTEN THE INFECTION CURVE. WE MUST MOVE PAST HALF-ASSED MITIGATION AND HAVE A FULL LOCKDOWN, APPLY "THE HAMMER" - Only then can we move onto "THE DANCE" AND CAREFULLY RESTART OUR COUNTRY AGAIN.

The key word here is MITIGATING MEASURES. the government refuse to take any hardline measures to, in-effect, flatted the infection from an exponential rate to a linear rate and eventually reaching a level that will not overwhelm our already beleaguered healthcare system. 


We have healthcare workers who can only be considered superheroes at this point who are quickly becoming so overwhelmed that they are, in-effect, samurai in nurses and doctors clothing. They are risking their own lives and also having to take the extra burden and risk because, again PRIVATE BUSINESSES REFUSE TO CLOSE ON THEIR OWN! THEY MUST BE FORCED RIGHT NOW!

TIME IS KEY IN ALL OF THIS. WE MUST GET OUR COUNTRY FROM AN EXPONENTIAL STAGE OF INFECTION RATE TO A LINEAR STAGE AND FLATTEN THE CURVE. WE CAN ONLY DO THIS WITH A FULL LOCKDOWN FOR AT LEAST 2 WEEKS!

To citizens I recommend you STAY INDOORS AS LONG AS POSSIBLE, we can only fight this virus and stop it becoming a permanent part of the microbial ecosystem in this country if we band together on this and most importantly HOLD BUSINESS AND GOVERNMENT TO ACCOUNT! DON'T FORGET THAT THEY SHOULD HAVE HAD A WUHAN OR RUSSIAN STYLE LOCKDOWN AT LEAST A WEEK AGO IF NOT TWO!

THE DATA DOES NOT LIE - The John's Hopkins Website has been providing more up to date information than any of the highly funded and subsidized quangos and institutions in IRELAND. The most important detail is to read the shape of the graph, not merely quote numbers.

This is Our Status (March 26th 2020):




WE ARE HEADING INTO EXPONENTIAL GROWTH. WE MUST ACT NOW!

Compare this to Italy, the Worst Country Hit in Europe:




EXPONENTIAL INFECTION RATE HAS CAUSED THE VAST NUMBER OF DEATHS WITHIN ITALY.

SEE John's Hopkins COVID 19 Updates live here: reading the curve is far more essential than the individual numbers: https://coronavirus.jhu.edu/map.html

The Ideal is to APPLY THE HAMMER, initiate a 2 week controlled lockdown and flatten the infection rate, as CHINA and most East Asian Countries have done:




WE HAVE BEEN TOLD CONSTANTLY BY THE IRISH GOVERNMENT AND THE MAINSTREAM MEDIA (RTE Are most culpable) THAT THE WORD "LOCKDOWN" IS A SCARE TACTIC. WELL, IN TIMES LIKE THIS FEAR WORKS TO GET THE JOB DONE. WE DON'T HAVE TO BE AFRAID FOREVER BUT THIS CRISIS REQUIRES US TO REALLY BE AFRAID FOR OUR LOVED ONES AND FOR OURSELVES.

Morever, it has not been the word "lockdown" that the citizenry have had a hard time understanding: The term "Social distancing" has been apparently interpreted by the public with a mixture of confusion and reckless abandon, maybe because it was written by the same second-rate advertising gurus that write the script our politicians read. 

As the Comedian George Carlin once said "Euphemisms are bullshit, people need direct honest language" - So, my fellows, please STAY INDOORS FOR AS LONG AS YOU HAVE FOOD AND MEDICINE! 




Again, to the Government, INITIATE A CLOSURE OF NON-ESSENTIAL PRIVATE BUSINESSES TO ENSURE THAT WE FLATTEN THE INFECTION RATE! YOU WILL BE HELD ACCOUNT FOR THIS NEGLIGENCE!

We can hopefully look back on this moment and perhaps laugh and have a kind of "I was there" moment with the kids in the future. Perhaps. But if we do not act now we are going to plunge this country into loss and sorrow that will take generations to heal. Please do what is in the common good.

IMPORTANT NOTE:

As I write this America has overtaken every other country as the most heavily infected. this was not unexpected given the delayed response there however they are due to get much worse due to the high level of exponential growth leading to a great strain on their healthcare system: