Welcome to NIR PAT Basics, a blog post series where we discuss the fundamental concepts needed for success in NIR PAT.
You may have heard of, or even used, spectral preprocessing methods to turn NIR spectra into information, like blend potency or particle size. There’s a lot of spectral preprocessings available in the literature and in most data processing packages. Let’s talk about the two most preprocessing methods used in powder processing applications and why they’re so popular.
One of the most common spectral preprocessing methods is taking a derivative. This spectral preprocessing is typically used for two reasons: it can highlight changes in peak shoulders or subtle peak shape changes that may not be obvious in a raw NIR spectrum, which often consists for very broad peaks.
Taking a derivative can also remove spectral offsets and some baseline curvature, depending on what type of derivative is taken. This can is useful in making an NIR PAT method robust to changing instruments, small differences from installations of probes, and small intensity changes due to variability in NIR illumination source intensity.
In practice, a simple derivative is not often used. Rather, a derivative is usually combined with Savitzky-Golay smoothing. A Savitzky-Golay filter smooths each data point with the weighted average of neighboring data points, with data points closer to the data point to be smoothed given the most weight. Smoothing is useful as taking a derivative can result in a very noisy preprocessed spectrum. Find an example of NIR spectra before and after taking a Savitzky-Golay smoothing derivative below:
Another common spectral preprocessing technique in the analysis of powders is the standard normal variate transform. This transformation is applied by subtracting the mean intensity value of a spectrum from every data point and then dividing by the standard deviation of intensity values of the spectrum.
The SNV transform is usually used in diffuse reflectance analysis of powder samples to minimize the impact of changing particle size distribution and bulk density on the NIR spectrum. NIR spectra of powders are sensitive to both chemical and physical changes of the powder under investigation. If the powder’s bulk density or particle size distribution changes, the NIR spectral response may change. This is due to the diffuse reflectance measurement mode; changing the bulk density or particle size distribution changes the functional pathlength of the measurement or a wavelength-by-wavelength basis. The SNV transform minimizes this effect, which is useful when trying to establish a linear relationship between spectral response and concentration as is the case with methods like partial least squares (PLS).
Find an example of NIR spectra before and after taking a SNV transform below:
That said, if you want to derive some information about physical characteristics, using an SNV transform may not be very useful. Knowing why spectral preprocessing methods are used is critical to success of an NIR PAT method!